[bugs now fixed!] LG kernel source code for Optimus V is broken

[picasticks]

Update: LG addressed the issue and posted fixed source code on 4/12. You will notice "LGVM670(Thunder) Android Froyo / kernel bugs were fixed" now if you go to download the source. Personally, I think the problems with the source were accidental (this seems obvious from the code) and their response within 1 business day was great. Anyway, I have compiled my kernel for the V and will post it now!

-------------------- -------------------- --------------------

Hi Optimus V people!

I have an Optimus S and have been doing kernel-hacking on it for the last month or two, and have an OC kernel out on the Optimus S forum, called picasticks. Anyway, my friend got a V so I downloaded the official VM670 source from the LG site so I could build the kernel for him.

Unfortunately, the VM670 kernel source code is broken. I've detailed the compile failures and workarounds in this blog post and will be contacting LG. So, if anybody has had any contacts with LG re: source code or GPL compliance, please reply or PM me as I'd appreciate any advice or contact info.

Basically, they've run all the code through a scrubber to remove code comments, and in the process broken the C code in two files (that I know of). Plus, a bunch of the Linux netfilter code and headers are mysteriously missing. I've fixed the two files, restored the missing 15 files from the LS670 source, and built my kernel, but the larger point is that LG should be releasing working source code, dammit! The fact that it's fixable now by merging LS670 code won't help if the next LS670 release is also borked.

I haven't looked at the Froyo OS code, but it's also possibly broken if they're stripping comments.

Anyway, I thought I'd post here to see if anyone had contacts/advice, had already complained about this, or anything else. If not then hopefully my bug list and workarounds will be helpful to someone.

EDIT: I want to be clear that LG did not break the kernel source code on purpose! They just packaged the code archive incompetently. This is nothing to flame them about. I *really* meant this post as a "Hello Optimus V people, here's some helpful info, please let me know if you know anybody relevant at LG" and not to recruit you for a holy war.

 

[BiafraRepublic]

Believe me, you're not the only one frustrated over LG's source code.

Is there any chance you can post your kernel sources somewhere?

Also, what toolchain are you using?

 

[obijohn]

Correct me if I'm wrong, but if they obviously didn't use this source to compile the kernel themselves, then they are not complying with the GPL in releasing this "scrubbed" version, regardless of whether or not anyone else can get the source to compile.

Like you said, it may be as innocent as an automated comment scrubber. But, on the other hand, maybe there is more to it. It's interesting that the missing files are all related to netfilter. Perhaps they used proprietary code for the V from somewhere, maybe a hardware vendor? Who knows.

Giving them the benefit of the doubt and calling it just a mistake... well, I still don't like it. I can understand someone in management thinking that "comments aren't code", but as a developer myself I can emphatically say that yes, dammit, comments are (admittedly not always) essential to the code. Especially if they explain what the heck some arcane bit of inline assembler is supposed to do. Or management could have thought that the comments might be embarrassing (bad words and such) or contain something else they just didn't want the public to see, like maybe referring to a product that hasn't been released yet. But again, for every bad word you strip out with a comment scrubber, you strip out some essential comment explaining the code.

But I can probably guarantee that we will never see the corrected source from them. The best we can hope for is that they don't do the same when they release the 2.3 source (if they ever actually push a 2.3 update for the V).

 

[asadullah]

I'm glad it wasn't just me. I recently got into messing with kernels. I successfully compiled the hero and metro optimus but could not for the life of me get the V's kernel to compile. I hope it was a mistake on there part and not intentional.

 

[mmarz]

I just don't understand what these companies are afraid of? If anything, releasing the source allows their hardware to have better support by a free community of programmers. If I have already bought the phone from them legally, what do they have to lose by letting me turn it into a hotspot or overclocking it?

It's not like they are offering premium firmware upgrades for a profit. If anything, they are constantly scrambling to fix security holes so that they don't get sued. Well, congrats LG. Now you'll get sued for not releasing the correct source. Enjoy!

 

[obijohn]

I've sent a violation notice to the FSF. I'll post back if I hear anything back.

 

[whyaphoenix]

delete

 

[overthinkingme]

I just don't understand what these companies are afraid of? If anything, releasing the source allows their hardware to have better support by a free community of programmers. If I have already bought the phone from them legally, what do they have to lose by letting me turn it into a hotspot or overclocking it?

It's not like they are offering premium firmware upgrades for a profit. If anything, they are constantly scrambling to fix security holes so that they don't get sued. Well, congrats LG. Now you'll get sued for not releasing the correct source. Enjoy!

They'll lose on the hot spot. People tend to abuse. Otherwise, no reason to not support the community.


Sent from my VM670 using Tapatalk

 

[obijohn]

Ok, I've done some comparing of the VM670 kernel source with that of the LS670. It appears that somebody on the V team didn't like the fact that their lge.com email address would be in the source, so all lines with comments that have an LG user id or email are stripped. Unfortunately, in a couple of places the comment is on the same line as actual code, so that code is stripped too. Also unfortunately, it looks like they were over-zealous and deleted an extra line here or there immediately above or below these comments.

It looks like the only thing they are testing for in the scanning is names contained within brackets in a comment, or an lge.com email address in a comment. If it's found, that line is gone.

This is just ******ed[*]. If they are going to that much trouble to get rid of LG names and email address, why don't they, oh, I don't know, just get rid of the names and email address and not the entire dang line???

My best guess is they hired an intern to manually search the files for the names/emails to delete them. This intern then selected and deleted each whole line thinking it was the quickest way to do. Idiot. Probably spent a couple of hours on this job and broke the source in the process. Could have just done a dead simple search-and-replace-with-space scheme and finished in 5 minutes.

[*]Edit: Evidently, the censor thinks me using the word ******ed is somehow wrong. As Michael Scott would say, it's only wrong to call a ******ed person ******ed.

 

[picasticks]

Correct me if I'm wrong, but if they obviously didn't use this source to compile the kernel themselves, then they are not complying with the GPL in releasing this "scrubbed" version, regardless of whether or not anyone else can get the source to compile.

You are right, and if I get no response from them we should definitely go the GPL route, but I would like to give LG a chance first. I totally appreciate you're enthusiasm in reporting them to FSF but I would like to give LG a try on their own. I will at least say that the LS670 source is totally complete, with a README explaining how to build it, and works perfectly. If nothing else I am just curious to find out what LG will do. If they'll be our friends or not give a crap until we sic someone on them. (Also btw, the person to report them to is the copyright holder, not the FSF itself, so in this case Linux or Linus).

Like you said, it may be as innocent as an automated comment scrubber. But, on the other hand, maybe there is more to it. It's interesting that the missing files are all related to netfilter. Perhaps they used proprietary code for the V from somewhere, maybe a hardware vendor? Who knows.

Giving them the benefit of the doubt and calling it just a mistake... well, I still don't like it. I can understand someone in management thinking that "comments aren't code", but as a developer myself I can emphatically say that yes, dammit, comments are (admittedly not always) essential to the code. Especially if they explain what the heck some arcane bit of inline assembler is supposed to do. Or management could have thought that the comments might be embarrassing (bad words and such) or contain something else they just didn't want the public to see, like maybe referring to a product that hasn't been released yet. But again, for every bad word you strip out with a comment scrubber, you strip out some essential comment explaining the code.

The comments they're scrubbing are just "LGE change: developername@lge.com" ... so they are useful to us because they show LG changes. But, I can also see why they'd want to keep their developers' email addresses out of public code. IANAL so I don't know the technicalities of that, I read the GPL v2 (which covers Linux) but couldn't tell. Mainly, I would just like them to ship working source code first and foremost.

The reality is that anyone can always choose to scrub comments and no one will know unless they screw up as in this case.

But I can probably guarantee that we will never see the corrected source from them. The best we can hope for is that they don't do the same when they release the 2.3 source (if they ever actually push a 2.3 update for the V).

If I get no response, we can definitely go to Linux copyright holders over this. LG is a big company and even though this violation is minor and possibly accidental I would guess that Linux wants to set a precedent with big companies like LG that they have to follow the rules. Google is going after handset makers right now over fragmentation issues, and private forks are part of that problem. Maybe I am overly optimistic but I think there's hope.

 

[mmarz]

Ok, I've done some comparing of the VM670 kernel source with that of the LS670. It appears that somebody on the V team didn't like the fact that their lge.com email address would be in the source, so all lines with comments that have an LG user id or email are stripped. Unfortunately, in a couple of places the comment is on the same line as actual code, so that code is stripped too. Also unfortunately, it looks like they were over-zealous and deleted an extra line here or there immediately above or below these comments.

It looks like the only thing they are testing for in the scanning is names contained within brackets in a comment, or an lge.com email address in a comment. If it's found, that line is gone.

This is just ******ed[*]. If they are going to that much trouble to get rid of LG names and email address, why don't they, oh, I don't know, just get rid of the names and email address and not the entire dang line???

My best guess is they hired an intern to manually search the files for the names/emails to delete them. This intern then selected and deleted each whole line thinking it was the quickest way to do. Idiot. Probably spent a couple of hours on this job and broke the source in the process. Could have just done a dead simple search-and-replace-with-space scheme and finished in 5 minutes.

[*]Edit: Evidently, the censor thinks me using the word ******ed is somehow wrong. As Michael Scott would say, it's only wrong to call a ******ed person ******ed.

NM, looks like it's been done: http://lyncd.com/2011/04/bugs-lg-kernel-source-optimus/

 

[picasticks]

Is there any chance you can post your kernel sources somewhere?

Also, what toolchain are you using?

Sure, it'll take me a while to package it up and undo some patches but I'll edit this post and put a link in when it's done.

The usual open source "no warranty on this code" applies extra, because while my kernel built, I don't have a V to test it on so there may be other problems. BTW if anyone wants to be a guinea pig and test picasticks kernel on their V I can post the .zip to flash.

If you look at the blog post, it explains how to fix the source. There are 2 files that have to be fixed. For each file, there's a patch that fixes the VM670 file (there's also a vd_to_vm670 patch to illustrate the comment-stripping, but you don't want those). The first one just inserts a missing } and the second one just inserts 1 line of missing code.

Apply those patches and replace the 15 listed missing files from the LS670 source, and that should do it. If you look at the list of missing files, you'll see there are 2 directories you could just copy over to save time. Also you will need to delete the directory include/asm (the symlink include/asm is created during compile).

Toolchain: Originally I was using the standard Google Android toolchain from the froyo version of the platform, which is what the LG kernel is compiled with. Currently I'm using the CodeSourcery G++ Lite toolchain, which has newer GCC (4.5.1 vs. 4.4.0) and produces slightly smaller binaries. And possibly infinitesimally faster. I have had zero issues with either toolchain.

Update: I promised kernel source, and have a 66M tarball, so if anyone wants to host it, PM me. I just recompiled it to make sure it builds (but no promises that it's complete). Instead (and you may prefer this if you already have VM source) here is a single patch you can just apply to the tree:

cd kernel
patch -p1 ~/LGVM670_kernel-picasticks_patched_20110409.diff
rmdir include/asm

 

[picasticks]

NM, looks like it's been done: Bugs in LG kernel source code for Optimus V | lyncd.com

Yeah, I had the advantage of being an Optimus S guy, and knowing that the phones are identical except for 2 or 3 silly things like swapped buttons, so all I had to do was diff the LS670 and VM670 and what they had done was obvious.

But it totally sucks for anyone starting from the V sources, because you just get broken code.

FWIW, what I would do to build a "known good" VM source bundle is a little different. I would take the VM and LS sources and diff them with all comments stripped, and that way identify the 2 or 3 places where the code is actually different. Then, just copy those files from VM to LS.

In any case, I don't think we should waste too much energy trying to reconstruct their broken source, unless we get absolutely nowhere.

It is not like there is some ulterior motive on their part. I mean, they have *already released all the code* and there's nothing exotic or proprietary in it. They just f'ed up this V-specific bundle and need to fix it.

Sorry if I sound like I am defending LG. I promise I will be way more pissed off if they don't fix this.

What they ought to do is care more about the source release, because they get free testing from people like us, and we're the most outspoken, enthusiast part of their customer base. They ought to be building from the source they actually release.

 

[obijohn]

replace the 15 listed missing files from the LS670 source, and that should do it. If you look at the list of missing files, you'll see there are 2 directories you could just copy over to save time.

I again did some more checking, and the blog post is slightly wrong on this point. The files aren't missing, they're just named with the wrong case. For example, include/linux/netfilter/xt_dscp.h is actually there, it's just named xt_DSCP.h. So just change the case (the contents are the same). The weird thing is some go the other way and have to be uppercased. I think two of the files he lists don't even have to be replaced.

I tried to send a message to LG from their open support site contact form, letting them know the code is borked and therefore violates the GPL. And I formally requested the unaltered source as is my right as an owner of the device. However, when I clicked "submit" I got a 404 error. Tried again, same result. Methinks someone doesn't want to actually be contacted.

I may just have a look through the LS670 source for those contact emails and hit a couple of them up for legit contact info on getting the correct source for the VM670.

 

[Eollie]

Reading this it kinda sounds like some people are saying LG intentionally fouled the V model source code. But how would that make sense? THey know the developer community is pretty damn smart, and somebody would eventually catch on right?
I mean its happened and people have went as public as they can with it. I kinda think picasticks is on the right track in it being a rtard mistake. And nobody caught it until it went public and then nobody wanted to accept blame for the foul ball.

 

[picasticks]

I again did some more checking, and the blog post is slightly wrong on this point. The files aren't missing, they're just named with the wrong case. For example, include/linux/netfilter/xt_dscp.h is actually there, it's just named xt_DSCP.h. So just change the case (the contents are the same). The weird thing is some go the other way and have to be uppercased. I think two of the files he lists don't even have to be replaced.

Thanks for the info, I'll update to remove that. What a bunch of goofballs, they're building on a case-insensitive FS?

EDIT: No, the files are missing. i.e. xt_dscp.h and xt_DSCP.h are not the same thing. But that at least explains how they could unintentionally omit them, by using a case-insensitive FS (which is totally dumb, obviously you couldn't build a kernel on such a machine with files overwritten and missing).

I tried to send a message to LG from their open support site contact form, letting them know the code is borked and therefore violates the GPL. And I formally requested the unaltered source as is my right as an owner of the device. However, when I clicked "submit" I got a 404 error. Tried again, same result. Methinks someone doesn't want to actually be contacted.

Can you please stop trying to contact people? I thought I made it clear that I was going to do this! I didn't post this info here so that everyone could flood them with complaints. That's the LAST thing we should do. It's very important to be friendly and there's no accountability if lots of people are contacting them, they have no idea who to respond to.

If they don't respond to a private inquiry the next correct step is to make a private report to Linux, so that they can make a private inquiry as the copyright holder who's actual rights have been violated. We have no legal standing in this matter.

 

[picasticks]

Reading this it kinda sounds like some people are saying LG intentionally fouled the V model source code. But how would that make sense? THey know the developer community is pretty damn smart, and somebody would eventually catch on right?

No, I don't think they intentionally released bad code. They were trying to remove code comments and were otherwise sloppy in building the source tarball, which they clearly never tested.

Since they also didn't delete the netfilter code (props: obijohn), just renamed the files, it's even more obvious that this was unintentional.

All they did on purpose was strip the code comments with LGE developers' names. That is less than ideal but it is very different from releasing broken code.

I now kind of wish I could turn down the heat on this thread from 10 down to 2. I originally just meant it more as an FYI and "do you have any contacts at LG?"

 

[obijohn]

Can you please stop trying to contact people? I thought I made it clear that I was going to do this! I didn't post this info here so that everyone could flood them with complaints. That's the LAST thing we should do. It's very important to be friendly and there's no accountability if lots of people are contacting them, they have no idea who to respond to.

If they don't respond to a private inquiry the next correct step is to make a private report to Linux, so that they can make a private inquiry as the copyright holder who's actual rights have been violated. We have no legal standing in this matter.

I can see where you don't want to piss them off. But I didn't do it to harass them, I did it because as an owner of the device I have the right to the complete GPL source, and they haven't provided that and I had to explain why the download didn't suffice. Everyone who owns the device has that same right to request the source as that right is provided by the GPL. If a bunch of people are requesting the source from them because the download isn't correct working source, it will be in their interest to put up the corrected source and just point everyone to it (they don't have to send each person the source if it's available for download). If it's only one person, they might find it easier just to send that one person the source than to update their download servers. It's my experience that big companies like this don't do something unless they feel pressure from somewhere to do it.

Anyway, that was my thinking when I sent the request. And I still think it's perfectly fine for anyone else to request the source from them as well. I don't advocate harassing them with useless complaints, but a legitimate request is just that -- legitimate.

 

[picasticks]

I can see where you don't want to piss them off. But I didn't do it to harass them, I did it because as an owner of the device I have the right to the complete GPL source, and they haven't provided that and I had to explain why the download didn't suffice. Everyone who owns the device has that same right to request the source as that right is provided by the GPL. If a bunch of people are requesting the source from them because the download isn't correct working source, it will be in their interest to put up the corrected source and just point everyone to it (they don't have to send each person the source if it's available for download). If it's only one person, they might find it easier just to send that one person the source than to update their download servers. It's my experience that big companies like this don't do something unless they feel pressure from somewhere to do it.

Anyway, that was my thinking when I sent the request. And I still think it's perfectly fine for anyone else to request the source from them as well. I don't advocate harassing them with useless complaints, but a legitimate request is just that -- legitimate.

I know what you're thinking, but that's just now how it's done, you have to meet them halfway, and talk to the right person. If one person contacts them then there's accountability, either they respond or they don't and I can report exactly what they say. It's important to try to work with companies that are trying to comply in a good way (as LG is, they're not obligated to have public source-code download). More than that I thought I was clear that I was in the process of contacting LG and I think it was totally presumptuous to step on that without even asking.

Also, your idea of "rights" is wrong. The only person's rights that have been violated are the copyright holders of the Linux kernel (i.e. Linus and co.). We have no more standing as LG customers than anyone else. Any random person
can request source code (or an Aston-Martin) from LG, but if they refuse to provide code, they haven't caused a tort to you or me, they've only broken a contract they made with the Linux copyright holders when they made a product using their code. When the terms of that license are broken, only the owners of that code have legal standing to complain. LG responds to someone like me only as a courtesy.

Take a look at the GPL site, this is all explained.

 

[mmarz]

I know what you're thinking, but that's just now how it's done, you have to meet them halfway, and talk to the right person. If one person contacts them then there's accountability, either they respond or they don't and I can report exactly what they say. It's important to try to work with companies that are trying to comply in a good way (as LG is, they're not obligated to have public source-code download). More than that I thought I was clear that I was in the process of contacting LG and I think it was totally presumptuous to step on that without even asking.

Also, your idea of "rights" is wrong. The only person's rights that have been violated are the copyright holders of the Linux kernel (i.e. Linus and co.). We have no more standing as LG customers than anyone else. Any random person
can request source code (or an Aston-Martin) from LG, but if they refuse to provide code, they haven't caused a tort to you or me, they've only broken a contract they made with the Linux copyright holders when they made a product using their code. When the terms of that license are broken, only the owners of that code have legal standing to complain. LG responds to someone like me only as a courtesy.

Take a look at the GPL site, this is all explained.

Read the GPL that is on your phone. The one the copyright holders forced LG to provide to you with every purchase of the phone. Settings -> About -> Legal Information

LG provided this license because it is required to by the copyright holders, but it is an agreement between you and LG. They promise to provide working source code at your request. The tort comes in the form of a false sale to you. You bought the phone assuming you would have access to the source. They have broken their own license. It is equivalent to not honoring a warranty.