VZW software testing

[Maddog241]

I was wondering who does the testing of software builds for Verizon? Are the testers all Verizon employees? Do they have a sample of actual users who participate in the process?

With the MR1 fiasco that happened back in May, it seems that whatever testing process or testers they have in place are not very effective. There is no way that the MR1 should have passed with all the reboot issues it created and/or made worse.

Sign me up Verizon. I was a beta tester for Blackberry before joining the Android world. I will put my autograph on an NDA and I will put the software to the test... No way MR1 would have made it past me !

 

[Charmed Juan]

Moto has a group of "Soak" testers. they get the update first and if there aren't any problems, it continues to roll out. Not true testers, but close to what you want.

 

[FrankXS]

You know it's funny... I installed the MR1 update the day it came out. I immediately afterward drove a road trip of 5500 miles traversing 10 states, through numerous 3G/4G transitions and never had a signal or reboot issue. If I was a tester I would have said it was good.

A few weeks after that I spent a week in Mexico using my phone for voice and wireless data. Nary a problem there either.

It's really weird some of us have different experiences.

-Frank

Sent from my HTC ThunderBolt 4G/LTE using Tapatalk

 

[Maddog241]

Moto has a group of "Soak" testers. they get the update first and if there aren't any problems, it continues to roll out. Not true testers, but close to what you want.

I wonder if HTC does the same thing? I have the Thunderbolt and I would love to have the chance to help make it better than it is.

 

[robrecht]

They must have known something was amiss because after they first started rolling out MR1 they halted it for a couple of weeks before starting it up again. Heck, even I knew there was an issue within 24 hours. Giving them the benefit of the doubt, I figure they estimated that MR1 would fix more problems than it caused while a better fix was developed. I think that was the point where they twice misunderestimated, both the extent of the problems they would cause with MR1 and how long it would take to develop a new, better fix, perhaps because they did not yet understand how their own eHRPD network was implicated in the difficulties. Recall that the eHRPD outage was on April 27th, shortly after MR1 was leaked, and just two days before MR2 was announced on Verizon's website. Could be different parts of the company not yet talking to each other as they're just getting up to speed on their new network tech and their only phone designed to interface with this eHRPD network. Total guesswork on my part, but given Verizon's silence on the subject, we gotta start guessing somewhere.

Thanks, Robrecht

 

[Maddog241]

Guessing indeed is our only real option. Big Red is not one to communicate openly with its customers. Sometimes, it seems as you pointed out, that there is a lack of internal communication there as well.

I'm still wondering if we will see anything this week from them since there has been no leak.

 

[FrankXS]

They must have known something was amiss because after they first started rolling out MR1 they halted it for a couple of weeks before starting it up again

Because of all the varying experiences I'm beginning to entertain the thoughts that the primary issue might have been the installation process itself. IOW, the patch was never installed properly to begin with. Maybe due to programming glitches, maybe due to hardware issues.

I doubt that you're old enough to remember this, but... as an IT guy I remember a time when flashing anything was a somewhat risky procedure. The risk was that back then they had not perfected flash memory. There was no such thing as a "flash drive". The point is that the technology used to hold data in a PCs BIOS and Modems (i.e. 300bps/1200bps era) was often defective. The factory would load the BIOS/Modem/Non-Volatile memory (whatever you want to call it) and if it succeeded it was out the door. If it failed the non-volatile memory (flash memory) was replaced until it succeeded. Then it was out the door. However, often, when a "flash update" was released, it was larger than the original firmware, taking up additional (non-tested) space on the non-volatile memory chip (EPROM- erasable programmable read only memory). If the chip had a defective spot in it, it would fail. Then, back to the factory it had to go. Warnings to this effect usually accompanied the "update instructions". This theory would also account for VZW saying to "just replace the unit" for these reboots.

Bottom line... "what-if" some of the TBolts had some bad EPROM memory?

Another theory... the installation program itself, like virtually all programs today have what is called "error checking" as part of the code. For example: "IF ABC succeeds, DO XYZ, ELSE PRINT Error occurred". "What-if" the TBolts MR1 update had some bad installation code missing some errors? It may write some of the data but not all. Then, when a certain set of circumstances were true (i.e. sequence of events Wifi/3G/4G, Video, etc.) when the system needed that code that was not there - BOOM, Reboot.

Anyway, my only point is that this whole issue and people's experiences has been so varied it does kinda sound like it could have been some unpredictable non-repeatable circumstance like this.

We'll probaby never know... just thinking out loud I guess

-Frank

 

[Maddog241]

Because of all the varying experiences I'm beginning to entertain the thoughts that the primary issue might have been the installation process itself. IOW, the patch was never installed properly to begin with. Maybe due to programming glitches, maybe due to hardware issues.

I doubt that you're old enough to remember this, but... as an IT guy I remember a time when flashing anything was a somewhat risky procedure. The risk was that back then they had not perfected flash memory. There was no such thing as a "flash drive". The point is that the technology used to hold data in a PCs BIOS and Modems (i.e. 300bps/1200bps era) was often defective. The factory would load the BIOS/Modem/Non-Volatile memory (whatever you want to call it) and if it succeeded it was out the door. If it failed the non-volatile memory (flash memory) was replaced until it succeeded. Then it was out the door. However, often, when a "flash update" was released, it was larger than the original firmware, taking up additional (non-tested) space on the non-volatile memory chip (EPROM- erasable programmable read only memory). If the chip had a defective spot in it, it would fail. Then, back to the factory it had to go. Warnings to this effect usually accompanied the "update instructions". This theory would also account for VZW saying to "just replace the unit" for these reboots.

Bottom line... "what-if" some of the TBolts had some bad EPROM memory?

Another theory... the installation program itself, like virtually all programs today have what is called "error checking" as part of the code. For example: "IF ABC succeeds, DO XYZ, ELSE PRINT Error occurred". "What-if" the TBolts MR1 update had some bad installation code missing some errors? It may write some of the data but not all. Then, when a certain set of circumstances were true (i.e. sequence of events Wifi/3G/4G, Video, etc.) when the system needed that code that was not there - BOOM, Reboot.

Anyway, my only point is that this whole issue and people's experiences has been so varied it does kinda sound like it could have been some unpredictable non-repeatable circumstance like this.

We'll probaby never know... just thinking out loud I guess

-Frank

Interesting ideas. I think the first one is very plausible that there may be some quantity with bad hardware. And yes I am plenty old enough to remember back to the days of punch cards and the first Apples.

If there was some mistake in the code of the update itself, would that not affect all who installed it? I was trying to determine how some people installed it. I remember that some had to force their phones into 1X in order to get the update. I was wondering if the actual download method might have somehow affected the outcome since it was done in different manners. I really don't know, and like you said, we probably never will know the true answer for sure.

 

[robrecht]

... I doubt that you're old enough to remember this, but... as an IT guy I remember a time when ...

Thanks, but that is just hilarious. I was trained on a mainframe using FORTRAN by an old aeronautical engineer who got interested in computers as a hobby after he retired. This guy was so old, he had been my father's engineering professor too, prior to World War II. In one company where I worked I was the first guy to use both email and then PowerPoint.

 

[FrankXS]

...I am plenty old enough to remember back to the days of punch cards and the first Apples.

I was trained on a mainframe using FORTRAN.

Thank you guys for making me feel so much better today

-Frank

 

[robrecht]

Thank you guys for making me feel so much better today

-Frank

You young whippersnapper!

 

[worwig]

Because of all the varying experiences I'm beginning to entertain the thoughts that the primary issue might have been the installation process itself. IOW, the patch was never installed properly to begin with. Maybe due to programming glitches, maybe due to hardware issues.

I doubt that you're old enough to remember this, but... as an IT guy I remember a time when flashing anything was a somewhat risky procedure. The risk was that back then they had not perfected flash memory. There was no such thing as a "flash drive". The point is that the technology used to hold data in a PCs BIOS and Modems (i.e. 300bps/1200bps era) was often defective. The factory would load the BIOS/Modem/Non-Volatile memory (whatever you want to call it) and if it succeeded it was out the door. If it failed the non-volatile memory (flash memory) was replaced until it succeeded. Then it was out the door. However, often, when a "flash update" was released, it was larger than the original firmware, taking up additional (non-tested) space on the non-volatile memory chip (EPROM- erasable programmable read only memory). If the chip had a defective spot in it, it would fail. Then, back to the factory it had to go. Warnings to this effect usually accompanied the "update instructions". This theory would also account for VZW saying to "just replace the unit" for these reboots.

Bottom line... "what-if" some of the TBolts had some bad EPROM memory?

Another theory... the installation program itself, like virtually all programs today have what is called "error checking" as part of the code. For example: "IF ABC succeeds, DO XYZ, ELSE PRINT Error occurred". "What-if" the TBolts MR1 update had some bad installation code missing some errors? It may write some of the data but not all. Then, when a certain set of circumstances were true (i.e. sequence of events Wifi/3G/4G, Video, etc.) when the system needed that code that was not there - BOOM, Reboot.

Anyway, my only point is that this whole issue and people's experiences has been so varied it does kinda sound like it could have been some unpredictable non-repeatable circumstance like this.

We'll probaby never know... just thinking out loud I guess

-Frank

A couple of points.

Back then, eproms had a very limited number of write cycles, so testing memory would basically wear out the memory significantly. Flash roms are limited in write cycles too, but not to a single digit number.

But the key is that flash memory has built in memory management. If it finds a bad location, it just marks it, redirects that location to another memory location, and moves on. They can't expect thousands of millions of locations to be perfect.

 

[FrankXS]

it, redirects that location to another memory location, and moves on. They can't expect thousands of millions of locations to be perfect.

Exactly. What if this function didn't work correctly in certain hardware?

Anyway, there are tons of possible examples. I just chose a couple. The point is, this could still easily be a hardware or Install pgm issue. That's all I was trying to get across.

Let's take the Install program (take mine, please! ). Anyway... from memory, Here is how my install went. Keep in mind that being in IT I sometimes do things differently than a typical user. Just due to my experience.

I got the message about the update and I chose OK (or do it now, or whatever). But, I was actually working on another project at the time so I didn't pay much attention during the operation (I was working on my Laptop with the phone about a foot away on its kickstand - point being that I gave the phone plenty of time to do whatever it was gonna do.). I figured any prompt that might be there would still be there when I looked at it.

I noticed that it was taking one heck of a long time, but, I DID have a solid 4G connection at the time so I was not worried about that (could have been an issue for some people. I doubt the install program would have liked losing a data connection during the install) - potential bomb #1.

Then, I noticed that my phone had rebooted. That took a very long time too. But, I was busy so I just waited - suppose someone got antsy and powered off/on early - potential bomb#2.

When I finally looked at it, it looked like it had finished. But there was no "you have successfully completed... etc. message on the screen, like I would have liked to see. So... based on my past experience, I just did a routine Restart anyway - perhaps this maneuver was necessary - potential bomb#3.

After the manual restart, the phone came up and all looked normal. I slid the screen lock off (I had let it sit until it went into sleep mode) and I noticed what looked like the exact same "you have an update ready do you want to proceed [OK ] [Detail] [Cancel]" (or similar). I chose OK and the text went away. Nothing happened. But, again, having experience in this sort of stuff on other devices I figured, hmm... weird. So, just for the heck of it, I manually rebooted again! What if that reboot was necessary, not just redundant? - if some folks didn't do it - potential bomb#4.

Now I have no idea if any of the things I did made any difference whatsoever. But I do know that my phone does not have the reboot issues. So, who's to say?

Could it be that the Install program itself was poorly written? I say maybe. It could be. Am I sure? Of course not.

Anyhow, just another example of a possible Install program issue that might account for the varying experiences.

-Frank