I didn't know about this issue, since I came over from an HTC Thunderbolt I had for 2 1/2-3 years.  It's kind of strange that the S4 has this issue, considering everything else it can do, and seems to do well.  This feels like a basic issue that smartphones have handled for years.  All these new features I've been having so much fun with, then all of a sudden I find something that feels like a basic feature (at this point) is so wrong.
I had taken a bunch of pictures at a convention and when we go home and were going through them (DNLA) on the tv, they were all landscape, no matter how I had taken them.  This being the reason I ended up here.