It looks to be a Samsung camera app issue rather than a hardware one. I use Vignette and it keeps the image in portrait when taken that way (my guess is other camera apps will do the same but I haven't tested). The stock app seems to prefer landscape regardless of how you hold it (like how videos are always treated in landscape).
I use Go SMS and when i send pics via GoSMS They always attatch sideways and the person i send them to they get them sidways, is there a way for them not to attatch/come out like that?
I have encountered this same issue. I use Go SMS also. When I want to send a picture, I open up Go SMS, press the MMS button on the bottom right, press pictures, and then choose which picture I would like to send. It will give you the option of rotating the picture (plus add a filter to it if you like) before you send it.
Not exactly a fix for the issue but it does keep from sending the picture sideways.