I do not know if you are a male or female, however if you are a female you can always stick it in your sports.....top. That's what I normally do with mine. I do not run fast or gracefully though, more like a wounded moose lope. It's better than nothing.
Get a running belt/pouch. Not sure what they are called but I have one for the gym. It's not quite a "fanny pack" but similar. The one I have held my Note 2 and my Note 3 without issue and I didn't have to worry about dropping it.
I use a Seidio surface clip with a low profile case like the body glove. The clip on my side is perfect although I know I look like an ***** with a giant slab clipped to my shorts.. works great though.
The armband is best. I often change the music and an arm band works better for that. If you are doing situps, pushups. or any kind of weight training a belt would get in the way. For running and biking a fitness belt would be fine so long as you don't want to access the phone. Google Now/S-Voice would work but I have enough trouble when I'm not out of breath trying to get them to work.