I almost was stuck in this dilemma a few days ago. I Love my Note 4 but was having problems with Verizon and left them with a Verizon Note 4. Then I had nothing but problems with it on AT$T. Everything worked, but hotspot and I kept dropping signal to nothing during calls. I was going to switch to a different phone on AT&T, because I didn't want to pay THEIR price for the Note 4. I ended up just trading it in and at least getting something off the price, but needed the Note 4. I use the S Pen a lot (suck at texting), and need the bigger phone for my older eyes. The Note 4 is actually the first phone I can see myself keeping a very long time and still be happy with it.
I was considering other phones, like S5 or S6, even a Motorola, but ended up back to the Note 4. Cost me a bit, but well worth it. I'll never go back to Verizon, they screwed and Lied to me One time too many to ever get any business from me again. So happy on AT&T with my New Note 4 and if I had to get something else, would have saved a little money and gone back to the Note 3. Iphones never will be an Option, toooooooo boring! LOL