I've owned a couple of Note 3s for over a year. I don't see anything that would make me spend the $200 difference you're facing to upgrade to a Note 4. (I wouldn't even spend $100 for that upgrade - the 3 is a good phone.)
My wife had to get a replacement from Verizon. It was a refurb, which is the same as a "certified like new" phone. It was a dumb phone, so it's sitting in a carton, but If I put a working battery in it now, it would come right up - and it's at least 5 years old. (My daily driver is an AT&T refurb - you get it over everyone's dead body - it goes to my grandson when I die if it's still worth using by then. I won't go out of the house without having it on my belt [in an Otterbox Defender].)
Look at the differences between them (in the store, where you can play with working ones). If there's something in the 4 that you absolutely must have and are willing to spend another $200 to get, go with the 4. Otherwise, I think you'll be happy with the 3.
(If it's the difference in versions of Android, ignore them - if they don't have Lollipop on the 3, they will - if you like bugs. [I may update to 6, but unless they do a MAJOR rework on 5 - to the point that it should be called 6 - I don't want it.])