Swappa is a great place. Used note 4s for about $400. Plus you have 180 days of buyer protection through PayPal (to handle misrepresentation or fraud).
Buy a used note 4 in good condition. Put a new battery in it ($15) and it's as good as brand new.
I've bought 3 used note 4s on swappa and all have been exactly as described.
If your buying outright, then what was stated above is a good idea. If your on contract and eligible to upgrade then I suggest checking Best Buy daily. I got my Note 4 for a $1 on 2yr. Upgrade with Sprint. BB had it for a $1 on July 14th and July 24th.