If you post on the S6 Forum, expect to get lots of love for the S6!
I have an S6 edge and an iPhone 6 Plus and my wife has the Note 4 so I have experience with all of them. The S6 is a brilliant device. I think the best phone ever built to date. It's the PERFECT size for a phone...not too big, not too small. One handed operation is no problem. The screen is incredible, the fingerprint scanner amazing and TouchWiz is hugely improved.
Biggest advantages of the Note 4 or the iPhone 6 Plus is battery life. Some people on the Forums are getting nearly as good battery life on the S6 as their Note 3's, but the Note 4 is a step up again in battery life. My wife hasn't tweaked a single thing and easily gets all day battery life on her Note 4 with heavy use, and winds up the day with 50% battery still left over. But, of course, the Note 4 (and iPhone) are much bigger devices so that's the trade-off.
Note 4 will give you removable battery and a SD card slot, so if those things are important to you, then consider that. To me, the S6 represents the next generation in phones that don't need replaceable batteries (because battery life is sufficient) or SD Cards (because you can get up to 128 GB on the device).
Last HUGE advantages of the S6 are fast charging (it's truly remarkable) and build quality (best build quality of any phone I've ever seen). I say go for the S6!!
Either decision you make, though, is a good one. Both terrific devices, for sure!!!