On the Tab Pro, I can go to Settings>Device>Sound>Samsung applications, and select an app like Email. There is an option to select a different notification tone there. I assume the menu will be similar on the Note 4, but there might be some variation.
For non-Samsung apps, open the app and tap Menu, then check the app settings. If you can select a specific notification tone for that app, that option should be in the app's settings.