I don't know about the Samsung Messages, but I was having this problem in Google Messages. I could not set a notification tone for Messages that was different from the system default. In the obvious place, Messages -> Three dots -> Settings -> Notifications, there is a setting for Sound, but it doesn't change the sound. Looking at the top of that page, it says, "Other notifications," i think meaning other sounds within Messages. I DID find a way to set a non-default sound for an individual conversation. Two ways actually. But it appears to set it for all conversations I have set for each separately.
Two ways: 1) Open Google Messages, go to the conversation list. Select the conversation you want to change the sound for. Then tap the three dots menu -> People & Options -> Notifications -> Sound and pick the sound you want. 2) Go to System Settings -> Apps -> (Google) Messages -> Notifications (under App Settings). There will be a list of conversations with toggles. You can turn on or off sounds individual conversations by tapping the toggle. If you tap the name of the conversation, the Sound, you can pick then sound for that conversation.
Hope this helps!