It's difficult to isolate the culprit just using the safe mode, specially with a Samsung device. In other phones you can enable one by one third party apps while on safe mode, making a bit easier the search, but that's not possible with Samsung.
You need to check the battery stats after the drain and see if some app shows an unusual percentage. A battery monitor like Gsam can give you more details than the stock stats.