Finally I got it Gone. The smart remote is part of samsung watch on. Settings, applications manager, watch on and uninstall all the versions it would let me and it was GONE!
Open Smart remote app, and go to settings by pressing the three dots at the top right corner, scroll down and unchecked the Auto Display Remote Control on Lock Screen option.
Ohhhh thank you so much, that little button was driving me crazy! I don't even know how I turned it on but it's gone now thanks to you. Thank you again!