Yeah, if you turn off the "Pointer" tracking in the settings and they still show up then something is wrong with the phone. This is the only thing that should pop up when you hover the S Pen over the screen.
(I tried posting a screenshot but it won't let me for some reason. Check under...