Hitting the X only turns it off temporarily. It comes back the next time the lock screen appears.
Paul's solution works. But what if someone wants to keep the music app, and just not have it on the Lock screen? This does not seem possible, or they made it difficult to do. Under the "Lock...