Welcome to Android Central! What do you mean by "turn on"? Do you mean that when you boot up the phone from a powered-off state, the first thing that shows up is the Phone dialer? Or do you mean when you wake the phone and unlock the screen?
If it's coming up when you unlock the screen, make sure you're swiping in the correct direction. I'm not sure exactly how it's set up on the Note 4, but on the Nexus 5, I need to swipe up. If I swipe from the left to right, it unlocks to the phone. Swiping from right to left unlocks to the camera.