To be certain, so that there's no confusion, in order for your device to be powered off, you would need to press and hold the sleep/wake button until the power off option appears, then you would have to physically choose the power off option, then the phone Powers down.
Now if you are just pushing the sleep-wake button on the side of the phone, until the screen goes dark, and it brings up no options to power down, then you are just in sleep mode, thus the device will still be able to function with notifications / alerts.