As NHBass39 said... it will shut off, no way around it.
But 2 Minutes? Seems a little long, I just timed this on 4.3.605.2, completely stock
From initial power press:
Lock screen displayed: 52 sec.
WiFi connected: 58 sec.
Voice/Text connected: 64 sec.
4G connected: 72 sec.
Boot complete (home screen widgets, live wallpaper, GSam Batt monitor running in notification bar): 81 sec.
I change batteries occasionally since I do not like the extended battery, quickest way to do it is use a task manager of some kind to kill all task (I don't know if this is necessary, but seems safer to me) then pull the battery (I don't bother to power off the phone), then replace battery and start powering up, replace back cover and case well it's booting... whole process is about 2 minutes from initially popping case off to ready to use. Have not found a way to shave off any more time or keep the phone from shutting off and I have tried everything I could think of to speed up the process.