I just got my unlocked Nexus 6P in the mail today and plan to go to AT&T tomorrow to have a SIM card put in. Can I go ahead and charge it?
Absolutely - at least overnight. Read
The Care and Feeding of Lithium Polymer Batteries.
If so, should the power be on or off
It shouldn't get turned on until the first time the battery is fully charged (even in the store to set it up - I make them set my phones up using the charger, or hand them to me in the sealed box. Which may be why the battery in my 2004 Motorola V551 is still about 98% good.)
and how long does it typically take to fully charge it?
From 40% - which is about as low as you should let a lithium battery get - not much more than an hour. But you can lave it plugged in all night. The charge stops when the battery is fully charged.
I'm concerned because there does not seem to be an external indicator on this phone that lights up when the device is fully charged.
After the first time, when the battery gets down to the recharge point (you should be able to get it down to 20% the first 3 charging cycles), you can leave the phone on so you can check the charge after an hour or so. But, as I said, leaving it on the charger all night (I think my record was about 35 days) won't hurt anything but your wallet. (If the charger is plugged into the wall it's using power.)