you have to enter a 4 digit pin everytime you use it, so unless they have your pin and phone, they can't do anything. You should also get the multi-authenticator from google play store, so if someone does get your phone, they can't get into your google accounts.