Only numbers entered with your Google account (edit the number and look at the first line) will be saved to Google Contacts. If you have some entered as Phone, Device or SIM, see the Contacts section at Backing up an Android Device to convert them - then save them.
Smart Switch will back up ringtones, contacts, messages and wifi settings, and that seems to be about it.