Welcome to Android Central! Do you notice if there are problems with your Volume Down button? If it's stuck in a pressed state, then that will always force the phone to boot up into Safe Mode.
Also, are you sure it's Safe Mode, or could it be Factory Mode, like in this video?