Welcome to the forums. It's a possibility you could damage the phone's USB port, but unlikely unless there's a lot of force involved or it happens repeatedly. Usually it just pulls the cable out of the phone or charger. You're more likely to damage the cable itself doing something like that, but they wear out over time anyway and I suggest always having one or two spares for when that eventually happens.