You are right that this won't harm your phone, but it will harm your battery. Li battery chemistry only allows so many full discharge cycles so every time you do this, you've just decreased your battery life by some amount. It is much better on the battery in terms of lifespan to keep it closer...