"Removable" just means that the phone has a battery that is easily replaced or removed. The back pops off easy and the battery sits in a cradle and is not hard-wired into the phone. This allows users to carry spare batteries if they needed to and also makes it easy to replace if the battery wears out over time.
Granted, most phones with 'non-removable' batteries can also have their batteries replaced, but it just takes a few more steps and the right tools to do the job. I've owned phones with either configuration and I don't have a preference either way... I've never owned a phone long enough for the battery to wear out nor have I ever bought spare batteries.