Just changing the IP isn't enough, you'd need to also rid the device of whatever was installed by the person tracking you. Also, your IP changes depending on what network (your home WiFi, public WiFi, the cellular network) the device is connected to. On top of that, on a WiFi network it's very possible the IP address the phone has is not public-facing and it would be difficult to get the public-facing address to change on a network that isn't yours.
If you need additional help or have more questions or details to share, please join the site so you can reply in this thread. See
this link for instructions on how to join Android Central.