Literally just hit this problem. The app in question is not PIA itself, but the Android System App.
When you turn on the phone and are presented with the lock screen, you can partilly swipe the notification bar to the left to reveal a "settings" icon. If you select this, it will bring up the...