Try logging into GV from a PC and go to Settings > Voice Settings and on the first tab (Phones) under the "Forwards to" section next to your mobile number you should have a link that says "Deactivate Google voicemail" .... try that.
Follow these steps to deactivate Google Voice on your mobile phone .
Enter the following exactly as written, as if you are dialing someone:
Press "Send" or "Call" on your phone to dial that number. The phone will appear to make a call and then hang up. When that is completed, you are done!