I don't think the issue is with the phone. No other solutions mentioned above worked, so I tried this below;
Click on the voicemail alert OR dial your voicemail number
Select voicemail settings
There will be an advance notifications switch off option
Switch it off!
I don't know why and how...