Voicemail is very much a carrier-based feature, if you're accessing using a dial-in number. I suggest searching your carrier's website or calling customer service.
However, if you're getting "visual voicemail" though the dialer app on your phone, go there, tap the voicemail you wish to delete and then tap the trashcan that appears below it, this will delete the message.