Quietvice, I also had the same issue where I had a new voicemail notification that wouldn't go away. It started happening only after I upgraded to 4.4.2. I tried everything, rebooting, clearing the cache, force stopping, but the notification would always return. Then, after I just couldn't take it anymore, I installed the Verizon "Software Upgrade Assistant" software. For me, every time I connected my phone to my PC I would get a pop up requesting me to install the software but I never did until now. I ran the "Repair Assistant", it took about 15 minutes to complete, but in the end it got rid of the notification. There are some cons though, the "repair" basically formatted the phone to factory settings. All accounts were wiped and all apps were deleted. However, after restarting the phone, all apps were then re-downloaded automatically but all account information was lost. I would suggest being connected to Wi-Fi for this process because of the app re-download, they were not re-loaded from a back-up file.
Anyways, hope this helps. Anyone not on VZW, I've got nothing.
Carrier: VZW
Device: Galaxy S4
Android: 4.4.2
Computer: PC