There has to be some other factor at play. I have Verizon GN5. I have Advanced calling turned on (HD voice and VOLTE/video calls). I don't disable anything. I get very good battery life both on Wifi and LTE. I'm not sure what's going on, but it isn't affecting me.
And for the folks who say the HD calling isn't really noticeable, you must not have talked to anyone else on HD calling, because the difference in call quality is so great it's almost jarring. If you call a land line , or a cell on another carrier or without HD voice, it's just like another mobile call. Calling another person on Verizon with HD calling enabled is as good as - or better than - my landline.