You are absolutely correct. Per the contract a Verizon customer signs with Verizon upon signing up for service dictate that Verizon may not terminate services unless for very specific reasons (i.e. non-payment or manipulation of services with malicious intent). Complaining is most certainly not in that specific list.
Furthermore, even if your contract had expired, there is a term in the contract stating that all previous contract terms apply even after contract expiration, except the early termination fee, of course.
I have previously had acquaintances tell me that they would not pay their cell phone bill until they got a working phone. I personally advise against it without consulting an attorney first, or you could potentially end up on the wrong end of a collection agency. This demand for "working phone before bill payment" is not a statute covered by contract with Verizon Wireless' ToS, and since Verizon did not agree to that at the time you signed a contract, it is not substantiated. However, you can, through proper channels, get out of your contract without an early termination fee for having a "lemon" phone, depending on your region's lemon laws. Those laws, however, do provide that Verizon make a best effort to resolve it (i.e. provide a *like-model* alternative at no charge, which Verizon actually will go beyond the standard provisioning as a good-faith measure) before it gets to that, so you have to be prepared for some patience.