One quick follow-up: I've found that if I open an exchange contact, save it, then delete it.... the delete is successful. It makes my daily routine a bit more annoying but so be it if there's no alternative.
Anyone know how to delete these exchange contacts en masse?