Welcome to Android Central! Which HTC phone? Do you really have a Google Nexus One, or is it something different?
Use a can of compressed air to blow out the jack, in case there's debris in there. If the jack is completely clear (shine a light in there to look) and the problem persists, then the jack is probably faulty and needs to be replaced.