So are all of your contacts associated with the Phone account, rather than your Google account? If you're not sure, you can select any contact in the People app and edit it, and the account it's associated with is shown at the top of the contact info. Are you sure those duplicate contacts aren't in your People app as well? In the People app, make sure you're showing all contacts, and not just those from one account.
The phone app might be displaying the same person twice because it's recognizing someone who called as having a Google+ profile. This would probably only be seen in the Phone app, and wouldn't clutter up your actual People app, so you probably don't have to worry about it.