A can of compressed air would be your best friend. I do hardware repairs on both 360's and pcs all the time. You never want to use anything sharp around sensitive electronics, no matter how careful you are there's always a chance you could mess something up. In this case you could scratch the contacts for the usb hub inside your phone, thus degrading usb quality and life.
I personally use q-tips when I don't have any air laying around to use, dipped in a small amount of isopropyl alcohol.
As long as it's powered off and the battery is removed the alcohol won't damage it, just make sure it is completely dry before powering it back on. Wipe out any excess with a qtip and then let it air dry the rest of the way.
Sprint GS3 Running TN's Msg and Chubbs