I did the exact same thing on my first wear. I went for a bike ride with them and I was thinking the entire time that they were really awful buds. No base, really tinny sounding, max volume was terrible. I even tried the larger wingtips and still put them back in upside down.
It wasn't until the following day that I looked at them again and it hit me "Hey wait, these little copper dots are for charging, they don't go against the ear. Wow, I felt really stupid but it sure did improve the sound quality!
Now, the only remaining problem I am having is the intermittent beeping. Apparently they are prone to beep when they "overheat," which I have read can begin to occur in the 90F to 100F range. Mine started beeping yesterday on a walk with the dogs and the current temp was 92F. That is a bit of a problem since much of the world can reach summer temps over 100F. I hope the threshold is something that can be adjusted via a firmware update because otherwise I really like them.