I use a cheap TPU case from ebay and an armband for the gym.
Like the OP, it was important to me to maintain the thin profile of the Note. I wanted no added bulk.
I discovered the Incipio Feather Lite case to be the best at maintaining the protection I need, but with no excess weight or bulk to the device.
By the way, I also use an armband for the Note when I run. The lightweight and thin nature of the Incipio case allows the Note to fit into the armband without removing it.