It'll be in the specific app's settings. For example, in Gmail, hit the menu button, choose More, then Settings. Choose Select Ringtone and pick Silent from the list (should be at the top). You can actually do this per account if you have multiple ones set up.