You have to change that in the apps. For example, go to your Mail app, hit menu key, go to Settings, click on the account, and change notification settings from there. Where it says "select ringtone" just click on it, scroll up and choose "Silent".
Same for Twitter, messaging, etc. Go in the...