I don't have the galaxy note, but verify that in the Folder list that only the "inbox" is selected and not "all folders".
My email client was originally set to view "all folders" and when I deleted mail it moved from the inbox to deleted, but the deleteted folder was part of the "all folders" list, so the email was still showing up.
There is a setting to remove the mail from your mail server when the mail is transferred to your phone. Otherwise it will keep loading, I think. See menu account settings incoming.