Most likely the charger my friend.. But don't rule out the battery.. it's not going to be the charging port connection because you say it will charge if the Note is turned off.. To me it seems like the charger is not able to generate enough juice to counter act the natural drain of your phone.. You may also have some rouge app really draining resources.. that's why your charger can only catch up while your phone is at peace..