Looking over it again, I found out it was a notification, and with the changed standard settings, dismissing it worked now. Nothing to see, carry on :)
I just noticed that when clicking the error I am given other settings/displayed limits. But I am unable to modify anything on that usage page.
Anyone know where I can find and delete it?