Simply go to download manager and click on force stop, then click on clear cache and clear data.
In Android 6, you may don't see clear cache and clear data, don't worry, simply click on Storage text and then clear cache and clear data.
App-->Settings-->Applications-->Application Manager-->More-->Show System Apps-->Download Manager