This happened to me a few days ago too. Sadly like others here have reported, the cause of it is Google Chrome. Its not the SD card.
I found this out after formatting the card, swapping it with another but mediaserver keeps climbing to the top of battery rankings.
Problem was solved after...