I just wanted to correct this to say it is a directory called ".nomedia" and you will need to make sure gallery rescans the card after the directory is added. The easiest way I found to do this is open a bunch of other programs, or you can do a force close or a reboot of the phone.
Thank you for letting us know this. This extra clutter has been driving me nuts.
i'm going to have to re-correct you and say it is definitely a file called .nomedia
you can probably do both, the app most likely looks for anything with the name ".nomedia"
so for the OP's issue, the easiest way i see to do this, is:
1. create said .nomedia file
2. mount the sd card
3. drop that file into any folder you don't want the gallery app to scan
the only thing i haven't tested is if that will stop the music app from looking for songs...
also, if album art is your only issue with the gallery app, you could embed the album image into the music file itself and then delete the images in the music folders
if you have a lot of music, you can find programs that will automate this for you, or you can use something like mp3tag to do it yourself