I had the same problem and unfortunately reading the responses didn't really give me a direct answer... but I was able to figure out what the problem was... When I initially connected my phone to my computer to transfer files a folder for music was already displayed and so I saved all my music to it. The folder name was "com.google.android.music" and, from reading various posts here and elsewhere, I determined maybe that was the problem. So to fix it I went to "my flies" and selected the "SD memory card" then I 'pressed' the "menu" button to create a new folder. I titled the new folder "music" (though it probably doesn't matter what you name it). now when I open "my files" and select the SD card, I have a folder called "music" and one called "android". I selected the "android" folder then the "data" folder then the "com.google.android.music" folder, which was where all my music was saved. I selected all of my file folders with music in it and pressed the icon in the upper left corner (it has three bars and a triangle) and selected move. Once that is selected it takes you back to the initial screen where you select "SD memory card" or "device storage". I selected "SD memory card" and then the "music" folder I created. in the upper left hand corner, I selected "move here", which moved all my music to the new folder. I powered off and back on and walla! my music app can now find all my music, and has them in the folders as they were saved. Some of my folders contained folders, and the folder within a folder comes up as a separate folder in the player. i.e. I have a folder called "Instrumental" and within it was another folder called "Final Fantasy"; the music player shows the "instrumental" folder as well as a separate folder for "Final Fantasy". The player shuffles as well. (P.S. I'm using the music player that came with my device.)
I'm not a tech genius so all the tech lingo in other posts was a bit hard for me to follow. I hope that others who are just as confused by the lingo find this easy to follow along. Keep in mind I use Boost mobile so the way your carrier has things set out may be a bit different.