Try hitting the settings button to left of the home button. There should b an option to delete folder or delete item in there.
Nm just tried. Its not there. But I just held the folder and drug it to the top into delete item bin. Bin turns red and let it go. Works good lasts long time.
Editted for the second time... did you create a folder inside app drawer? If so I got it figured out. Just erase the name of the folder and hit done. As long as the folder is empty it should delete itself.