Use a file explorer to navigate to your download folder on the internal phone storage. Delete from there. I don't know if the factory "My Files" will let you, I have always used Root Explorer, another good option is ES File Explorer. It is free in the Market (Play Store).