Welcome to Android Central! You should be able to simply select the photo and tap the Delete button. You might have to tap the Menu button first.
However, if the photo you're trying to delete is actually a photo in a cloud album (i.e., Google+/Picasa), then you can't delete it from the Gallery app. You can do so from the Photos app (which is actually part of Google+).