Picasa, Gallery, and Photos are fairly redundant, but that's because Google is gradually phasing other apps out in favor of Google+ Photos.
Picasa is the original cloud photo storage site that Google bought, and gradually incorporated into Google+ as Google+ Photos. If you ever stored any photos in Picasaweb, you could see it in Google+ Photos. Google has now made it essentially impossible to go to the original Picasa website--it redirects you now to Google+.
Picasa on your phone is a holdover from when Picasa was a separate website. It's how Android phones have historically synced your photos from Picasa/Google+ to your stock Gallery app.
The stock Gallery app has been around ever since Android was developed, and is simply a viewer app for any photos you have stored locally, and in some (but not all) devices, also in the cloud. You don't physically store any photos in the Gallery app--it just reads the photos that are stored and displays them. My Razr Maxx's Gallery app stopped being able to display online albums, so I thought Google was trying to kill that function on the Gallery app (and therefore shunt everyone to Google+), but I saw that the new 2013 Nexus 7's Gallery app can still access online albums, so maybe it was just a bug on the Razr Maxx.
The new Google+ Photos app essentially has all of the stock Gallery app's functions, but also adds on all of the editing goodies from Google+. If you wanted to, you could disable the stock Gallery app in favor of Google+ Photos.
Facebook
Twitter
Instagram