The home Menu takes you back to your home screen, while the Apps menu takes you to your list of apps. Think of it like this: the home screen is like a desktop on your PC. You'd put those things you use frequently there so you can get to them quickly. You might also have apps you only use every few weeks or very rarely, and they can just stay in your app drawer, hidden away until you need them.
To remove an app from your homescreen, long press on it (hold you finger on it for a second until it "pops" off the screen), then drag it to the trash bin that will appear, and it'll disappear from your screen (it'll still be in your app drawer though). To add one to your homescreen, open the app drawer, find the app you want, and long press on it and the app drawer will disappear and you can drop the app anywhere on your homescreen.
Widgets are little programs that run on your homescreen. They don't really do anything in the app drawer other show you they are are available. Similar to adding apps to the homescreen, you can long press on one (in the app drawer) and then place it on your homescreen. Widgets can be anything from showing you the current weather or forecast to the price of a stock, or some of your recent unread mails or Facebook posts. They are really meant as a quick way to view information without having to open up several apps. Typically tapping on a widget will open the associated app (so a weather widget might just display that it's raining outside and the temperature is 68 degrees, but tapping on the widget will open up the actual app where you can view the entire forecast or radar or whatever the app has).
For example on my phone, on my homescreen, I use widgets to display my battery percentage, my next appointment on my calendar, how many missed calls/emails/text messages I have, the time my alarm will go off, the weather and forecast for the day and several other things. I get access to a lot of info that otherwise I might have to open several apps to see.
This allows you to really customize the look and feel of your homescreen to what works for you.
Hope that helps.