Yes, though it may vary from phone to phone. Try long-pressing a blank area of Home screen.
Long-pressing a widget on the Home screen gives the option to Remove it to the widget locker. Doing the same to one in the locker allows you to drag and drop to the Home screen.
To truly uninstall a widget, as you put it, you would need to Uninstall the associated app.