It's not a download manager, but you can access files you copy to the Xoom fairly easily.
First, plug your Xoom into your PC with the USB cable.
On the PC, navigate into the Xoom and under the folder labeled SDcard, look for a "download" folder. This is the default location for files downloaded from the Xoom Browser. You can use this for your own files you transfer over the USB cable, and for quick access on the Xoom, open the Browser, touch the menu in top right corner, and choose "Downloads". You'll see all of your files.
Another way to do it is to create your own folder under SDcard, and then install a file explorer like Astro on the Xoom. Use the Xoom file explorer to navigate to your custom folder and again you'll see your folders.
Finally, you can also just go to the Gallery on the Xoom, and if it finds compatible file formats it will list them in the Gallery, where you can touch the folder name and then the file to play it using the native video player.
There are several other threads here about video formats, so check them out, as the Xoom only natively plays a couple of formats.