Personally I prefer DVD Catalyst over Handbrake, but that's not your question. According to DVD Catalyst Samsung Galaxy Nexus | Tools4Movies is the same as the Xoom which is built into the program as a selectable profile.
The process for DVD Catalyst is:
In the left bottom corner select Android > Xoom > Go
If you still want to use Handbrake the process is:
Download Handbrake and install
Download VLC and install
Open up handbrake and select your DVD as the source
Download this Xoom text file.
Change the extension from txt to plist.
Import the plist file to handbrake.
Select the imported plist and then click the export button.
Alternatively you can view the contents of the text file and read the settings yourself.
Actually, there are profiles for the Galaxy Nexus already. The resolution is slightly different than that of the Xoom, but either way will work just fine.