I think the screen is actually 854x480 (according to the Moto website). Generally MP4 is a decent format to use since it works with most everything and often has hardware decode support.
I haven't used either of those programs, but I'll be keeping an eye on the DVDFab forum for posts like these, until then I've been using a Droid profile with a few minor edits.
Standard def DVD resolution is lower than the Droid X screen resolution, so you can't encode them to that size. It does sort of put in perspective just how good the screen is on your X though, doesn't it?
