Yup, this toolbar is populated automatically if the app dev specifically codes in the support (and, truth be told, there's really not too much to it, just a few lines of code Samsung already provides). However, most devs do not go through the trouble of adding it and we end up with a semi-deserted list.
Like dpham00 says, there is a way to 'hack' the system into letting any program you install run in MultiWindow mode, but it requires you to root the device, effectively voiding your warranty. No free lunch, huh?