I wouldn't bother mirroring because due to the other concerns, I would recommend leaving it open to only a very select few, preferably those building during the hours your family isn't on. That may work sometimes, but like Jerry mentions, he was building for 8 hours on mine. Odds of that matching perfectly with your families sleep/work patterns are slim. And they WILL complain. Depending on your router, you could also throttle based on time of day, that may help, but reduces the main function, which is speed.
Unfortunately, every rule you add to home routers, slows them down. Just a few traffic rules can bring them to a crawl, just another reason for a high end router. I have mine set to prioritize my own traffic (except torrents), and I limit it to 1 or 1.4megs (I forget) upload speed. With that, I can play Left4Dead2 pretty much lag free (I may need to add a download limit). If I remove either rule, I either get a 500ms lag spike for 15 seconds, or I get a 350ms lag for 5 minutes. Keep in mind, I have a pretty fat connection, 2meg up/16meg down with bursting enabled up to 60megs(!). If you have just a normal 3 or 6 meg DSL with a typical home wireless router, forget it, you will not be able to get your router to keep you running smooth during games, and even normal surfing will be dramatically effected. You may as well just build this for yourself.
Unfortunately, while you may have a fast system, your connection is really going to be a bottleneck no matter what you do. One bottleneck is all it takes. It may work okay for one or two people, but beyond that, it's going to come to a crawl at various times. You may want to see what an internet upgrade will cost you, it may not be as much as you think. You probably will want 8-10meg down at least though.
My internet and hosting fees can be written off on taxes and my websites cover it, so I have an advantage there.