Don't remove, freeze (you have to be rooted). Then if the app not being there causes a problem you can unfreeze it. (If it's a system app, you might not be able to reinstall it if you removed it.)
You can probably remove all the carrier stuff that you don't want. (You may want to keep some.) You can probably remove some of the manufacturer-installed stuff. You can remove any app that's not in /system/<anything>.
Just back up anything you're going to uninstall (use App Backup & Restore, so you create .apk files, which you can install by just tapping.)