This is done (using the
RECEIVE_BOOT_COMPLETED permission for the
ACTION_BOOT_COMPLETED broadcast) when Android has finished booting in order to do
something. In almost every case, this something is the initialization and start of a background service, but that doesn't mean the app has a component constantly running from that point on.
These services, if well-designed, spend a majority of their time inactive. When they're active, it's usually for operations with a sync adapter, one-time or ongoing notifications, or to bring up a visible component of the app. These actions are usually the result of receiving an intent broadcast, and so the background service is inactive until that time. An example:
the Download Manager. The "Downloads" item in the launcher, outdated icon and all, is the user-facing portion of the download manager. Its background service is started on boot, but it (expectedly) does very little until you (or an app) initiate some sort of download using the built-in download manager.
Apps can schedule one-time or recurring alarms. These are different from Clock alarms; these are system alarms that send a broadcast intent when fired, allowing apps to be called at any time, even if the device is asleep, without having to keep the device awake and waiting. The problem? They're cleared at reboot. This means any app that wants to do something in the future needs to use the start-at-boot feature in order to reschedule its alarms. If it doesn't, don't expect it to provide you with updates, notifications, or anything else until you open the app at least once (it should have logic that checks to be sure its background service is running when the app is launched, and so it will update its lost alarms at that point).
If this weren't the case, your wake-up alarm in the morning would keep the device awake all night until it goes off. Not the most efficient approach...
Things like Clock, Media Storage, Android System, Sync Feeds, System Updater, Download Manager, and so forth should probably be left as-is, and there are certainly more I'm forgetting to include offhand. For any others, the impact on you will depend on what the apps want to do at boot (which they may or may not explain in their Play Store description when describing permissions requirements) and what logic the developers have implemented for the app to "catch up" if the service isn't running for whatever reason.