Welcome to Android Central! Android doesn't really work that way. The system prefers to have apps open in RAM, because empty RAM is wasted RAM. See these articles for more info:
If there are apps you never use, uninstall them or (if they're system apps or preinstalled bloatware) disable them in Settings>Apps>All. Disable by selecting the app in that menu screen, then tapping Disable. This will prevent the app from opening.