Be aware that the system will then start reopening apps automatically in RAM, which is how Android is designed. If you do this frequently, it will probably contribute to increased battery drain, since it takes energy to close and open apps.
+ 1 what B. Diddy said. The system is designed that way. I have on average about 20+ recent apps and my phone runs just fine, and minimal battery drain. It takes longer for system to have to fully reload them also.
I normally don't close the recent apps unless something running is obviously taking a toll on my battery or performance (Shazam used to be a battery drainer if you left it open in the background, not sure if it still is though ). Like they said, most of the time, it's better for the OS to use as much RAM as possible.