The Kindle app stores books (at least the Amazon books) in the cloud for your permanent copies, but you can download whichever ones you want for offline reading. That is what I use, since I used to have a Fire, so I have some good books there. There is a blog for Free Kindle books and Tips, and they have an app as well... Most of what I have was free... with a few paid books to round out the available titles. There are ways to convert non-Kindle books to use on the Fire (I assume workable for the Kindle app as well) but I haven't used any yet.
So the short answer, is you don't need to be on wi-fi to read with the Kindle app. I assume all the other readers would also allow offline reading.