As a follow-up to this, after I finished the book I was reading I force stopped the app, cleared the cache and data and the duplicate books were gone. Of course, I had also gone through and renamed all the files on my device to remove all apostrophes, ampersands and exclamation points, so that may have also been a part of the solution.