| || |
I think it has something to do with "pinning". In the play store you have the option to pin content to your device like movies, music, or books. I think all you have to do it unpin it. It should be under the book selection on the play store page.
EDIT: Well currently it does not seem to call it pinning any longer. On my Moto X each of my books has a 3 dot menu button with the top option being to "Keep on device" and if it is kept on the device the option changes to "Don't keep" which deletes it from you device.