I know that the Chrome browser doesn't have a Homepage button, and I don't think the stock Browser has one either--so it's normal that if you open the browser, it will open to the last page you were on. There are other browsers like Dolphin that have a Homepage button, so you could give those a try.
If your browser is giving you errors like that, go to the App Manager, select the browser, and tap Force Close, then Clear Cache/Clear Data. Now open it and try again.
What other issues are you having? Perhaps if we can help clear them up, you won't hate it so much.