Yup. This is one of those things where Google's "Stock" experience fails. They won't let you get rid of the search bar even when you want to (and as you've found, it can easily be accessed by swiping.)
You need to install a 3rd party launcher to get rid of it. The same applies to the "App" button in the middle of the bottom row.
Yup, 3rd party launcher to get rid of it. Well I know for sure Apex gets rid of it. If the non free also gets rid of the Google now, I would by it. I'm happy that you don't have to have root to install launchers....