Ok. I think I figured it out, but it is not intuitive. In fact, it seems a backwards way to do it. Apparently the brightness slider is the base of the brightness, the lowest brightness will go. The adaptive brightness works from there.
I set the slider to a low amount, went into a dim room and after a few seconds, the screen darkened to that base, but it was not smooth and took a bit too long. When I shone a flashlight at the sensor, it brightened back up.
So when I had the slider all the way up, it looked like the adaptive brightness was not working because it could not go below the max brightness.
Strange way of doing it, but at least I understand it now.