You must be in Drawing mode when clicking the button to get it to cycle through the presets and then to the eraser. and as Relbeek said, the pen must be within an inch of the screen, not on it.
You will see the icon at the top of the screen cycle from the Pen, to any other presets and then the eraser icon will highlight.
If you have all the boxes checked under the menu that you can get to from the spen settings, air view, you will also see the different types of pens, markers and the eraser show itself on the actual screen while in drawing mode..
-bob