I did it!
I don't know which one did it, but here's all the story. Turned off the phone (battery removed).
Turned on and immediately kept the "volume down" button pressed until I was greeted with the (repeating) upgrading part (143 apps for me).
After that, I kept the button pressed again and...