Wow, thanks for the reply! You've confirmed my suspicions that it's a universal issue, and not just on select phones.
I also suspected that it may well be intentional. As I said above, it doesn't actually bother me and for my everyday use, I don't notice any impact. In fact, I can see it being a bit annoying in some situations if it did dim too quickly. However, there are definite situations where it can be annoying that it doesn't dim quickly. As some have noticed, one situation is when you are in bed with the room light on initially (say it's a really bright light), and then you switch the light off while still using the phone (that is, held vertically in your hands) - your eyes may find the screen too bright in the darkness!
But I guess this may not be a big issue for most people anyway - for example, I don't think many people dim their TVs or computer screens when they switch the light off in the room. I certainly don't.
And to be fair, to activate the auto-dimming in such a situation, it's not exactly difficult to bring up the swipe down menu and tap the auto-brightness icon twice, hold the phone horizontally for 4-5 seconds, or to switch the phone off and on. For actual usability, it's far more important that the phone auto-brightens quickly, which it does very well (in about 1-2 seconds as you noted) - not being able to see the screen because it's too dim is a big problem, while the screen being a bit too bright (which I personally don't really experience, perhaps because it's AMOLED display, so it's easier on the eyes even when it's bright etc?) is arguably not an issue at all.