I had the Pixel 3, and honestly I don't see a downgrade in performance. If anything, the 4a feels snappier than the 3. I don't know if that's due to better optimization, more RAM, or it just being a newer device. I know that startup times are much quicker than my Pixel 3, and standby times are much better ( when I'm sleeping over a 6 hour period, battery drop ranges from 1-3%, compared to 8-10% drop on my Pixel 3). And all this for less than half the price (I had the 128GB Pixel 3 which ran around $800 IIRC).
No regrets about going with the 4a. The Pixel 5 looks great going by the rumors, but I don't know if it's twice the money great. I'll watch and see though, and if it's really a noticeable improvement over the 4a, I might consider it. I just wonder if the 4080mAh battery is there more for the 90Hz screen or 5G, which I won't see for at least another few years, if at all. Where I live, we barely have LTE, and from what I've read 5G towers cover a smaller area, so unless Verizon increases the number of tower's, it's doubtful.