On what do you need to hit allow? What device do you have? What version of Android? Are you using the stock contacts and phone dialer apps?
I'm going to throw this out there as a possibility - is the dialog asking to allow contacts to access the phone app? You would need to allow that, and you might be able to go into the permissions for either of those (the direct path depends on the device but on Pixel devices it would be Settings > Apps & Notifications) and set the permissions to always allow rather than prompt, but the permissions system depends on what OS version you have.
If you need additional help or have more questions or details to share, please join the site so you can reply in this thread. See
this link for instructions on how to join Android Central.