Welcome to Android Central forums. I believe you have something else going on with your phone. The person you're calling has no effect on how your phone performs, such as calling a landline vs a mobile number. As you pointed out in your post this sometimes happens when calling a mobile number so it is more than likely your device.
So let's start at the beginning so to speak with a few questions.
When you attempt to use the speakerphone do you get any message, warning or error?
Are you going from a regular, phone to ear, call to speaker, or are you using some BT device before trying to turn on speakerphone?
Have you noticed this recently or has it been going on for some time? Do you remember when you started noticing this?
My gut reaction was something is using your speaker making it unavailable and therefore not possible for you to use speakerphone. On the devices I've owned this is usually accompanied by a message or pop-up saying something like "your speaker is being used by another app" and blocking the action. Those aren't the exact words but it conveys that meaning. Are you getting anything like that? Even if you're not I still feel strongly that this is what's happening.
To narrow this down a bit perhaps there are a few test you could run.
First call a mobile number and try to use speakerphone. If it all works immediately try a landline number. What happened?
Alternatively the next time you're on a landline call, try using speakerphone, if it fails immediately try calling a mobile number. What is the result?
Lastly the next time you have a failure, any call that will not allow you to use speakerphone, open your multitasking pane and close all background apps and then try again. What is the result?
Let us know what you find and the result of any tests you try and we will do what we can to offer you more advice. And you may also consider posting your question at the Google Pixel forums. No one knows a Pixel better than other Pixel users. This may be known issue with a solution within the community. Try this link...
https://support.google.com/pixelphone/threads?hl=en