Isopropyl alcohol is the choice for cleaning phones. We used to bathe wet phones in it, just to remove the water, if nothing else. It's not true that using any liquid can make things worse,. Even chemically pure water (triple-distilled water, for example) is safe.
I'd use a Q-tip, soaked in alcohol. Push the button in and bathe the whole area, letting the alcohol run under the button. Or, if you can, hold the button down, then swish the phone in a plastic tub of alcohol so the button area is submerged. then do the same with the other button. (I can't count how many phones I've saved by using alcohol on them.) And if you can't get 90% alcohol, 70% works fine. (At $3/quart [and boy, has that gone up] it's the cheapest "repair kit" there is for phones.)