How do you search what's on the screen? It says to hold the home button....what home button?

[PaulQ]

I was out tonight and wanted to search what's on my screen (German, hoped to get a translation). I squeezed and said, "What's on my screen?" It told me to hold the home button down (see image).

Since the gesture interface removes the home button, I didn't know what to do? I should mention I am using Action Launcher but I have the Pixel's gesture interface active and it works as intended.

Is it right in front of me or there's no way to do it right now?

FYI - I have the setting to do this turned on: Use text from screen.



Google's help page says to do basically what I did:

Your Google Assistant can show you info, apps, and actions related to the text on your screen.

1. On your Android phone or tablet, touch and hold the Home button or say "OK Google."
2. Tap What’s on my screen?

However, later on the page it says to turn on "use screen context" which must be wording used in the past.
https://support.google.com/assistant/answer/7393909?co=GENIE.Platform=Android&hl=en

 

[J Dubbs]

That's a good question I'm curious about the answer myself ;-)

 

[osubeavs728]

have you had success doing that motion before? I ask because I just did a couple of trial runs by doing just the squeeze. When the assistant picks something up on the screen that it has results for, then the first option available is "what's on my screen?", which when clicked will show you results.

However, when I did the squeeze and there were no results, if I said "what's on my screen?" I would get the instructions on needing to turn the permissions on.

Long story short, when you do the squeeze, it brings up assistant and searches your screen at the same time. I good way to make sure your screen is getting searched is to enable the setting that says "flash edges white when screen is searched" or something like that. When that's on and you do the squeeze you'll see the familiar visual indicator.

 

[Rukbat]

To get to recents in gestures, swipe up, then long-press the screen for a moment.

 

[osubeavs728]

To get to recents in gestures, swipe up, then long-press the screen for a moment.

C'mon, man... Read the OP... Not cost to what he's asking for.

 

[Guyinbox]

On my 3xl running Android 10, squeeze the sides to bring up Assistant, and underneath the prompt that says "Hi, how can i help?" There are two options to tap. "What's on my screen" and "share screenshot."

I will add though, the what's on my screen option hasn't produced any results for me in a long time. It just says couldn't find anything on your screen. For me at least, this feature seems to be broken.

 

[PaulQ]

have you had success doing that motion before? I ask because I just did a couple of trial runs by doing just the squeeze. When the assistant picks something up on the screen that it has results for, then the first option available is "what's on my screen?", which when clicked will show you results.

I'm not sure what you mean by motion (your first sentence). I was using the squeeze. Since you mentioned it though, I tried the swipe just now. Nothing.

However, when I did the squeeze and there were no results, if I said "what's on my screen?" I would get the instructions on needing to turn the permissions on.

This is what I am getting every time. I have tried it on different screens that I would think would generate search results. Can you do one that works and I'll try the same thing? A web page of some content.

Long story short, when you do the squeeze, it brings up assistant and searches your screen at the same time. I good way to make sure your screen is getting searched is to enable the setting that says "flash edges white when screen is searched" or something like that. When that's on and you do the squeeze you'll see the familiar visual indicator.

I have the flash edges on. This is what I see... <squeeze> immediately white lighting runs up both sides from the base of the phone around the top once... the "Hi how can I help" appears and rainbow colors along the bottom pulse. Nothing is searched on the screen.

I found a couple of help articles with similar compaints but pre-Pixel 4.

 

[PaulQ]

On my 3xl running Android 10, squeeze the sides to bring up Assistant, and underneath the prompt that says "Hi, how can i help?" There are two options to tap. "What's on my screen" and "share screenshot."

I will add though, the what's on my screen option hasn't produced any results for me in a long time. It just says couldn't find anything on your screen. For me at least, this feature seems to be broken.

I believe the assistant on the Pixel 4 is different from the 3. It only brings up the one line "how can i help" - it's much lower profile than what I recall on the 3 and 2. I have not seen anything come up under it. Maybe I am supposed to?

And.... thanks for pointing that out. It could be that it just doesn't work well in general.

 

[osubeavs728]

By motion I just meant the process of translating something on your screen via the Google assistant. I've never done that so I wasn't sure how it was supposed to work.

Here's an example where I access l pulled up assistant and it recognized something on my screen. I hit "what's on my screen" and those results pulled up. It did take 3 tries on three different posts for something to hit though.

I will say, I feel like it used to be better at generating results.

 

[PaulQ]

By motion I just meant the process of translating something on your screen via the Google assistant.

Now I get it.

Are you using a Pixel 4 for those screenshots? Isn't that the "old" assistant? I do recall that "what's on my screen?" button from my Pixel 3. I think a previous message in this thread from Guyinbox was referencing the same thing. I heard that using a launcher can kill the new assistant (but I am...?).

Below is how the assistant appears to me, the low-profile version which also doesn't seem to ever pop up any additional buttons or text.




I did chat with Google support last night and got nowhere. About halfway into the troubleshooting, I realized we were just doing things I would do for any misbehaving app. They ended up telling me to send feedback inside the app. In other words, I won't be searching screen content anytime soon.

 

[LeoRex]

Mine keeps telling me that I need to enable it... Even though it has been that way from the start.

It's a bug.. Happens. It'll get fixed soon enough

 

[strikeIII]

Yeah looks like it's not working. Even when I enable the home button instead of gestures then when I hold the home button down I still don't get the what's on my screen option. The only work around I found was to take a screenshot and use Google lens or use Google Lens from the camera app for things around you.

 

[PaulQ]

Yeah looks like it's not working. Even when I enable the home button instead of gestures then when I hold the home button down I still don't get the what's on my screen option. The only work around I found was to take a screenshot and use Google lens or use Google Lens from the camera app for things around you.

I forgot to mention what Google had me do and it was exactly that - enable home button. I got about a 50/50 success rate actually before it would just show a black screen upon activation and never come up with any results.

Thanks for the workaround. Not convenient but it could come in handy.