Last updated: August 29th, 2024 at 15:25 UTC+02:00
Google is rolling out two new very useful features to Gemini for Android: ‘Ask about this screen’ and ‘Ask about this video’ (via Gadgets360). Let’s take a look at how they work and what they do.
Starting with the ‘Ask about this screen’ feature, Gemini will give you the option to use it when you summon the virtual assistant by saying “Hey Google” while using the phone (except for when you are in the YouTube app). As for the ‘Ask about this video’ feature, Gemini will give you the option to use it when you summon the virtual assistant by the wake term while using the YouTube app.
What does Gemini’s ‘Ask about this screen’ feature do?
Once you click on the ‘Ask about this screen’ feature, Gemini will take a screenshot of the screen, analyze it, and explain what’s going on on the screen. For example, if you use the feature while reading an article on a website, it will explain what the article is about. However, note that it can analyze only what is in the screenshot. So, if only a portion of the article is on the screen, it will only explain that.
What does Gemini’s ‘Ask about this video’ feature do?
As for the ‘Ask about this video’ feature, when you click on it, Gemini will analyze the YouTube video (including analyzing its captions) and explain to you what the video is about, as you can see in the screenshot above. The feature will not work if the video does not have captions, which is kind of disappointing. Both features let you ask follow-up questions, helping you understand the context more clearly.
The story continues after the video…
Both features are live in the Google app for Android’s version 15.33.38.28. However, there’s no information if Google is offering this feature as an update or as a server-side change.
