Clarification has emerged regarding the division of local and cloud processing within "Magic Pointer"—a core AI feature on Googlebook. Details reveal precisely when on-screen data gets transmitted to the cloud.
This information was shared by Google's Alexander Kuscher during an appearance on the podcast "Android Faithful."
On-Screen Element Recognition Runs Locally On-Device
According to Kuscher, when Magic Pointer is activated, the initial process of identifying and detecting interactive elements on the screen is handled entirely on-device within the Googlebook hardware.
For example, automatically detecting and highlighting on-screen text, images, or incoming emails does not involve transmitting any data to Google's cloud servers.
Data Is Sent to the Cloud Only After Triggering an Action
Data is only sent to cloud-based AI models when a user selects a specific element and issues a command to Gemini, such as requesting analysis, translation, summarization, or verification.
Kuscher illustrated this with an example of checking whether an incoming email is spam. Identifying the object on screen as an "email" occurs locally; the actual content is transmitted to the cloud only when the user selects the email and asks Gemini to analyze it.
In short, having Magic Pointer active does not mean screen data is continuously streamed to the cloud. By sending data only upon user execution, the feature is designed with privacy and security in mind.
Mouse Cursor Serves as a Key Signal of User Intent
Kuscher also shared an insight from internal testing at Google, noting that internal users moved their mouse cursor an average of nearly one mile (about 1.6 km) per day during routine work.
This analysis reinforced that cursor movement is a crucial signal indicating where user attention is focused, ultimately inspiring the creation of Magic Pointer beyond standard pointer functionality.
Furthermore, because Magic Pointer is a built-in feature of Googlebook, it was confirmed that users will be able to access it without subscribing to a paid Google AI plan.