Google’s April Pixel Drop Shows how AI Can Be more Transformative than trivial pursuit

As reported by 9to5 Google’s Abner Li, Google this week released its monthly Pixel Drop for Android. April’s edition brings with it but one lone feature: Gemini Live’s new Astra camera. Li writes the functionality is now available to all Pixel 9 phones, free of charge.

The “Astra” name refers to Project Astra, a DeepMind initiative from Google characterized by the Bay Area-based tech titan as “a research prototype exploring future capabilities of a universal AI assistant.” On its website, Astra is touted as helping people “explore [their] world like never before,” which is precisely what Gemini Live’s newfound camera is purported to do. To wit, after installing the software update, users will have gained the ability to point their camera at anything in the real world and have a conversation about it. In a brief video posted to YouTube (embedded below), Google demonstrates the Astra camera’s functionality by showing a person asking Gemini to assist with choosing between color options for their newly-glazed piece of pottery.

The notion that artificial intelligence can help people “explore the world” has deep resonance to accessibility, particularly to those in the Blind and low vision community. That a Blind Android user, for instance, could point their Pixel 9 device at some object—say, a street sign—and have it described to them has enormous potential to help make getting around one’s neighborhood more accessible—which, in turn, engenders heightened feelings of self-esteem and independence. Likewise, the Astra camera could be used in the kitchen at home, where a Blind or low vision person could create a makeshift Be My Eyes experience by pointing their phone at a carton of milk and asking about the expiration date on the label. These scenarios, both ostensibly ho-hum aspects of everyday life, are made eminently more significant by technology; whereas typically most, if not all, people with blindness or low vision need assistance to keep tabs on the milk in the refrigerator, they now can do so with agency and autonomy with the help of Gemini Live on their phone. That isn’t at all trivial, especially considering the prevailing—arguably ableist—perception societally that disabled people are rather hapless and helpless in our ability to function with any standard sense of normalcy.

Spiritually speaking, the Astra camera isn’t dissimilar to Apple’s Visual Intelligence on the 2023 iPhone 15 Pro and last year’s iPhone 16 lineup. In fact, the basic conceit is exactly the same: point your phone’s camera at something and the virtual assistant tells you about it. (Also highly similar are the myriad Detection Modes found within the Magnifier app on iOS.) How performant one is versus the other is largely immaterial in an accessibility context, as both are shining examples of the genuine good AI can do for the world. Tech companies, whether Apple or Google, like to hawk their AI-powered wares to the mainstream as conduits towards convenience. Though it is true something like the Astra camera indeed is convenient, its accessibility merit ought to be celebrated with more revelry. Whether Astra or Visual Intelligence, people choose whichever fits them best; the salient point is simply these technologies can be as transformative and life-altering for some as they are bleeding-edge and fun for others.

Finally, a few cursory thoughts on Google Gemini. A recent spate of sponsorships on one of my favorite nerdy podcasts pushed me to install the Gemini app on my iPhone and try it out. The app supplanted ChatGPT on my Home Screen, which I’d heretofore been using for my generative AI needs. It’s been a little over a month since switching to Gemini and I gotta say: I really like it. In terms of performance and acumen, Gemini seems to stand toe-to-toe with ChatGPT; I’ve noticed some errors and hallucinations, but Gemini mostly gives me what I want from it. It also helps that, as far as UI design is concerned, I’ve come to prefer Gemini over ChatGPT. Whereas ChatGPT feels staid and utilitarian, Gemini feels more “human” and whimsical in terms of its design. Pragmatically, Gemini has taken over much of the grunt work from Google Search in Safari in my usage. I find it easier to give Gemini a question or command, then use the in-app browser if I want to dive deeper. It feels much more accessible than using the browser and scanning through umpteenth search results—which, from a disability standpoint, exerts good amounts of energies in terms of vision and fine-motor skills.

My close friend Joanna Stern at the WSJ has similar feelings, writing in a recent column that she isn’t “going back” to the tried-and-true Google Search mechanics after using a slew of popular AI agents in testing. Stern’s lede says it all: “Somewhere out there, Jeeves is raising his teacup to the AI revolution.” Jeeves, as she notes, was the ‘90s-era ancestor to products like Google Search, Gemini, and more. Jeeves was limited in its capacity, Stern goes on to say, with the current crop of AI-driven assistants actually fulfilling the promise espoused by their ancestral digital butler. Stern’s focus isn’t accessibility—although we’ve talked a lot about it over the years—but her experiences underscore my aforementioned point about AI being more than sheer amenity or a vehicle in pursuit of trivia. As I said, to use, for instance, Gemini—whether the app on Astra camera—can prove far more accessible when conducting research. It’s a sentiment Microsoft’s chief accessibility officer Jenny Lay-Flurrie has shared with me at length numerous times. It’s better for her teenaged daughter, who’s neurodivergent, to use the ChatGPT-charged Bing to assist with essay research than use Bing in the more traditional fashion. Again, not an insignificant development for AI or disabled people.

But back to this month’s Pixel Drop. Of particular import, Li reported the Astra camera feature is not tied to the Gemini Advanced subscription service. Instead, the software is tied to Pixel devices themselves. This means Astra camera will remain available to users a de-facto accessibility feature in perpetuity regardless of their subscriber status.