About a week ago, Google introduced AI Overviews for Search, a feature that compiles AI-generated summaries from various web sources to answer user queries. This feature, launched in the United States after Google´s I/O 2024 developers conference, has quickly drawn criticism for numerous factual inaccuracies reported by users.
Initially launched in beta in May 2023 as part of the Search Generative Experience, the rollout of AI Overviews aims to streamline user information retrieval, but the current inaccuracies highlight the complexities and responsibilities associated with deploying generative AI at scale. Ensuring the reliability and safety of AI-generated content is a significant challenge for Google.
Google's core business relies heavily on search and the ads generated alongside search results, making the reliability of its AI-generated answers crucial. Privately, Google employees believe that the problematic answers are fringe issues and that AI generally performs well. The shift from traditional search results to AI-generated answers represents a significant change in the responsibility and authority Google assumes in providing information.
User Reports Highlight Dangerous Errors
Social media platforms have been flooded with examples of bizarre responses from Google's AI Overview, such as advising users to eat rocks. One user received a bizarre suggestion to add “about 1/8 cup of nontoxic glue to the sauce” to fix cheese not sticking to pizza.
https://t.co/W09ssjvOkJ pic.twitter.com/6ALCbz6EjK
— SG-r01 (@heavenrend) May 22, 2024
Another user asked if it was safe to leave a dog in a hot car and received an affirmative response, citing fictional Beatles lyrics. These incidents have raised serious concerns about the reliability and safety of the AI-generated content.
https://t.co/U1wBLD67Mh pic.twitter.com/CHh74EMmUd
— napalm (@napalmtrees) May 23, 2024
Echoes of Bard's Launch Issues
The issues with AI Overviews are reminiscent of the challenges Google faced with its AI chatbot, Bard, which launched in March 2023. Bard was criticized for lacking adequate safety and ethical measures. In February 2024, Google had to disable the image creation feature in Bard's successor, Gemini, after it produced inappropriate images of people with darker skin tones. Google has since promised improvements before reintroducing the feature, but it remains inactive.
The company has ambitious plans for AI Overviews, including multistep reasoning for complex queries and video search in Google Lens, but its current reputation is at risk due to these initial issues.
Google is under pressure to compete with other AI-driven search engines like Bing and OpenAI's potential search engine, as well as new AI startups like Perplexity AI, which offers an AI-based web search.
Google's Response to Criticism
Google has acknowledged the issues with AI Overviews and is expected to take corrective measures. The company has been manually disabling AI Overviews for specific searches following the spread of these unusual responses on social networks. Spokesperson Meghann Farnsworth said that the company is taking swift action to remove AI Overviews for certain queries and is using these examples to improve their systems.
Google CEO Sundar Pichai says that Google has managed to reduce the cost of delivering AI answers by 80 percent through hardware, engineering, and technical advancements. However, amid a problematic launch, Google's commitment to improving the safety and accuracy of its AI features will be essential in regaining user trust.
Google argues that the majority of AI Overviews provide high-quality information and that many of the problematic examples are uncommon queries or doctored instances.
The history of the company with products like Bard and Gemini shows a trend of releasing AI products that subsequently need considerable modifications. Google has retracted AI products in response to user backlash already several times. This cycle of release, criticism, and retraction has been noted previously, and it seems unlikely to change despite growing competition in a market where the company has held a near-monopoly for about twenty years.
