HomeWinBuzzer NewsGoogle Faces Criticism Over Gemini Pro's Inaccurate Responses and Coding Limitations

Google Faces Criticism Over Gemini Pro’s Inaccurate Responses and Coding Limitations

Google's new AI model "Gemini Pro" faces criticism for inconsistencies in factual accuracy, translation errors, and limitations in coding capabilities.

-

Earlier this week, took the wrapper off its new AI model. Known as Gemini, the platform is available in three tiers and is meant to position Google against and in the increasingly competitive AI market. While Google has boasted about Gemini's advanced architecture and its equal or superior performance to other notable AI models, including OpenAI's GPT-4, user experiences shared online paint a less favorable picture.

Inconsistencies in Simple Queries Raising Eyebrows

A critical component touted by Google is Gemini Pro's factual accuracy. However, users are taking to social media to share discrepancies in the AI's responses. A notable error includes providing incorrect information about the 2023 Oscar winners. Despite claims of improved reasoning and understanding over its predecessors, Gemini Pro attributed the Best Actor award to Brendan Gleeson instead of the actual recipient, Brendan Fraser. Similarly, missteps occurred when asked about other Oscar category winners.

Additionally, translation capabilities seem to be a weakness for Gemini Pro, with users reporting that asks for simple translations are met with incorrect responses. For instance, a request for a six-letter word in French was met with a reply that included either a five or seven-letter word, highlighting concerns about Gemini Pro's multilingual performance.

Coding Capabilities Questioned Amid Competitive AI Landscape

Google emphasized Gemini's coding prowess, yet users experimenting with the new AI model have reported issues. These include Gemini Pro's inability to correctly execute basic coding tasks, like writing the intersection of two polygons in Python—a problem that seems to affect not just Gemini but other AI models as well.

Comparisons are being made to .5, the predecessor of , highlighting Gemini Pro's challenges in areas like developing code for analog clocks and games like Tic Tac Toe, where it lags behind in terms of output quality and accuracy of results.

Furthermore, Gemini Pro's approach to sensitive queries is to encourage users to conduct their own searches, a stark contrast to the detailed summaries given by . This cautious stance can frustrate users looking for quick, in-depth information.

Despite these reported shortcomings, Gemini Pro is not the final or most capable version of Gemini, as Google plans to release Gemini Ultra next year, which may address some of the current concerns.

As advances in generative AI models progress rapidly, Google's commitment to enhancing Gemini's performance remains paramount to meet user expectations and stay competitive. The technology giant has yet to publicly respond to the criticisms, and it is unclear how quickly they can or will address the feedback from Gemini Pro's early users.

Did Google Launch Gemini Too Early?

We have been here before with products. In February, the company was blindsided by Microsoft's all-on approach to AI with the launch of Bing Chat. While Bard was already under development, Google decided to rush the AI chatbot to market. The result was an embarrassing showcase of the AI's capabilities. Bard launched a month after the showcase and was buggy. It has since improved greatly, but that early launch was rough

It seems Gemini is going through a similar process. Interestingly, there were reports recently that Google would delay the launch of Gemini. Inaccuracies and issues with multi-lingual capabilities were said to be behind the delay. Of course, the postponement never happened, but concerns about the performance of Gemini are now unfolding post launch. 

Luke Jones
Luke Jones
Luke has been writing about all things tech for more than five years. He is following Microsoft closely to bring you the latest news about Windows, Office, Azure, Skype, HoloLens and all the rest of their products.

Recent News