Google quietly made its Gemini 2.5 Pro (Experimental) model available to everyone using its free web app starting March 29th, a remarkably swift expansion just days after its initial March 25th debut for paying subscribers and developers. This wide availability brings one of the model’s more intriguing, recently highlighted capabilities to a mass audience: an apparent knack for understanding not just the text within PDF documents, but their visual structure as well.
Analysis by Sergey Filimonov, Co-Founder of Matrisk, an AI startup specialized in insurance filing management, suggests Gemini 2.5 Pro marks a departure from previous large language models by offering superior PDF handling capabilities.
Filimonov focused on a persistent problem for Retrieval-Augmented Generation (RAG) systems – frameworks that combine LLMs with external knowledge retrieval – namely, accurately citing information within lengthy documents. He described testing models for nearly two years on their ability to pinpoint the exact location (bounding box) of a text excerpt within a PDF page image.
Previous attempts using other models yielded “abysmal” results, he wrote, until testing Gemini 2.5 Pro. On his internal evaluation, the model achieved an Intersection over Union (IoU) score – a metric measuring the overlap between the predicted bounding box and the actual one – of 0.804 for this specific task, indicating a strong grasp of where text sits visually on the page. Filimonov concluded this makes “precise, visual PDF citations… a reality.”
Decoding Document Designs
Google’s own developer documentation lends support to this observation. It confirms Gemini models process PDFs using “native vision,” allowing them to interpret content beyond mere text extraction, including diagrams, charts, tables, and overall layout.
This capability is aided by the model’s large 1 million token context window, allowing it to ingest and analyze lengthy documents effectively. The Gemini API documentation details functionalities like analyzing these visual elements, extracting structured information, answering questions based on combined text and visuals, and transcribing PDFs into other formats while attempting to preserve the original layout.
Technical specifications via Vertex AI note the model can handle up to 3,000 PDF files per prompt, with individual files up to 1,000 pages or 50MB. Some third-party commentary, like a post on The Prompt Engineering Substack, specifically notes this “Native PDF Support” as overcoming prior challenges in parsing complex document elements.
However, Google also explicitly cautions about the model’s precision in this area. Official documentation lists “Spatial reasoning” as a limitation, stating, “The models aren’t precise at locating text or objects in PDFs. They might only return the approximated counts of objects.”
This suggests that while Gemini 2.5 Pro shows promise in understanding layout for certain tasks, like the one Filimonov tested, achieving pinpoint accuracy for all spatial queries within a document remains an area under development, potentially leading to inconsistencies for users seeking exact locations.
Competitive Context and Rollout Realities
This development doesn’t exist in a vacuum. Competitor Anthropic introduced a “Visual PDFs” capability for its Claude 3.5 Sonnet model back around November 2024, allowing it to analyze mixed content within documents, though primarily for paid users or via API with different technical limits.
Google’s move to offer Gemini 2.5 Pro’s potentially similar, if officially limited, skills to free users represents a different approach to market accessibility.
The rapid deployment of Gemini 2.5 Pro to the public occurred amidst wider activity and some scrutiny. Google pushed the model out broadly before releasing detailed safety documentation. An initial “model card” published around April 16 drew criticism from AI governance specialists like Kevin Bankston at the Center for Democracy and Technology, who termed it “meager” and worried about a “troubling story of a race to the bottom on AI safety and transparency as companies rush their models to market.”
Google’s stated policy in the card is that “A detailed technical report will be published once per model family’s release…after the 2.5 series is made generally available.” This context of rapid iteration also saw the preview launch of Gemini 2.5 Flash on April 18, a model first discussed publicly on April 9 and optimized for speed and cost-efficiency via controllable reasoning, distinct from the high-capability focus of the Pro version.
Performance Profile
Beyond PDF handling, Gemini 2.5 Pro’s general capabilities, built on a 1 million token context window (with 2 million planned according to Google’s March 25th announcement), include strong performance in multimodal reasoning (scoring 81.7% on MMMU benchmarks) and complex mathematics (92.0% on AIME 2024).
Yet, it faces stiff competition, trailing models like GPT-4.5 in certain factual recall tests (52.9% on SimpleQA vs. GPT-4.5’s 62.5%) and Anthropic’s Claude 3.7 Sonnet in autonomous coding exercises. This positions Gemini 2.5 Pro as a powerful and versatile model with specific strengths, particularly in multimodal and long-context tasks, but one whose performance varies depending on the specific application domain when measured against its top rivals in the rapidly evolving field.
