Meta Uses OpenAI´s GPT-4 as Own Llama AI Models Are Not Good Enough

Meta’s approach to generative AI is marked by a dual reliance on its proprietary Llama model and OpenAI’s GPT-4.

Fortune reports that Meta uses a pragmatic strategy for Metamate, Meta’s internal AI coding assistant, launched in early 2024. The tool is using OpenAI´s GPT-4 in a hybrid functionality. This highlights the challenges of achieving self-sufficiency in generative AI, even as Meta positions Llama as a cornerstone of its innovation efforts.

Mark Zuckerberg, Meta’s CEO, has consistently touted Llama’s capabilities, claiming it is “competitive with the most advanced models and, in some cases, leading and projecting that Llama will “the most advanced open model in the industry next year.” However, the integration of GPT-4 into key tools like Metamate reflects the real-world complexities of deploying advanced AI systems at scale.

Metamate: A Hybrid AI Tool for Developers

Metamate exemplifies Meta’s reliance on multiple models to address the diverse needs of its developers. The tool dynamically alternates between GPT-4 and Llama based on the complexity of coding queries.

Employees familiar with Metamate describe it as a helpful assistant for basic tasks but acknowledge its limitations with more advanced engineering. One employee characterized it as “at least as good as an intern,” emphasizing its utility for repetitive programming but limited application in solving intricate challenges.

While Metamate’s integration of GPT-4 ensures robust performance, it also underscores the limitations of Meta’s proprietary Llama model. This stands in contrast to the ambitious claims made during the release of Code Llama, a specialized extension of Llama 2 introduced in August 2023.

Code Llama was designed to handle tasks like debugging, code generation, and documentation. It supports programming languages including Python, Java, and C++ and processes up to 100,000 tokens of context, enabling it to work effectively with extensive codebases.

Code Llama performed competitively on industry benchmarks, achieving scores of 53.7% on HumanEval and 56.2% on Mostly Basic Python Programming (MBPP). Despite these achievements, Meta’s continued reliance on GPT-4 for tools like Metamate highlights the difficulties of scaling Llama to meet the practical needs of enterprise users.

Google’s Gemini and the Expanding AI Coding Tools Landscape

While Meta refines its hybrid strategy, competitors like Google are also advancing in the AI coding space. Microsoft-owned GitHub is currently leading the AI coding space with GitHub Copilot in a market that Google is trying to capture as well.

Recently GitHub launched a new AI-powered code review tool for GitHub Copilot, offering developers a faster and more efficient way to iterate on code. And Microsoft continues to embed GitHub Copilot in its ecosystem of developer tools, with a plethora of new integrations and features announced just recently.

Google on the other side, recently launched Gemini Code Assist Enterprise, an AI tool designed to support enterprise software development.

Like GitHub Copilot, Gemini integrates with popular integrated development environments (IDEs) like Visual Studio Code and JetBrains, where it offers advanced features such as context-aware code suggestions, function generation, and unit test creation.

What sets Gemini apart is its ability to analyze the context of a developer’s local codebase. This feature enables more tailored guidance compared to generic code completion tools.

Gemini also supports customization for organizations, allowing code suggestions to align with internal standards. Developers can use Gemini across Google Cloud services, including Firebase and BigQuery, where it assists with SQL and Python queries to accelerate data analysis.

With pricing starting at $19 per user annually, Gemini aims to attract enterprises of all sizes. Google’s aggressive push into this space aligns with industry projections from Gartner, which suggest that nearly all enterprise developers will rely on AI tools for coding by 2028.

Criticism of AI Openness and Transparency

Meta has positioned Llama as an open-source alternative to leading models like GPT-4. However, this claim has faced scrutiny. A study by Radboud University criticized Meta for promoting openness while withholding critical training data, stating that Llama 2 fails to meet most criteria for openness.

This critique highlights a broader industry trend where companies selectively release information about their AI models while maintaining tight control over key datasets.

The tension between claims of transparency and proprietary strategies is not unique to Meta. OpenAI, Google, and Amazon have faced similar criticism as they balance innovation with competitive pressures. However, Meta’s reliance on GPT-4, despite its open-source rhetoric, underscores the practical limitations of achieving a fully open AI ecosystem.

Meta’s dual reliance on GPT-4 and Llama might reflect a pragmatic and wise approach amid the complexities in deploying generative AI tools. While Zuckerberg’s vision for Llama emphasizes independence and innovation, the inclusion of GPT-4 in tools like Metamate reveals the challenges of meeting diverse enterprise needs.