According to rumors, Meta is in the process of developing Llama 3, a language model that aims to compete with OpenAI's GPT-4. This information was reportedly overheard by OpenAI engineer Jason Wei, previously associated with Google Brain, during a Generative AI Group social event hosted by Meta.
Wei's tweet on the matter suggests that Meta has the computational resources to train both Llama 3 and its successor, Llama 4. Notably, if Llama 3 achieves performance comparable to GPT-4, it will still be open-sourced, as indicated by a statement from a Meta representative: “Yeah we will. Sorry alignment people.”
Timeline and Comparisons
Historically, there was a five-month gap between the launch of Llama 1 in February 2023 and Llama 2 in July 2023. While Llama 2 has been optimized by the open-source community and has reached performance levels similar to GPT-3.5 in certain applications, there remains a significant performance disparity when compared to closed-source models like GPT-4 and Google's PaLM-2.
Meta's strategy of open-sourcing its models has garnered significant attention and praise from the developer community. However, the company's commitment to open source has been questioned, especially considering that the decision to open-source LLaMA was only made after the model was leaked.
Meta claims to be open about its AI, offering a community license for Llama 2. It thinks that openness leads to more innovation and safety in AI. The company invites the community to test Code Llama, find problems, and fix them.
But a recent study says that Meta and other companies are not really open about their AI models. The study, done by AI experts from Radboud University in Nijmegen, Netherlands, shows that some of the strongest AI LLMs are hidden from the public, because the code that trained them is not shared.
The study names OpenAI and Meta as the most closed LLM makers and says that this hurts the AI community. It asks for more honesty and openness from companies, so that others can learn from their work and make it better.
OpenAI's Dominance in the LLM Market
Despite Meta's efforts, OpenAI remains a dominant force in the large language model (LLM) market. Meta's primary objective with the Llama models is to challenge OpenAI's stronghold in this domain. OpenAI's CEO, Sam Altman, mentioned in June 2023 that the launch of GPT-5 is not imminent. Meanwhile, Google is planning to introduce Gemini, its next-generation multimodal LLM, either later this year or early next year.
The move to open-source powerful AI models has raised concerns in the tech community. OpenAI's Sam Altman has previously joked about carrying a “kill switch” for OpenAI's systems, highlighting the potential risks of AI systems going rogue. With the possibility of Meta open-sourcing a GPT-5 level model, concerns arise about the lack of a “kill switch” for such models, potentially allowing malicious actors to weaponize them.
Despite the rise of open-source models, OpenAI's influence remains evident. Most models in the open-source ecosystem are benchmarked against GPT-3 and GPT-4, particularly on the HumanEval benchmark developed by OpenAI. Even if Meta releases an open-source Llama 3 model with capabilities on par with GPT-4, it will likely still be evaluated using HumanEval. Furthermore, OpenAI, in collaboration with Anthropic, Google, and Microsoft, has launched the Frontier Model Forum to ensure the safe and responsible development of AI models.