Kyle Corbitt, a notable figure in the tech industry with a background as a director at Y Combinator, an engineer at Google, and co-founder at Emberall, has shared insights into the future releases of two significant AI models, GPT-5 and Llama 3. According to Corbitt, who purportedly spoke to a Microsoft engineer involved in the development of GPT-6 for OpenAI, GPT-5 is expected to be launched in late April, with Meta’s Llama 3 following in June.
The tip comes just a week after previous reports that GPT-5 was nearing completion, with a summer launch in OpenAI’s sights. These developments suggest a rapid advancement in AI technologies, with Llama 3 anticipated to surpass the capabilities of its predecessor, Mistral.
GPT-5 AND LLAMA 3 TIMELINE LEAKS
Highest confidence rumors I've heard are late April for GPT-5 and early June for Llama 3 (expected to beat Mistral at all sizes).
Anyone hear something different? Reply in thread to confirm.
— Kyle Corbitt (@corbtt) March 25, 2024
Technical Challenges and Innovations
The conversation between Corbitt and the Microsoft engineer revealed intriguing details about the challenges faced during the development of GPT-6, particularly in terms of hardware infrastructure. The engineer highlighted difficulties in provisioning “infiniband-class links” between GPUs across different regions, a critical component for the efficient training of large AI models.
The attempt to centralize the training cluster in one region was thwarted by the limitation of not being able to deploy more than 100,000 H100 GPUs without risking the stability of the local power grid. This anecdote underscores the immense power requirements and logistical complexities involved in developing cutting-edge AI technologies.
Spoke to a Microsoft engineer on the GPT-6 training cluster project. He kvetched about the pain they're having provisioning infiniband-class links between GPUs in different regions.
Me: "why not just colocate the cluster in one region?"
Him: "Oh yeah we tried that first. We…— Kyle Corbitt (@corbtt) March 25, 2024
Market Implications and Future Outlook
The potential early release of GPT-5 and the subsequent launch of Llama 3 are poised to have significant implications for the AI market. OpenAI’s decision to release GPT-4 Turbo for free has already set a precedent for accessibility, and the upcoming versions are eagerly awaited by both the tech community and end-users. Meta’s plan to launch Llama 3 in July, albeit possibly sooner as suggested by Corbitt’s information, indicates a competitive landscape where major tech companies are vying to push the boundaries of AI capabilities.
These developments not only highlight the rapid pace of innovation within the AI sector but also raise questions about the sustainability and environmental impact of such advanced technologies. As companies continue to invest in the development of more powerful and efficient AI models, the balance between technological advancement and resource consumption remains a critical consideration.
Last Updated on November 7, 2024 9:25 pm CET
