Elon Musk, who owns X (previously known as Twitter) and the artificial intelligence company xAI, has disclosed that Grok 2 will launch in August, and Grok 3 will debut by the end of this year.
Training on Nvidia H100 GPUs
In his announcement about Grok 3, Musk noted the utilization of 100,000 Nvidia H100 GPUs for training. These GPUs are renowned for their performance in AI and machine learning, suggesting that Grok 3 will bring notable advancements to xAI’s model. The large GPU count reflects the project’s ambitious nature and hints at the potential capabilities of the new model.
Musk responded to Cohere CEO Aidan Gomez’s video addressing the issue of language models being trained on the outputs of OpenAI’s models, which can affect the uniqueness of the data. Gomez explained that his company’s models avoid this, providing a distinct user experience. Musk acknowledged the challenge of excluding language model outputs from training data on the internet and stated that Grok 2 aims to improve in this area. This indicates a focus on enhancing the authenticity and quality of Grok 2’s training data.
Sadly quite true. It takes a lot of work to purge LLMs from the Internet training data. Grok 2, which comes out in August, will be a giant improvement in this regard.
— Elon Musk (@elonmusk) July 1, 2024
Grok’s Competitive Position
Grok 1.5, the latest iteration released in March, featured enhanced reasoning and a context length of 128,000 tokens. Although it didn’t surpass GPT-4 in tests like MMLU, MATH, and GSM8K, it did outperform GPT-4 in the HumanEval benchmark. Despite these enhancements, Grok remains less popular than rivals like ChatGPT, Gemini, and Claude, partly due to not having a free version. Given the high operational costs of language models and Musk’s aim to boost X’s revenue, a free version of Grok seems unlikely in the near future.
Musk’s engagement with language models originates from his co-founding of OpenAI, although he later left due to internal disagreements. After ChatGPT’s release, Musk introduced Grok as a paid service on X, emphasizing its humorous elements to enhance its lifelike interaction. The impending releases of Grok 2 and Grok 3 are poised to push the boundaries of what these language models can accomplish.
Last Updated on November 7, 2024 3:43 pm CET
