Microsoft CTO Kevin Scott has revealed that forthcoming AI models can substantially surpass OpenAI GPT-4‘s reasoning power. According to Scott, the upcoming models demonstrated the capability to pass PhD qualifying exams, representing a leap from GPT-4's high school-level performance on Advanced Placement tests.
Scott disclosed these advancements during a Berggruen Salon event in Los Angeles, emphasizing improvements in both AI memory retention and reasoning skills.
Microsoft is OpenAI´s most important technology partner, providing the computing infrastructure to train upcoming models. Scott is working closely with OpenAI on the training of next-gen models, as highlighted by Sam Altman, who recently joined him on stage at Microsoft´s Build 2024 conference.
Sustained Memory Functionality
Scott pointed out sustained memory capacity as a notable improvement in the new AI models. Unlike existing AI models, which typically employ episodic memory and forget previous interactions, newer systems could preserve information over an extended period. This continuous memory retention enhanced the coherence of interactions and the efficiency in solving complex tasks.
He pointed out the significant strides made in reasoning. According to Scott, while GPT-4 performs on par with high school students on Advanced Placement exams, these new models can operate at the cognitive level of PhD students. This development allows the AI to handle more complex problems and yield more accurate solutions.
Scott described the new models' memory as “durable,” indicating a reliable capacity to retain information over long periods. This enhancement addresses the fragility of earlier models, resulting in more robust AI systems capable of handling complex tasks reliably. Enhanced memory and reasoning capabilities enable users to accomplish tasks more swiftly and effectively. This progress holds potential for various sectors by streamlining processes and improving decision-making efficiency.
Despite criticism from some like OpenAI CEO Sam Altman, who called GPT-4 “mildly embarrassing,” OpenAi´s model already has demonstrated remarkable real-world performance. For instance, GPT-4 has outperformed professional financial analysts in predicting future earnings trends without relying on qualitative data, underscoring the practical utility of such advanced AI.
A key highlight of these developments is the increased accessibility of AI tools. Scott noted that the barrier to entry has been considerably lowered, enabling high school students to perform tasks that once required significant mathematical and technical know-how. This broadens the potential user base and enables a broader range of individuals to tackle important problems using AI technology.
Study Finds GPT-4's Bar Exam Scores Overinflated
Recently, doubts about GPT-4´s reasoning capacity have emerged, pointing to methodically flawed benchmarks. A study published in the Artificial Intelligence and Law journal questions the reliability of GPT-4‘s performance on the Uniform Bar Exam (UBE). The paper suggests these figures, presented as conservative, may not accurately represent the model's genuine abilities.
When OpenAI released GPT-4 last year, it was said to have human-level performance on various professional and academic benchmarks, such as passing a simulated bar exam or writing creative stories.
