Google DeepMind has started a new initiative to create advanced artificial intelligence (AI) systems capable of simulating physical and virtual environments.
Tim Brooks, a former researcher at OpenAI, now leads the effort, which focuses on “world models”—AI systems designed to predict and interact with real-world dynamics. In a post on X, Brooks stated, “DeepMind has ambitious plans to make massive generative models that simulate the world.”
This project is closely tied to Google’s larger strategy to advance artificial general intelligence (AGI). World models are seen as a foundational step in achieving AGI, a form of AI capable of performing any intellectual task that a human can.
The new team will collaborate with existing DeepMind projects, including the Gemini multimodal AI model, the Veo video generation platform, and Genie, an environment generator for interactive 3D simulations.
DeepMind has ambitious plans to make massive generative models that simulate the world. I'm hiring for a new team with this mission. Come build with us!https://t.co/pqvALtAvLs https://t.co/vtwgeXl9Dl
— Tim Brooks (@_tim_brooks) January 6, 2025
AI World Modeling
World models represent a significant departure from traditional AI systems, which primarily react to data inputs. Instead, these models simulate complex environments by analyzing multimodal data, such as text, images, and videos. This predictive capability enables applications in various fields, from robotics training to interactive gaming.
A job description for the new team highlights the broader goals: “We believe scaling pretraining on video and multimodal data is on the critical path to artificial general intelligence. World models will power numerous domains, such as visual reasoning and simulation, planning for embodied agents, and real-time interactive entertainment.”
By simulating real-world dynamics, world models provide a virtual sandbox for testing and learning, enhancing AI’s ability to adapt and respond in real-world scenarios.
DeepMind’s Genie project offers a glimpse into the possibilities. Launched in December, Genie 2 can generate playable 3D worlds based on user prompts. Demonstrations included a sailing expedition simulation and a cyberpunk-themed Western, showcasing the platform’s versatility in creating interactive environments.
The work on world models is inherently complex, requiring cutting-edge infrastructure and vast computational resources. DeepMind’s job offer for a Research Engineer role in world modeling outlines the technical challenges involved. Responsibilities include:
- Training large-scale multimodal transformers capable of analyzing diverse data types.
- Building infrastructure for video data pipelines, ensuring efficient curation and annotation.
- Optimizing inference systems for real-time applications, enabling seamless interactivity.
- Developing quantitative evaluation metrics to measure physical accuracy and intelligence.
- Exploring ultra-long-context transformers, which allow AI to analyze extended sequences of data.
The emphasis on scaling reflects a commitment to making these systems both robust and efficient. DeepMind’s philosophy, summarized as the key responsibilities in the job description, underscores this approach:
“Implement core infrastructure and conduct research to build generative models of the physical world. Solve essential problems to train world simulators at massive scale, develop metrics and scaling laws for physical intelligence, curate and annotate training data, enable real-time interactive generation, and study integration of world models with multimodal language models. Embrace the bitter lesson and seek simple methods that scale, with emphasis on strong systems and infrastructure.”
Applications and Implications
World models have diverse applications across industries. In robotics, they enable the creation of virtual environments where machines can learn to navigate and manipulate objects. This reduces the time and cost of physical testing.
Genesis, an open-source physics simulation platform developed by Carnegie Mellon University and private industry researchers, shows how AI systems can be trained for 3D physics in a completely virtual environment much faster than in the real world.
In gaming, world models create immersive experiences with dynamic, responsive environments. The technology also has potential in healthcare, where simulations could assist in diagnostics and personalized treatment planning.
Despite their promise, these advancements come with challenges. Ethical concerns loom, particularly regarding the displacement of workers. The Animation Guild estimates that over 100,000 U.S.-based jobs in film, television, and animation could be affected by AI technologies by 2026.
Legal issues also arise, as some world models rely on unlicensed video game footage for training. While Google asserts that its practices comply with YouTube’s terms of service, it has not disclosed specific data sources.
Competition in the AI Space
DeepMind’s initiative positions Google in a competitive race with other major players. Nvidia’s new Cosmos platform focuses on physical AI and robotics, while Fei-Fei Li’s World Labs develops large-scale world models with spatial intelligence for diverse applications. Startups like Odyssey and Decart are also making strides, contributing to the growing field of AI world simulations.
DeepMind’s access to Gemini AI, Veo, and Genie offers a unique advantage. By integrating these systems, the team aims to create AI that not only predicts outcomes but also adapts to changing scenarios in real time. This capability can be critical for achieving AGI, where adaptability and generalization are key.
DeepMind’s Vision for AGI
While artificial general intelligence remains a distant but achievable goal, world models are a crucial step on this path. By simulating physical and virtual environments, these models provide a foundation for AI systems that can reason, plan, and interact like humans.
The Research Engineer job description captures the essence of DeepMind’s vision: “World models will power numerous domains, such as visual reasoning and simulation, planning for embodied agents, and real-time interactive entertainment.”
