Microsoft has introduced the DeepSpeed4Science initiative through its DeepSpeed team. The initiative focuses on the application of deep learning in the natural sciences, targeting areas such as drug development and renewable energy. The DeepSpeed system, an open-source AI framework from Microsoft, plays a central role in this new venture, aiming to enhance the speed and scale of deep learning processes.
DeepSpeed4Science's Design and Collaborations
DeepSpeed4Science seeks to address the complexities specific to scientific discoveries, extending beyond the capabilities of the current DeepSpeed system. DeepSpeed empowers automatized ChatGPT-like model training, offering 15x speedup over SOTA RLHF systems with unprecedented cost reduction at all scales. SOTA RLHF stands for state-of-the-art reinforcement learning from human feedback systems, a type of AI system that learns to perform tasks by interacting with a human “teacher”. The teacher provides feedback to the system, which then uses that feedback to improve its performance.
SOTA RLHF systems have been used to achieve impressive results on a variety of tasks, including playing video games, writing different kinds of creative content, and answering your questions in an informative way. However, they can be computationally expensive to train, and they often require a large amount of human feedback.
DeepSpeed is a system that optimizes that kind of training of large language models. It can be used to train SOTA RLHF systems, although it does not eliminate completely the need for human feedback.
The DeepSpeed4Science initiative is in collaboration with various teams that specialize in AI-driven science models, spanning domains like climate science and molecular dynamics simulation. The goal is to develop DeepSpeed4Science as a platform for sharing AI technologies that support scientific research, in line with Microsoft's “AI for Good” ethos.
Key Features and Partnerships
DeepSpeed4Science has garnered support from several key science models from Microsoft Research AI4Science, Microsoft WebXT, Bing, and U.S. DoE National Labs. One notable collaboration is with the Scientific Foundation Model (SFM) from Microsoft Research AI4Science. This model aspires to establish a comprehensive large-scale foundation model that bolsters natural scientific discovery. Another significant project is ClimaX, the first foundation model designed for a broad spectrum of weather and climate modeling tasks. This model can assimilate varied datasets, potentially enhancing weather prediction accuracy. Furthermore, the initiative is also working with external collaborators like Columbia University's OpenFold and Argonne National Laboratory's GenSLMs, both of which focus on structural biology research.
