Japan is a global leader in technology, but it is falling behind in the field of generative artificial intelligence (AI), which can create text and other content from massive data sets. Unlike the US, China, and the EU, which have powerful chatbots such as ChatGPT and Bing Chat, or Baidu's Ernie, Japan lacks the skills and resources to develop large language models (LLMs), the backbone of generative AI.
One reason for this gap is that Japan does not have enough software engineers to build the infrastructure and applications needed for deep learning, the technique behind LLMs. According to Japan's Ministry of Economy, Trade and Industry (METI), the country will face a shortage of nearly 800,000 software engineers by 2030.
Another reason is that Japan does not have its own AI supercomputers, such as IBM's Vela or Microsoft's Azure-hosted system, which can train LLMs efficiently. Instead, it has to rely on foreign machines or cloud services.
However, Japan is not giving up on generative AI. It has several initiatives to boost its capabilities and catch up with its rivals. For example, METI plans to introduce a new supercomputer through its affiliated laboratory, the National Institute of Advanced Industrial Science and Technology (AIST), as early as 2024.
Creating an In-House Solution to Develop AI
The new machine will have a computing power 2.5 times greater than AIST's current one and will be available to Japanese companies developing generative AI via a cloud service. AIST will also establish a new research center for supercomputers and quantum technologies in July, with a budget of 32 billion yen (US$226 million).
In addition, the Tokyo Institute of Technology and Tohoku University are collaborating with Fujitsu and Riken, the developers of the Fugaku supercomputer, to create LLMs based on Japanese data. They aim to publish their results in 2024 and share them with other researchers and engineers in Japan.
Furthermore, the Japanese government plans to invest 6.8 billion yen (US$48.2 million) to build a new supercomputer in Hokkaido that will specialize in LLM training and start operating in 2024. The supercomputer, assembled by cloud service provider Sakura Internet, will have more than 2,000 graphics processing units from Nvidia and will be able to develop GPT-4 level AI capabilities within a year.
Japan is determined to overcome its challenges and become a leader in generative AI. With its new supercomputers and LLMs, it hopes to unleash its creativity and innovation in the digital age.
Will Japan Be Able to Compete in AI Development?
While Japan is moving towards developing AI through supercomputers on home soil, the country is increasingly falling behind. Certainly, Japan will be competing in a field that is already progressing at a rapid pace. It is not just the AI models that are becoming mainstream, but also the AI supercomputers that build those models.
Just this week, Cerebras Systems, a pioneer in accelerating generative AI, and G42, a UAE-based technology holding group, announced Condor Galaxy, a network of nine interconnected supercomputers designed to significantly reduce AI model training time. The first AI supercomputer on this network, Condor Galaxy 1 (CG-1), boasts 4 exaFLOPs and 54 million cores.
Most powerful AI supercomputers after Cerebras
- Summit: This AI supercomputer is a joint project of IBM and Oak Ridge National Laboratory. It can perform 200 quadrillion calculations per second and handle 3.3 billion gigabytes of data. Summit helps scientists and doctors explore various topics, such as climate change, drug discovery, and genomics.
- Sunway TaihuLight: This AI supercomputer is made by China's National Research Center of Parallel Computer Engineering and Technology. It can perform 125 quadrillion calculations per second and process 10.65 billion gigabytes of data. Sunway TaihuLight is mainly used for industrial and engineering purposes, such as weather forecasting, oil exploration, and aerospace design.
- Selene: This AI supercomputer is created by NVIDIA and hosted by the New Mexico Consortium. It can perform 63 quadrillion calculations per second and store 1.6 billion gigabytes of data. Selene supports NVIDIA's research and development in AI, such as natural language processing, computer vision, and recommender systems.
- Andromeda: This AI supercomputer is built by Cerebras. It has 13.5 million cores that can achieve speeds over a quintillion calculations per second. Andromeda is designed specifically for AI and has shown remarkable efficiency in scaling up AI workloads for large language models.
- IBM Vela: This AI supercomputer is IBM's first cloud-native supercomputer optimized for AI. It is located within IBM Cloud and is used by the IBM Research community. Vela's design allows it to scale up easily and deploy similar infrastructure into any IBM Cloud data center around the world.
- DGX SuperPOD: This AI supercomputer is designed by NVIDIA for enterprise-scale AI infrastructure. It is powered by NVIDIA A100 Tensor Core GPUs and delivers 700 quadrillion calculations per second of AI performance.
- Meta's AI Research SuperCluster (RSC): This AI supercomputer is designed by Meta to speed up AI research. It is one of the fastest AI supercomputers in the world and is used to train large AI models, including natural language processing and computer vision models.
