Researchers at Tsinghua University in Beijing have presented Taichi, a photonic AI chiplet architecture detailed in an recent publication in Science. Developed by Associate Professor Lu Fang from the Department of Electronic Engineering and Professor Qionghai Dai from the Department of Automation, the chip targets the escalating energy demands of large-scale artificial intelligence.
The team reported an energy efficiency of 160 tera-operations per second per watt (TOPS/W), a key metric for AI processing power relative to energy consumed. According to analysis by China Academy, this represents a notable improvement over comparable photonic neural network chips developed previously.
Tackling the AI Energy Problem With Light
Modern AI models require immense computational power, leading to significant electricity consumption and hardware needs, presenting a bottleneck for further advancements. As IEEE Spectrum highlighted in its coverage, the training costs for models like OpenAI’s GPT-3 underscored the energy challenges of conventional electronics. Photonic computing, using light for calculations, has been explored for its efficiency potential, but prior efforts faced scaling difficulties due to noise amplification.
The development of Taichi also occurs amidst US trade controls restricting access to AI accelerators like Nvidia’s H100 and H20 GPUs in China, adding context to the search for alternative hardware. Comparisons cited by IEEE Spectrum suggest Taichi operates with potentially over 1,000 times the energy efficiency of the H100 chip.
In future domestic AI chips, Chinese producers of AI chips could leverage the technology to close the gap with Nvidia and other rivals. Huawei just these days unveiled the CloudMatrix 384 AI cluster which beats Nvidia’s leading GB200 NVL72 architecture in performance, but at the cost of much higher power comsumption.
A Hybrid, Distributed Photonic Design
Taichi employs a distinct strategy combining light diffraction and interference techniques within its chiplets, aiming to merge the high density of diffraction with the reconfigurability of interference.
This work builds upon the team’s prior research, including an optical chip named OPCA focused on rapid image processing. Instead of using deep optical layers, which can amplify errors, Taichi uses a distributed architecture. Lu Fang described this to IEEE Spectrum as a “‘shallow in depth but broad in width’ architecture [that] guarantees network scale.”
Complex AI tasks are divided across parallel chiplets, enabling the system to manage large networks. The Science abstract notes “millions-of-neurons capability” stemming from its 13.96 million parameters, while the university’s announcement framed the effective scale as supporting “billions” of neurons.
The architecture boasts a computational density reported at nearly 880 trillion MACS/mm², supported by chiplets with input/output dimensions as large as 64×64. Funding for the project included support from China’s Ministry of Science and Technology and the National Natural Science Foundation of China, according to data associated with the study.
Demonstrated Capabilities and Practical Hurdles
The Tsinghua team validated Taichi on complex AI benchmarks. It achieved 91.89% accuracy classifying the 1,623 categories in the Omniglot handwritten character set and 87.74% accuracy on the 100-category mini-ImageNet task.
The chip also powered AI models for content generation, producing music in the style of Bach and images emulating Van Gogh and Munch. “Optical neural networks are no longer toy models,”
Lu Fang asserted in the IEEE Spectrum interview. “They can now be applied in real-world tasks.” However, while the chiplet itself is compact, the complete system currently requires substantial external equipment. Lu Fang noted to IEEE Spectrum that components like the laser source and data interconnects remain bulky, occupying significant lab space (“almost a whole table”). Future work aims to integrate these functions more closely, targeting applications in large AI models, content generation, and robotics.
