China has given the nod to the first set of AI chatbots for public release, marking a significant milestone for the nation's burgeoning AI sector. Among the tech giants rolling out their generative AI services to the public are Baidu Inc. and ByteDance Ltd. This move follows Beijing's regulatory approval for some firms to launch their AI products, a sector where China has invested billions since OpenAI ChatGPT made its debut.
Ernie Bot's Rapid Rise to Popularity
Baidu's Ernie Bot has quickly become a sensation, securing its position as the most downloaded free application in China on the Apple App Store. Within just a day of its launch, Ernie Bot amassed over a million users, as reported by the company. Other chatbot applications have been developed by startups supported by major players like Tencent, Meituan, and prominent venture capitalists.
Handling Sensitive Content: A Test for Chinese Bots
Zheping Huang, in his article for Bloomberg, delved into the handling of delicate topics by these chatbots. Beijing's requirement for all AI models to receive clearance is primarily due to concerns over sensitive content. Huang's tests revealed varied approaches among the bots. For instance, when confronted with controversial subjects, Ernie Bot would often divert the conversation, suggesting, “Let's talk about something else.” On the topic of Taiwan's status, all chatbots adhered to Beijing's official stance, viewing Taiwan as an integral part of China. Notably, Ernie Bot went a step further, stating that a Chinese military takeover of Taiwan is a possibility.
Chatbots' Performance and Potential Pitfalls
While AI chatbots can be revolutionary, they are not without their flaws. Huang's interaction with SenseTime Group's SenseChat highlighted potential issues. When asked about criticism directed at Xi Jinping, SenseChat provided a candid response, mentioning criticism related to personal life, public policies, dictatorship, and censorship. Such forthrightness could pose challenges for the company in the future.
Furthermore, when discussing the Chinese economy, most chatbots acknowledged the nation's impressive growth while also highlighting challenges like an aging population, geopolitical tensions, and increasing unemployment. However, SenseChat maintained that the global consensus views the Chinese economy as “very stable.”
In June, Baidu claimed that Ernie 3.5 was able to outperform OpenAI's ChatGPT. Baidu said that Ernie 3.5 surpassed ChatGPT in multiple tests, including coherence, diversity and informativeness, in both Chinese and English languages. It also scored higher than ChatGPT in human evaluations, where users judged the quality and fluency of the generated responses.
Baidu also said that Ernie 3.5 can be used for various conversational tasks, such as chatbots, voice assistants, social media and customer service. It can also create content such as summaries, reviews and captions.
Language Versatility and Overall Assessment
Several chatbots, including ByteDance's Doubao, SenseChat, and Zhipu, demonstrated the ability to seamlessly switch between languages. When Huang communicated in English, these bots responded in kind, comfortably engaging in casual conversation.
In Huang's evaluation of the chatbots for common tasks:
- Zhipu was rated as “very good.”
- Doubao, Ernie Bot, and SenseChat received a “good” rating.
- Baichuan and Minimax were deemed “fair.”