HomeWinBuzzer NewsBaidu Topples Microsoft to Lead GLUE Natural Language Processing Benchmark

Baidu Topples Microsoft to Lead GLUE Natural Language Processing Benchmark

Baidu has confirmed its ERNIE natural language processing system has beaten Microsoft’s MT-DNN-SMART to top GLUE.


Chinese tech giant has toppled both and in terms of natural language processing. Specifically, the company's model now sits atop the leaderboard of the General Language Understanding Evaluation (GLUE) benchmark.

GLUE is a training, evaluation, and analyzing benchmark platform for natural language understanding services. Developers of the technology can test their models on GLUE to gain better understanding of performance.

Microsoft's MT-DNN-SMART has been the standout natural language processing system. If has topped GLUE, pushing Google's T3 system into second place.

However, there's a new winner in town as Baidu now tops the standings. The company's ERNIE (Enhanced Representation through kNowledge IntEgration) system managed a benchmark score of 90.1.

In a blog, Baidu promised to continue development to further prefect natural language understanding:

“While language understanding remains a difficult challenge, our results on GLUE indicate that pre-training language models with continual training and multi-task learning are a promising direction for NLP research. We will keep improving the performance of the ERNIE model via the continual pre-training framework,” wrote Baidu Research team.


ERNIE has already been successful in real-world scenarios. Baidu uses it in its search engine for answering spoken searches. The company claims the model has driven a 16% increase in customer satisfaction in terms of search results.

“ERNIE is a continual pre-training framework that builds and learns incrementally by pre-training tasks through sequential multi-task learning. We introduced ERNIE 1.0 early this year and released the improved ERNIE 2.0 model in July. The latter outperformed Google's BERT and Carnegie Mellon University's XLNet – competing pre-training models – in 16 NLP tasks in both Chinese and English.”

Luke Jones
Luke Jones
Luke has been writing about all things tech for more than five years. He is following Microsoft closely to bring you the latest news about Windows, Office, Azure, Skype, HoloLens and all the rest of their products.

Recent News