Chinese tech giant Baidu has toppled both Microsoft and Google in terms of natural language processing. Specifically, the company's model now sits atop the leaderboard of the General Language Understanding Evaluation (GLUE) benchmark.
GLUE is a training, evaluation, and analyzing benchmark platform for natural language understanding services. Developers of the technology can test their models on GLUE to gain better understanding of performance.
Microsoft's MT-DNN-SMART has been the standout natural language processing system. If has topped GLUE, pushing Google's T3 system into second place.
However, there's a new winner in town as Baidu now tops the standings. The company's ERNIE (Enhanced Representation through kNowledge IntEgration) system managed a benchmark score of 90.1.
In a blog, Baidu promised to continue development to further prefect natural language understanding:
“While language understanding remains a difficult challenge, our results on GLUE indicate that pre-training language models with continual training and multi-task learning are a promising direction for NLP research. We will keep improving the performance of the ERNIE model via the continual pre-training framework,” wrote Baidu Research team.
ERNIE has already been successful in real-world scenarios. Baidu uses it in its search engine for answering spoken searches. The company claims the model has driven a 16% increase in customer satisfaction in terms of search results.
“ERNIE is a continual pre-training framework that builds and learns incrementally by pre-training tasks through sequential multi-task learning. We introduced ERNIE 1.0 early this year and released the improved ERNIE 2.0 model in July. The latter outperformed Google's BERT and Carnegie Mellon University's XLNet – competing pre-training models – in 16 NLP tasks in both Chinese and English.”