Microsoft has announced a new feature for its Edge browser at Build 2024 today that will translate and dub videos in real-time. This capability will be available on popular platforms such as YouTube, LinkedIn, and Coursera. The announcement was made during this year's Build event, highlighting the integration of AI to enhance user experience.
Supported Languages and Accessibility
Initially, the feature will support translations from Spanish to English and from English to German, Hindi, Italian, Russian, and Spanish. This development not only caters to users who prefer consuming content in their native language but also aims to make videos more accessible to individuals who are deaf or hard of hearing. The real-time translation will be available through both dubbing and subtitles.
Expansion to News Sites
In addition to educational and social media platforms, Edge will extend this functionality to news websites, including Reuters, CNBC, and Bloomberg. Microsoft has indicated plans to expand the range of supported languages and websites over time, making this feature more versatile and widely applicable.
This new translation feature builds on Microsoft Edge's existing suite of AI capabilities, which includes integration with Copilot. Edge already offers the ability to summarize YouTube videos, although it relies on the availability of video transcripts to generate these summaries. This new feature represents a significant enhancement to the browser's AI toolkit.
Historical Context of AI Translation in Microsoft Products
Microsoft has a track record of incorporating AI translation into its products. Since 2018, Microsoft Teams has featured inline message translation, and in 2022, live translation for captions was introduced for Teams Premium subscribers. The Edge browser has supported the translation of web pages and selected text since 2020. Additionally, the Windows 11 2022 Update brought AI-generated captions to streamed and pre-recorded videos, although without translation capabilities.
Dubbing, which involves replacing spoken audio with an AI-generated voice in the user's native language, is a new addition to Microsoft's suite of features. This process can be resource-intensive. For example, services like synthesys.io offer limited audio/video generation for $20 per month, while ElevenLabs.io provides free translation services with certain limitations. This indicates the potential cost and complexity involved in implementing such a feature.
The automatic translation of web pages is already a well-regarded feature in web browsers. The addition of real-time translated or dubbed video content could set Edge apart from its competitors, potentially attracting more users to the browser. This new capability underscores Microsoft's commitment to leveraging AI to improve accessibility and user experience across its products.
