Video Breakdown Background Microsoft Official

In August, we talked about Video Breakdown, the latest Microsoft Garage project. At the time the service was not official, but that changes today. Microsoft took the wrappers off the concept today, making it available to everyone.

Video Breakdown uses machine intelligence to analyze a video content using Cognitive Service and Azure Media Analytics. The service also uses in other Azure products (Websites, Blob storage, Search, and Media Services.

From the analysis, the Microsoft Garage project will produce details on the content. This report includes face tracking, audio transcripts, identification, different speakers, and even sentiments of subjects.

The service will achieve that by using cognitive insights. More specifically, Video Breakdown will use intelligent features like OCR, Linguistic Transcript, and Face Detection. The software will make finding video content easier by using these intelligent features to “breakdown” the video.

Key features of Video Breakdown:

  • Linguistic Transcript – Convert audio to text based on acoustic language models
  • Face Detection – Find when does each face appears in the video
  • Speaker Diarization – Map and understand who spoke when
  • OCR – Extract text that appears in video as overlay, slides or background
  • Face Identification – Understand who is the person that appears as face
  • Voice Activity Detection – Separate background noise and voice activity
  • Contextual Search – Understand the context of search results
  • Sentiment Analysis – Understand the level of positive vs negative spoken or written content

Microsoft Garage

Microsoft Garage is known for allowing teams within the company to collaborate. They can create things that Microsoft would not otherwise make and work with people from other teams. Many Garage projects take some features from existing Microsoft services. Video Breakdown certainly draws some comparisons with Azure video services.

The Garage division has churned out some excellent concepts and apps this year. However, Video Breakdown may be the most dynamic and innovative of them all. You can head to the official preview for the service at the source.