Microsoft has open-sourced yet another internal tool. This time it’s Trill, a data processing engine used in Bing, Azure, and even Halo. According to the company, Trill can process “trillions of events per day”.
“We believe there is no equivalent to Trill available in the developer community today,” said James Terwilliger, principal software engineer, Azure. “In particular, by open-sourcing Trill we want to offer the power of the IStreamable abstraction to all customers the same way that IEnumerable and IObservable are available. We hope that Trill and IStreamable will provide a strong foundation for streaming or temporal processing for current and future open-source offerings.”
The tool enables access extreme speeds with low latency while being easily accessible to any .NET app or service as a single-node engine library. It also has a temporal query language to “allow users to express complex queries over real-time and/or offline data sets.”
A Six-Year Journey
Trill has a long history at Microsoft, beginning as a research project in 2012 and eventually making its way into numerous products. Its customizability has enabled a wide range of use cases, with developers able to change the intervals of incoming data, extend native query functions, and more.
As well as Azure Stream Analytics and Halo, it enables products like Exchange, Azure Networking, and Windows 10’s controversial telemetry analysis. However, Microsoft is still working on projects with the engine:
- “Digital signal processing with the capability and performance normally seen in R or better.
- An improved ability to handle out of order data for allowing users to specify multiple levels of latency.
- Allowing operator state to be managed using the recently open-sourced FASTER framework.”
For those who don’t want to deploy Trill on its own, Microsoft is considering a ‘user defined operator model’ as part of Azure Stream Analytics.
You can read more about the release on the Azure blog or Trill’s GitHub repository.
