Microsoft has this week announced TensorWatch, a new open source suite of debugging tools for machine learning training. Microsoft Research says current tools are focused on logging what users see because logging is expensive. Engineers mostly avoid accurate logging, but with TensorWatch they can leverage tools to expediate logging.
In a blog, Microsoft Research says it has been seeking a solution to reduce the workload and receive more system information. The company describes TensorWatch as “the Swiss Army Knife of debugging tools”.
Customization is important in the solution, so Microsoft uses Jupyter Notebook instead of a prepackaged interface. Real-time machine learning training debugging is available in TensorWatch through composable UI Jupyter Notebooks and live dashboards in Jupyter Lab. Microsoft built TensorWatch as a Python library, users can also create their own UIs.
Streams are a major part of the TensorWatch experience, with data treated as streams within the service. Files, cloud storage, visualizations, and more are allow used as streams:
“TensorWatch trivially allows you to implement a variety of advanced scenarios. For example, you can render many streams into the same visualization, or one stream can be rendered in many visualizations simultaneously, or a stream can be persisted in many files, or not persisted at all.”
More Tools
Lazy Logging Mode is a feature within TensorWatch. Users can automate logging information by allowing the tool to watch variables. As a free service, users can watch multiple variables during machine learning training. Once complete, the service allows interactive queries that runs within the context of the variables and results in a stream.
“At Microsoft Research, we care deeply about improving debugging capabilities in all phases of model development—pre-training, in-training, and post-training. Consequently, TensorWatch provides many features useful for pre- and post-training phases as well. We lean on several excellent open-source libraries to enable many of these features, which include model graph visualization, data exploration through dimensionality reduction, model statistics, and several prediction explainers for convolution networks.”
TensorWatch is now available as an open source project on GitHub here.
