The tools extend Eclipse to allow developers to create and develop HDInsight Spark applications and easily submit Spark jobs to Microsoft Azure HDInsight Spark clusters using the Eclipse development environment.
They integrate seamlessly with Azure and enables developers to navigate HDInsight Spark clusters and view associated Azure storage accounts.
To further boost productivity, they also offer the capability to view Spark job history and display detailed job logs.
Features in The HDInsight Tool For Eclipse:
- “Create Spark projects with built-in templates, sample code and intelligence to auto create artifacts and locate assemblies.
- Develop Spark applications with native authoring support (e.g. IntelliSense, auto format, error checking, etc.) from Eclipse (mainly Scala and Java).
- Submit Spark applications to HDInsight clusters.
- Allow user to stop a running Spark application before its completion.
- Test and validate Spark applications locally.
- View all the Spark clusters associated with a user’s azure subscriptions.
- Navigate the associated storage resources of user’s HDInsight Spark cluster.
- Cache the Spark logs to disk in case the Spark logs are too big.”
To get started with HDInsight Tools for Eclipse, install Eclipse and download the prerequisite files including Java SDK, Scala IDE for Eclipse and Microsoft HDInsight Spark Cluster.
Then, get the latest bits by going to the Eclipse repository and searching “Azure Toolkit for Java”. Eclipse will also prompt you for the latest updates if you have previously installed the Azure Toolkit for Java plugin.
The HDInsight Tools for Eclipse are a part of the Azure Toolkit for Eclipse and will be of particular interest to HDInsight Spark developers. It is an open source project as part of Azure Toolkit for Eclipse, whose source code is available under the MIT License at https://github.com/microsoft/azure-tools-for-java.