Apache Spark for Azure HDInsight Reviews

The best part about using Spark on HDInsight is the seamless integration within the Azure ecosystem. It allows for effortless scaling of compute power to handle massive datasets. The managed nature of the service means I don't have to worry about the underlying infrastructure overhead, and the Jupyter Notebook integration makes iterative development and data exploration extremely efficient for our engineering team

One downside is the cluster spin-up time, which can feel slow when you need immediate compute for ad-hoc tasks. Additionally, while it is a robust managed service, the pricing can escalate quickly if clusters are not managed or scaled down properly during idle times. It also feels slightly less 'modern' compared to newer alternatives like Azure Databricks, particularly regarding UI and collaborative features

We use it primarily for heavy ETL (Extract, Transform, Load) processes and large-scale data processing that traditional SQL environments can't handle. It has significantly reduced our processing time for daily data pipelines.

I enjoy the environment that the data lives in. Its really clean and intuitive

The way you input the data is easy enough for all users to do and be able to incorporate the data

Have a open mind integrating solution

makes it easy to create and configure a spark cluster in

I like the dynamism nature of it by having ability to support multiple languages.

There is no file management system in the system and do needy for integration with other platforms.

Guarantees very large loads of data processing, performance is that incredible and promising.

provides fault tolerance by handling the failure of any worker node in the cluster. Thus, it ensures that the loss of data is reduced to zero.

Apache provides quick overview of data analytics and cluster computing using in memory processing. we use Resource management to create the HDinsight spark cluster. it is good for faster data analytics.

I wish there would be quick solution with one click less then 10 second processing window because all the customers are expecting for the faster result as you know Cloudera is working towards it.

It is good to use in order to understand the in-depth what runs inside.

We mostly work on running the Data analytics, It is a good connecting point with the data.

.What is most helpful about Apache spark for azure hdinsight is the options it provides to make to do easy

8 times I feel like the their process was not user-friendly

Suggest doing your research

Apaches work helped with my problems with the eds rhd insight by providing an easier way to collect data into get the information more from 4/5

Its very easy to implement and integrate the spark jobs on Azure HDInsight cluster and lot of options to load or store your data building spark data pipelines.

The configurations or creating links for various sources to load data.

To make use of the power of on demand cluster creation and completely/efficiently using the resources for your spark applications.

As part of our POC our entire application was deployed on Azure HDInsight and AWS to evaluate the ease of use and finally we went a head with Azure HDInsight due to its drag and drop functionality and easy approach to spin on demand HDInsight cluster.

The most important thing about Apache Spark is that 10X faster than another data analysis platform as Hadoop.

There is nothing in particular, expect how to deal with data object that requires a different approach as you build your data processing workflow.

It has allowed me to analyze streaming data and get fastly my output.Especially when it comes to apply machine learning algorithms.

Simple tool, easy to install. Doesn't take too much to setup and start running codes.

The main disadvantage is the expenses incurred in running the programs in cloud.

Useful when you have sufficient free credits available for testing out cases before you decide to implement it.

Handling multiple data sources and running big data queries.

.Absolutely its in memory processing framework

I dont see anything like that but varies if it fits all use cases

Solving organizational data related problems like cleansing and processing the data