Azure Data Factory Reviews

ADF is cloud based Data integration tool on azure cloud. Its the fastest and best ETL tool out there comapred to onpremise and other cloud based ETL tool like Informatica, IICS, Talend, IBM datastage etc. It support so many connectors and lot of file formats. you can do so mant data integration transformation using ADF with ease and drag and drop features. Also you can view your data activities and its transformed data in real time instead of navigating each time to database and files like Azure sql server, oracle, JSON etc. it has awsome integration with big data tools like databricks, synapse analytics and blob storage.

Microsoft done a great job to create this tool and there are nothing much but if the can provide more support to other different vendor connectors, databases for its transfromation activities as most of it only support to azure related storage. I mean there are few crucial activites like get metadata, until, stored procedure etc, although they are keep adding and supporting to new connectors in each update.

We are using ADF to do data collection, extraction, transformation and loading purpose. also we are using it for scheduling the data pipelines with our business time. we getting data from mutiple sources specially heathcare raw data and it need some rules and transformation before going to into databases for business to do analytics and reporting. previously we were using the on premise informatica etl tool to all this but ADF is just lot better.

Works very good with complex data set. Very nice data cloud solution.

I don't think I can say something I dislike

Highly scalable.

Implementing pipelines is very easy as it drag and drop

connectivity datasources which only allow JWT is challenging and some data scenarios cannot be implemented

Very effective and efficient tool for intergrating with mutiple Big data platforms. Helps resolve Big data management problems in a effective way.

The best thing in ADF is the data flow debug where we can direct check the data flow output in every task and find the errors with pipeline run.

Deployment to prod but it's not too dislike but it's hard to deploy to prod from dev . We have to use cl cd pipelines etc.

Creating a fully functional enterprise Data Warehouse and we are using Azure services and we have used ADF as ETL tool and it was very easy to implement SCD Types and other data integration.

It helps to orchestrate different jobs through pipeline creation.

Also, various environments can be connected easily through ADF, and the no code environment makes it easier.

Azure Data Factory doesnt allow you to send customized mails based on the failed activity. If any activity of a pipeline fails, it will send a mail that the pipeline has failed, but doesnt mention which activity has failed.

Its helps us to schedule jobs based on a specific time or event based. It also helps to orchestrate various activities and thus removies the manual dependency a lot.

Data Integration and Orchestration: ADF allows you to efficiently integrate and orchestrate data from various sources, both on-premises and in the cloud. It provides a visual interface for designing data pipelines, making it easier to define and manage complex data integration workflows.

Broad Data Source Support: ADF supports a wide range of data sources, including Azure services, on-premises databases, SaaS applications, and various file formats. This flexibility enables you to extract, transform, and load (ETL) data from diverse sources, making it suitable for heterogeneous data environments.

Scalability and Performance: ADF leverages the scalability and power of the Azure platform to handle large volumes of data and process it at scale. It can parallelize data processing activities, optimize resource utilization, and provide efficient data movement capabilities, leading to improved performance.

Till now I havnt found any cons of ADF in my 4 years of IT experience wotking with ADF

It helped the clients all accros the world to save a large amount of money

The best thing is that it have lot of connectors for many on- premise servers and cloud servers and best part is data factory UI, where we can design complex etl pipelines by simply drag and drop the activities .

I haven't faced any challenges with ADF , however there are some limitations where we can't perform some activities directly like nested control and also for complex transformation Adf mapping data flows not that much effective, but it's a best product.

We use ADF to built ETl pipelines which will apply the transformations on financial data at one place and load them to Azure synapse i.e data warehouse and also help us to schedule jobs based on event based and removed the manual intervention.

In Azure data factory I like most about it's data pipeline feature, synapse, databricks, storage account and all type of connectors, and it has the best scheduler where any non-programmer also can create pipeline and schedule

Nothing in dislike, if ADF resource price can reduce then it would be great and if sandbox account also gets ADF resource that would be also best to test and practice. Rest it's a best data pipeline product

For our retail clients, we use ADF in many ETL and data pipelining projects. ADF is mainly used to apply the transformation on different ERP, Sales, Finance and all departmental store data at one place and send them to data warehouses.

We use Azure data factory to pull data from multiple sources and import it into our data warehouses. We have many disparate data sources that rarely share a similar format. Using ADF, we can pull in the data automatically, normalize it, run queries against the current DW, import it into our DW, and archive files in cold storage. It's truly a lifesaver for anyone who prefers points and clicks to code.

The learning curve can be pretty steep to learn how to use ADF. I used YouTube videos to supplement my knowledge, which was quite helpful. Once you get the concepts down, it's easy to apply to other projects, but finding out where to start can sometimes be daunting.

We use it to transform and load data into our DW. We have plans in the future to also use it for lots of other projects, but the primary use case was to load data.

Easiness to use , no setup required and integration available with 100+ sources to fetch data.

Data fetched through JSON API is limited to 1000 records and stored at intermediate location, this can be improved.

Data Orchestration and automation of data pipelines and 100+ connectors available to fetch data from multiple sources