Google Cloud Dataflow Reviews

Best thing about Dataflow about its fully managed capability so that we don't need to manage infrastructure and scales easily. It also provides lot of templates which is useful for beginner and intermediate level developers and top of that they can easily update the configuration and pipeline and can run process petabyte of data. Also it supports Yaml SDK which removes Apache Beam dependencies as well.

When we are working with distributed processing, its difficult to get correct configuration especially for new user its very complex to set it up and most of the time it charges a lot if not set properly. And as it supports only Apache Beam, some of the concepts are very difficult to understand. Also they can work on monitoring and logging, sometime its not clear.

Whenever gets new usecase for Data migration, Data processing, Data transformation, for me first choice is always Dataflow because the way it handles both batch (GCS, onprem servers etc..) and streaming (Kafka, Pub/sub) data andcan easily scales horizontally for faster processing. Overall it solves challenges around large scale processing in GCP environments while reducing operational effort to almost zero.

Google Cloud Dataflow is extremely easy to use for processing stream of events. Building complex streaming pipelines is simple and effiicent with Dataflow. Offers real time monitoring of the streaming pipeline with important metrics such as Throughput, CPU and memory utilisation.

It is easy to integrate with Data sources and sinks such as Cloud Pub-sub, Kafka and Cloud Spanner.

It is costly as compared to other solutions such as Apache Flink.

Difficult to implement watermarks.

- using jyupter notebook to write code and not to worry about backend services.

GCP Dataflow effortlessly manages big data tasks, unifying code with Apache Beam. It's a Google Cloud team player, excelling with services like BigQuery.

- not much documentation and resources available on it

I really like Apache Beam, which I use with Dataflow. It works well, on Google Cloud, for both Batch and Stream processing. The compute just works, and let's our Data Engineers get things done.

Some of the marketing - years ago - was overkill. These days it lives up to the hype. I am saddened that more people don't use, because it is so powerful.

We use Dataflow for both Big Data and Stream processing.

Google cloud dataflow is automatically optimize and manages resources for you this platform supports multiple programming languages including Python, java and SQL and makes it easy for developers to focus on writing codes

Till now I have not found any kind of drawback.I just love this and every update of this platform makes me fall again

Google cloud dataflow is automatically optimize my data so I don't have to worry about about things like load balancing, scaling or fault tolerance and it also provide me wide range of monitoring and debugging tools.

The UI is very simplistic and the schedules and jobs can be created very easily alongside. Also the support for templates in Java and python are incredible.

The parameter and templates are very rigid and further debugging is hard as the logs aren't very resourceful. If a pipeline fails often the errors and logs are poor.

Google dataflow helped me steam GBs and TBs of data and supply it to my processing pipeline.

Dataflow can easily autoscale to multiple instances if it is handling a lot of data.

It is sometimes difficult to debug a problem in your pipeline.

Serverless tool for batching and streaming data

It's a PaaS-based offering that is based on open sourcing. It can also be used to process streams of data and batch data.

It only takes a few minutes to hours to move a large amount of data to users database

the initial deployment was risky and faced few issues

A great tool to move a large amount of data, but poor documentation made implementation a problematic experience

Google Dataflow is great as it provides the easiest and smoothest experience of loading data, be it in Batch or Streaming. The best aspects are ;

1. It provides 100% efficient Big Data processing along with easy data migration.

2. Ability to create custom Apps and design APIs.

3. Its developer-friendly user- interface and working.

Cloud Dataflow offers serverless processing for handling Big Data. It can crunch millions of records in any form event based or batch and has excellent throughput. With the different sdks it provides like Python, Java etc. it is very developer friendly as well.

Nothing really. There need to be some added features to the Python sdk like in Java, but that will happen as the product grows.

Processing Big data. Both stream and batch processing.