Flink Forward San Francisco Preview (part 4 of 6): Ecosystem
Hey Apache Flink community! Hope you’re getting ready to attend Flink Forward San Francisco on April 9-10 to learn about interesting Flink use cases, internals, ecosystem, latest updates, and much more. If you haven’t registered for the conference and training yet, make sure to reserve your spot here.
Today, we’ll run through sessions that cover the Apache Flink ecosystem.
- Enrico Canzonieri will share how the team at Yelp managed to solve the data pipeline challenges using Apache Flink. He will also discuss the integration of Flink into an existing multi-regional Kafka based streaming ecosystem at scale, its challenges and best practices.
- Flavio Junqueira from DellEMC and Till Rohrmann from data Artisans will introduce a stream store Pravega that ingests and stores stream data permanently, and discuss how Flink jobs leverage Pravega’s stream scaling feature to rescale stateful jobs according to variations in the workload.
- Jinkui Shi will explain how Apache Flink is used for real-time analysis in CloudStream Service of Huawei Cloud. He will talk about how Flink jobs are run on YARN, Apache Mesos, and Kubernetes.
- Holden Karau from Google Cloud will discuss how TensorFlow can be used together with Apache Beam, Apache Flink, and Apache Spark to create a full machine learning pipeline.
- Joerg Schad from Mesosphere will talk about best practices and pitfalls of operating large Apache Flink clusters on top of Apache Mesos, including deployments, monitoring, scaling, upgrades, and debugging.