Technology Deep Dive Track

Upgrading Apache Flink Applications: State of the Union

Apache Flink streaming applications are typically designed to run indefinitely for long periods of time. As with all long-running services, the applications need to be maintained and upgraded, including improvements to adapt to changing business logic and bug fixes. With this in mind, being able to smoothen the process of upgrading Flink streaming applications, as well as enabling users to have enough flexibility in doing so, has always been one of the priorities of the Flink community.

This talk will provide an overview of common considerations when it comes to upgrading Flink streaming applications, and progress or discussions the community has made so far in tackling each one of them. Specifically, topics include 1) Application state compatibility and allowing schema evolution, 2) Application topology changes, 3) Swapping state backends, and 4) Pre-flight detection of incompatible upgrades.

Authors

Gordon Tai
Gordon Tai
data Artisans
Gordon Tai

Tzu-Li (Gordon) Tai is an Apache Flink PMC member and software engineer at data Artisans. His main contributions in Apache Flink includes work on some of the most widely used Flink connectors (Apache Kafka, AWS Kinesis, Elasticsearch). Gordon was a speaker at conferences such as Flink Forward, Strata Data, as well as several Taiwan-based conferences on the Hadoop ecosystem and data engineering in general.

Fill out the form to view
the Slides and Video

* All fields required