Flink 1.0: General availability and pushing the envelope in open source stream processing
We are delighted to see that the Flink community has announced the availability of Apache Flink™ 1.0. This release is one of the largest Flink releases ever, with about 64 individuals resolving more than 450 JIRA issues, and, most importantly, marks the beginning of the Flink 1.x.y series, which initiates backwards compatibility for all minor releases moving forward. We see this release as the most important milestone in the project since Flink graduated from the Apache Incubator one year ago. Additionally, we see this release as (1) validating production-readiness for Flink, and (2) significantly pushing the envelope in stream processing with features that are unique in the open source world.
Production readinessWhile the needs of production users can widely vary, Flink by now covers all points needed to back typical production data applications with a smooth operational experience.
- Backwards compatibility: Flink 1.0 removes the hurdle of changing the application code when Flink releases new versions. This is huge for production users who want to maintain their business logic and applications while seamlessly benefiting from new patches in Flink.
- Operational features: Flink by now boasts very advanced monitoring capabilities (this release adds backpressure monitoring, checkpoint statistics, and the ability to submit jobs via the web interface). This release also adds savepoints, an essential feature (and unique in the open source world) that allows users to pause and resume applications without compromising result correctness and continuity.
- Battle-tested: Flink is by now in production use at both large tech and Fortune Global 500 companies. A team at Twitter recently clocked Flink at 15 million events per second in a moderate cluster.
- Integrated: Flink has always been integrated with the most popular open source tools, such as Hadoop (HDFS, YARN), Kafka (this release adds full support for Kafka 0.9), HBase, and others. Flink also features compatibility packages and runners, so that it can be used as an execution engine for programs written in MapReduce, Apache Storm, Cascading, and Apache Beam (incubating).