General

Reflections on Flink Forward 2015

October 28, 2015 - General

twitterredditlinkedinmailtwitterredditlinkedinmail
Flink Forward 2015 was the inaugural conference around the Apache Flink™ community and took place at the beautiful Kulturbrauerei in Berlin, a former brewery turned into a fantastic event space. Overall, we are both delighted and overwhelmed by the success of the conference, and how the Flink community is rapidly growing and connecting. Flink Forward Around 250 participants had the opportunity to attend a total of 33 technical talks (organized in 2 parallel sessions), as well as participate in free Flink trainings. The talks had been selected by a program committee comprising of five Flink PMC members: Marton Balassi, Stephan Ewen, Vasia Kalavri, Henry Saputra, and Kostas Tzoumas. The slides of all the talks are available online. Video recordings will be added soon. Flink Forward featured two keynotes, one by Kostas Tzoumas and Stephan Ewen from data Artisans, and one from William Vambenepe from Google. Both keynotes emphasized how stream processing is more than just “fast data”, but, really, a new paradigm for programming data-intensive applications which embraces the unbounded and continuous nature of data as it is produced in the real world. Instead of ignoring or implicitly managing continuous data using batch frameworks or hybrid (lambda) architectures, developers can now use modern stream processing frameworks to get timely answers from their data. Since the value of data is directly correlated to its freshness, a streaming-first infrastructure increases the actual value of the insight we get from our data signals. Both keynotes asserted that stream processing technology has now matured, citing Apache Flink™ (with an emphasis on the upcoming Flink 0.10 version) and Google Cloud Dataflow, two systems that share a lot in common, and two communities that are closely cooperating to provide compatibility between the two frameworks. The movement towards stream processing was very visible in the conference, with several talks dedicated to how companies put Apache Flink™ to their production environments for real-time data processing, as well as talks on how the Flink framework internally treats streaming data:Stream processing is, of course, only one of the things that people are doing with Flink. Michael Häusler from ResearchGate (rightly) argued that batch is not dead. He shared ResearchGate’s methodology to choose a framework that makes simple things easy by comparing solutions to solve a simple task. Other talks that focused on evaluation of different systems and performance were:Another focus of the conference was Machine Learning, interactive analytics, graph processing, and integration of Flink with other pieces of the Big Data infrastructure:FF_Training The training sessions were very well attended, with hands-on training on Flink’s DataStream, DataSet, and Gelly APIs. As always, you can access the latest data Artisans training material for Apache Flink™ online. Of course, not everything about Flink Forward 2015 was perfect. As this was the first installment of the conference, glitches like spotty internet or mix-ups with registration did happen at times, but were quickly resolved. A more important issue that we noticed is the lack of diversity in the community, and especially the very small percentage of women speakers and attendees. While this is an issue of the tech community at large and not just specific to the Flink community, we would like to start an open discussion about it as early as possible and see how we can improve in the future.

What next?

Browse all slides and videos of Flink Forward 2015 here, and see other reviews of the conference here. See you at Flink Forward 2016!
twitterredditlinkedinmailtwitterredditlinkedinmail