Flink Forward Berlin Preview: Flink Use Cases and Applications
Attention stream processing squirrels around the world! Flink Forward Berlin sessions were announced last week, and the program is outstanding. The topics range from production Flink use cases, to Apache Flink® internals, to the growth of the Flink ecosystem.
In this post, we’ll highlight just a few of the use cases and applications sessions that you’ll see at the conference. And don’t forget–you can buy your Flink Forward Berlin tickets here.
• Xiaowei Jiang of Alibaba (a keynote speaker) will share how his team deployed Flink at massive scale in production inside the largest e-commerce company in the world. And his colleague Shaoxuan Wang will describe a system Alibaba built to allow product managers and data analysts to write SQL to process real-time data efficiently and reliably.
• Bas Geerdink of ING will share how his company created a streaming analytics platform with Flink and Kafka that offers high-throughput and low-latency, ideally suited for complex and demanding use cases such as customer notifications and fraud detection in a global bank. Wouter Ligtenberg, also of ING, will describe Tink, a temporal graph analytics library focused on interval graphs in which every edge is associated with a starting time and an ending time.
• Matt Zimmer of Netflix will discuss implementation details around using Flink’s Window API at Netflix to aggregate events into windows customized along varying definitions of a session.
• Gyula Fora of King (a longtime Flink user) will talk through 1.5 years of running production Flink streaming jobs that consume over 40 billion events every day and maintain over 10 TBs of user state.
• David Rodriguez of Cisco will introduce the notion of approximate filter, join, and groupby operations for arrays will introduce the notion of and will highlight how Cisco Umbrella streams large signals stored in arrays and then clusters them using approximate filter, join, groupby methods to detect waves of botnets and cybercrime online.
• Dongwon Kim of SK Telecom, South Korea’s largest wireless carrier, describes his company’s Predictive Maintenance system based on a Deep Neural Network (DNN)-based prediction model for precise prediction and a Flink-based runtime system, which applies the model to a sliding window on sensor data streams.
• Mihail Vieru of Zalando will discuss using Flink to materialize event streams from the company’s central event bus into persistent cloud storage.
• Patrick Gunia of ResearchGate will describe how and why his company moved their stats infrastructure from Apache Storm to Flink, highlighting both the improvements and challenges they encountered along the way.
• Aris Koliopoulos of Drivetribe will discuss how his company’s product is built on top of the Event Sourcing/CQRS pattern and uses Apache Kafka as its source of truth and Apache Flink as its processing backbone. (Editor’s note: Drivetribe gave some insight into this topic in a very popular blog post earlier this year)
• Hao Wu of HanSight (a security startup based in China) will discuss how his company’s user behavior analytics engine runs on top of Flink and achieves dynamic ETL rule configuration and hot deployment.