• Ecosystem
  • Keynote
  • Managing continuous applications
  • Other
  • Technology deep-dive
  • Use Cases

Alibaba’s common algorithm platform on Flink

There are many batch and stream scenarios in Alibaba, and many data analysts are non-technical, like to use GUI or script tool to deal with data to help business decisions. We’d like to share our experiences on developing algorithms on Apache ...

Use Cases
Speakers Xu Yang (Alibaba) View Video & Slides

Apache Flink + Apache Beam: Expanding the horizons of Big Data

"Over the past few months, the Apache Flink and Apache Beam communities have been busy developing an industry leading solution to author batch and streaming pipelines with Python. This was made possible by a significant effort to revamp Beam’s portability ...

Speakers Anand Iyer (Google Cloud) View Video & Slides

Bootstrapping State In Apache Flink

Apache Flink is a popular stream computing framework for real-time stream computing. Many stream compute algorithms require trailing data in order to compute the intended result. One example is computing the number of user logins in the last 7 days. This creates a ...

Managing continuous applications
Speakers Gregory Fee (Lyft) View Video & Slides

Building a scalable focused web crawler with Flink

Is it possible to build an efficient, focused web crawler using Flink? That was the question that led to the creation of the flink-crawler open source project. In this talk I’ll discuss how we use Flink’s support for AsyncFunctions and ...

Speakers Ken Krugler (Scale Unlimited) View Video & Slides

Building Flink As a Service platform at Uber

Stream processing plays an important role in Uber’s real-time business. It has been widely used to support many use cases in Uber, like surge pricing and restaurant manager. To support all the stream processing use cases at Uber, the stream processing ...

Managing continuous applications
Speakers Shuyi Chen &
Rong Rong (Uber)
View Video

dA Platform – Production-ready stream processing with Apache Flink

"In this talk, we are going to present dA Platform, a production-ready platform for stream processing with Apache Flink® from data Artisans.

The platform includes open source Apache Flink and Application Manager, a central deployment and management component. dA ...

Managing continuous applications
Speakers Patrick Lucas &
Robert Metzger (data Artisans)
View Video & Slides

eBay monitoring platform preprocessing and alerting on Flink

eBay monitoring platform collects metrics, events and logs from network devices, kubernetes clusters, applications and other monitoring tools, data will be feeding into preprocessing and alerting engine to enrich, normalize, dedupe, alerting, etc. We start by building the first generation preprocessing and ...

Use Cases
Speakers Garrett Li &
Ralph Su (eBay)
View Video

Embedding Flink Throughout an Operationalized Streaming ML Lifecycle

"Operationalizing Machine Learning models is never easy. Our team at Comcast has been challenged with operationalizing predictive ML models to improve customer care experiences. Using Apache Flink we have been able to apply real-time streaming to all aspects of the Machine ...

Use Cases
Speakers Dave Torok &
Sameer Wadkar
(Comcast Corporation)
View Video & Slides

Extending Flink metrics: Real-time BI atop existing Flink streaming pipelines

Flink metrics module allows to use Dropwizard-like metrics and reporters in Flink pipelines. It opens a rich opportunity to not only monitor health of Flink pipelines but also attach real-time business intelligence metrics to run alongside existing Flink data jobs, thus avoiding ...

Managing continuous applications
Speakers Andrew Torson (Walmart Labs) View Video & Slides

Finding Bad Acorns

Within fintech catching fraudsters is one of the primary opportunities for us to use streaming applications to apply ML models in real-time. This talk will be a review of our journey to bring fraud decisioning to our tellers at Capital One using ...

Use Cases
Speakers Andrew Gao &
Jeff Sharpe (Capital One)
View Video & Slides

Flink real-time analysis in CloudStream Service of Huawei Cloud

CloudStream service is a Full Management Service in Huawei Cloud. Support several features, such as On-Demand Billing, easy-to-use Stream SQL in online SQL editor, test Stream SQL in real-time style, Multi-tenant, security isolation and so on. We choose Apache Flink as streaming ...

Speakers Jinkui Shi &
Radu Tudoran (Huawei)
View Video & Slides

How to build a modern stream processor: The science behind Apache Flink

Stream Processing has evolved quickly in a short time: a few years ago, stream processing was mostly simple real-time aggregations with limited throughput and consistency. Today, many stream processing applications have complex logic, strict correctness guarantees, high performance, low latency, and maintain ...

Technology deep-dive
Speakers Stefan Richter (data Artisans) View Video & Slides

Operating Flink on Mesos at Scale

Flink has supported Apache Mesos officially since the 1.2 release and many users have been using them together even before that. The latest releases 1.4 and 1.5 (not released at the time of writing) add a deeper integration for resource schedulers, such as Mesos, which ...

Speakers Jörg Schad &
Biswajit Das (Mesosphere)
View Video & Slides

Optimizations in Blink Runtime for Global Shopping Festival at Alibaba

November 11th (double eleven) has become Alibaba’s Global Shopping Festival, and Alibaba generated gross merchandise volume (GMV) of US$25.3 billion on Nov 11 this year. On that day, Alibaba’s realtime computing engine Blink, which was Alibaba’s version of Flink, processed ...

Technology deep-dive
Speakers Feng Wang (Alibaba) View Video & Slides

Panta Rhei: designing distributed applications with streams

Event streams as source of truth for applications have risen in popularity. Thoughtworks lists it in the assess phase of their technology radar (Nov 17). At DriveTribe we’re slightly ahead of the curve as we’ve been using this technique in production ...

Use Cases
Speakers Aris Koliopoulos &
Alex Garella (DriveTribe)
View Video & Slides

Powering Tensorflow with Big Data (Apache BEAM & Flink)

Tensorflow is all kind of fancy, from helping startups raising their Series A in Silicon Valley to detecting if something is a cat. However, when things start to get “real” you may find yourself no longer dealing with mnist.csv, and instead ...

Speakers Holden Karau (Google Cloud) View Video

Powering Yelp’s Data Pipeline Infrastructure with Apache Flink

Last year, during the Flink Forward conference, a group of Yelp engineers saw in Apache Flink a perfect candidate to solve many of the pressing challenges that the Yelp’s fast growing data pipeline was encountering. One year later, Yelp is running ...

Speakers Enrico Canzonieri (Yelp Inc.) View Video

Real-time monitoring of Mobile Internet Quality of Experience using Flink

“Customer experience is the next big battle ground for telcos,” proclaimed recently Amit Akhelikar, Global Director of Lynx Analytics at TM Forum Live! Asia in Singapore. But, how to fight in this battle? A common approach has been to keep “under control” ...

Use Cases
Speakers David Reniz &
Dahyr Vergara (Everis)
View Video & Slides

Scaling Flink in Cloud

Over 109 million subscribers are enjoying more than 125 million hours of TV shows and movies per day on Netflix. This leads to massive amount of data flowing through our data ingestion pipeline to improve service and user experience. They are powering various data ...

Technology deep-dive
Speakers Steven Wu (Netflix) View Video & Slides
Event Processing

Scaling stream data pipelines

Extracting insights out of continuously generated data requires a stream processor with powerful data analytics features such as Apache Flink. A stream data pipeline with Flink typically includes a storage component to ingest and serve the data. Pravega is a stream store ...

Speakers Till Rohrmann &
Flavio Junqueira
(data Artisans)
View Video & Slides

Scaling Uber’s Realtime Optimization with Apache Flink

Many marketplace products (e.g pricing, positioning etc.) in Uber require intensive realtime optimizations. Such applications help Uber automatically maintain marketplace reliability, generate market insights and improve the network efficiency across more than 600 cities in realtime. Underneath, Uber engineers leverage Apache Flink ...

Use Cases
Speakers Xingzhong Xu
(Uber Technologies Inc.)
View Video & Slides

Stream Processing Revolutionizing Big Data

Stream Processing in conjunction with a Consistent, Durable, Reliable stream storage is kicking the revolution up a notch in Big Data processing. This modern paradigm is enabling a new generation of data middleware that delivers on the streaming promise of a simplified ...

Speakers Srikanth Satya (Dell EMC) View Video & Slides

Testing Stateful Streaming Applications

As more workloads migrate from batch to stream processing there is ever increasing demand on streaming applications to operate based on more complex business rules. Apache Flink provides many high and low level api’s for writing a wide breadth of stateful ...

Managing continuous applications
Speakers Seth Wiesman (MediaMath) View Video & Slides

What turns stream processing from a tool into a platform?

"Stream Processing is a powerful paradigm, especially when backed by a system like Apache Flink. With each release and year, we see Flink being used for more challenging use case and applications.

But beyond the individual application (though it ...

Speakers Stephan Ewen (data Artisans) View Video & Slides

Why and how to leverage the simplicity and power of SQL on Flink

SQL is the lingua franca of data processing and everybody working with data knows SQL. Apache Flink provides SQL support for querying and processing batch and streaming data. Flink’s SQL support powers large-scale production systems at Alibaba, Huawei, and Uber. Based ...

Speakers Fabian Hueske &
Timo Walther (data Artisans)
View Video & Slides