Use Case Track

Data lossless event time streaming processing for revenue calculation

One of the main characteristics of the good streaming pipeline is correctness for event time processing. Real challenges become when such pipeline should be resilient to different types of failures. In this talk, we describe how Criteo runs Flink on one of the biggest Yarn clusters in Europe and computes 100k messages per second to acknowledge revenue of our platform within the delay of 5 minutes. Real-time revenue monitoring system calculates data under 1% of discrepancies and minimizes business impact in case of revenue anomalies.

Authors

Oleksandr Nitavskyi
Oleksandr Nitavskyi
Criteo
Oleksandr Nitavskyi

Staffs Software Engineer in Criteo, Currently technical lead of Invalid Traffic detection team. Worked for Grammarly in the past. Likes JVM and functional programming. Fun of improving development productivity.

Fill out the form to view
the Slides and Video

* All fields required