Use Case Track

Anomaly Detection Engine for Cloud Activities using Flink

Microsoft Cloud App Security provides organizations with enterprise grade protection to cloud applications. One of the main capabilities of CAS is the real time detection of threats like compromised accounts, insider threat and ransomware, based on abnormal user activity.
In this talk we will describe our search for a right stateful streaming platform to empower our detections engine, the reasons that led us to choose Flink, and the architecture we built on top of Flink. We will share details about the challenges of constructing a complex job with multiple levels of statistical analysis, custom windowing, and inline machine learning model updating. We will also share our experience running Flink in Azure and connecting it to our production eco-system.

Authors

Yonatan Most
Microsoft
Yonatan Most

Yonatan holds an M.Sc. in theoretical physics, and currently leads the Data Science team for Microsoft Cloud App Security, utilizing Machine Learning tools to detect anomalies in user activity in the cloud.

Avihai Berkovitz
Microsoft
Avihai Berkovitz

Avihai is an architect in the Microsoft Cloud App Security group, where he leads the development of strategic infrastructure changes and major product tracks. His main focus is designing and building applications at mass scale.

Fill out the form to view
the Slides and Video

* All fields required