Alexander has been working as Big Data Solutions Architect at CleverDATA since 2015. He has more than 10 years of experience in Software Engineering, mainly in server side development. Started as a PHP web developer, he turned then to server-side Java development. He worked at Qiwi for a long time where he was responsible for integrating with external services and developing a payment processing system. Now he is responsible for developing solutions that work with large amounts of data using Hadoop stack and Apache Spark framework.
Using Arbitrary ML Models for Big Data Classification
This talk is about the solution used in our company for big data classification using arbitrary ML models. It covers the following topics:
– achieving technical compromises between Data Science and Software Engineering teams;
– representing ML model as an external service;
– organizing effective asynchronous communication between Hadoop cluster and the external service with ML model;
– building a solution using Reactive Streams approach and akka-http toolkit.
Date: October 11, 2018 | Time: 00:00