Scalæ By the Bay has ended
Back To Schedule
Friday, November 11 • 1:10pm - 1:30pm
Endpoint Security with Complex Data

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

One of the challenges of modern data science is handling complex data structures like structs, arrays, maps, queues, images, etc. in datawarehouse environments (as oppose to flat table representations). In this talk I will cover the existing ways to work with complex data structures with open-source projects like Pig, Hive and Impala as well as how E8 Security used Scala to simplify data pipelines with complex data structures. I will use an example of endpoint security computations. An endpoint in Enterprise Security is any computing device exposed to the clients or customers that request access to the corporate network, which yet cannot be entirely controlled or administered by the network administrators for one or another reason. The challenge is usually solved by additional monitoring of the devices themselves and the network traffic emanating from/to the device. In this particular instance, E8 built a machine learning based solution that tracks the footprint of the system and builds threat models based on the changes in the device footprints, which requires extensive use of complex and nested data structures. Currently E8 Security has customers tracking more than 0.5 million of endpoints running 10,000s different processes.

avatar for Alex Kozlov

Alex Kozlov

Architect, E8 Security, Inc.
Just a Humble Big Data Architect

Friday November 11, 2016 1:10pm - 1:30pm PST
Off by One