Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Saturday, November 12 • 2:10pm - 2:50pm
Implement a scalable statistical aggregation system using akka

Sign up or log in to save this to your schedule and see who's attending!

At Symantec email security group, a common problem we face is to aggregate multiple metrics with different time granularity in real-time from hundreds of millions emails per day. Various existing solutions try to address the problem by using batch and/or streaming algorithms. Often such approach requires the use of many different technologies and are expensive to run. Another approach is to use statistical data structures such as Count Min Sketch that can greatly reduce the overheads of storage and processing at the cost of accuracy. However, implement such algorithms at large scale poses several problems. In this talk, we introduce Algegate (algebra + aggregate) a pure statistical, distributed platform implemented using Akka. It is designed to be fault-tolerant, back-pressure compliant and easily to scale out to multiple nodes.

Speakers
avatar for Stanley Nguyen, Vu Ho

Stanley Nguyen, Vu Ho

Software Engineer, Symantec
Stanley Nguyen is a software developer in Email Security group at Symantec where he helps to build a high availability big data platform, write high performance backend services and develop interactive visualisation interfaces. He writes a majority of code in Scala, Go and NodeJS. | | Vu Ho is currently working in Symantec Email Security team. He is responsible for developing a big data platform for various email security operations. Key... Read More →


Saturday November 12, 2016 2:10pm - 2:50pm
Caching

Attendees (23)