Scalæ By the Bay has ended
Back To Schedule
Sunday, November 13 • 9:50am - 10:30am
Twitter Heron on YARN/REEF

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Twitter Heron is the next generation streaming system that has been in production for more than 2 years. Heron is used for diverse use cases including but not limited to ETL, BI, machine learning and media processing. In this talk, we will talk about how Microsoft is improving the Heron real-time streaming engine. First, we will discuss how one can deploy Heron with the widely used YARN scheduler to seamlessly integrate with the popular Hadoop solutions stack. Second, we will present the ongoing efforts aimed at making Heron use hardware resources more efficiently by optimizing performance and resource utilization of Heron topologies. Finally, we will round off the talk with a peek at the exciting upcoming future work.

avatar for Ashvin Agrawal

Ashvin Agrawal

Ashvin is a software engineer with a work experience of more than 10+ years. He specializes in developing large scale distributed systems. Ashvin is currently a Senior Research Engineer at Microsoft where he works on streaming systems and contributing to Twitter Heron project. Ashvin... Read More →

Avrillia Floratou

Avrillia is a Senior Scientist at Microsoft's Cloud and Information Services Lab, where her research is focused on scalable real-time stream processing systems. She is also an active contributor to Heron, collaborating with Twitter. Prior to her current role, she was a research scientist... Read More →
avatar for Karthik Ramasamy

Karthik Ramasamy

Engineering Manager, Twitter
Karthik is the engineering manager for Real Time Compute at Twitter and co-creator of Heron. He has two decades of experience working in parallel databases, big data infrastructure and networking. He cofounded Locomatix, a company that specializes in realtime streaming processing... Read More →

Sunday November 13, 2016 9:50am - 10:30am PST
Off by One