Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Sunday, November 13 • 9:50am - 10:30am
Twitter Heron on YARN/REEF

Sign up or log in to save this to your schedule and see who's attending!

Twitter Heron is the next generation streaming system that has been in production for more than 2 years. Heron is used for diverse use cases including but not limited to ETL, BI, machine learning and media processing. In this talk, we will talk about how Microsoft is improving the Heron real-time streaming engine. First, we will discuss how one can deploy Heron with the widely used YARN scheduler to seamlessly integrate with the popular Hadoop solutions stack. Second, we will present the ongoing efforts aimed at making Heron use hardware resources more efficiently by optimizing performance and resource utilization of Heron topologies. Finally, we will round off the talk with a peek at the exciting upcoming future work.

Speakers
avatar for Ashvin Agrawal

Ashvin Agrawal

Microsoft
Ashvin is a software engineer with a work experience of more than 10+ years. He specializes in developing large scale distributed systems. Ashvin is currently a Senior Research Engineer at Microsoft where he works on streaming systems and contributing to Twitter Heron project. Ashvin has worked at VMware, Yahoo, and Mojo Networks and holds a M.Tech in Computer Science from IIT Kanpur, India.
AF

Avrillia Floratou

Avrillia is a Senior Scientist at Microsoft's Cloud and Information Services Lab, where her research is focused on scalable real-time stream processing systems. She is also an active contributor to Heron, collaborating with Twitter. Prior to her current role, she was a research scientist in IBM Research working on SQL-on-Hadoop systems. She holds a PhD in data management from University of Wisconsin-Madison.
avatar for Karthik Ramasamy

Karthik Ramasamy

Engineering Manager, Twitter
Karthik is the engineering manager for Real Time Compute at Twitter and co-creator of Heron. He has two decades of experience working in parallel databases, big data infrastructure and networking. He cofounded Locomatix, a company that specializes in realtime streaming processing on Hadoop and Cassandra using SQL that was acquired by Twitter. Before Locomatix, Karthik was at Juniper Networks where he designed and delivered platforms, protocols... Read More →


Sunday November 13, 2016 9:50am - 10:30am
Off by One

Attendees (22)