Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Saturday, November 12 • 9:50am - 10:30am
Spark and Protocol Buffers - An Awesome Combination

Sign up or log in to save this to your schedule and see who's attending!

Have you ever been building a project using JSON, and managing the data format became too complex? Schema changes and type safety become a big pain. I will share one of my favorite solutions we used at Google, that many Scala programmers can appreciate: Protocol buffers. Protocol buffers are Google’s cross-platform language-agnostic mechanism for parsing and serializing structured data. With protocol buffers, you define your data schema, and the compiler generates parsers and serializers for your data in many different languages. ScalaPB is a library and code generator that brings the awesome power of protocol buffers to Scala and Spark. ScalaPB takes your data schema and generates case classes, along with parsers, serializers, and even lenses for convenient field updates within deeply nested structures. In this talk, I’ll give a gentle intro to protocol buffers and ScalaPB. I will show how you can use protocol buffers and ScalaPB as your project’s data exchange format to take the pain away from schema evolution and cross-team data sharing. We will do some live coding and build a Spark application that processes millions of protobufs!

Speakers
avatar for Nadav Samet

Nadav Samet

CTO, TrueAccord
Nadav Samet is the CTO and co-founder of TrueAccord, a Scala-based Fintech startup. He started programming when he was five years old and has been passionate about it ever since. Before TrueAccord, he was working at Google, where he helped stabilizing GMail, and then went on to emerging projects like Google Flights and Google Hotels.


Saturday November 12, 2016 9:50am - 10:30am
Off by One