Scalæ By the Bay has ended
Back To Schedule
Saturday, November 12 • 9:50am - 10:30am
Spark and Protocol Buffers - An Awesome Combination

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Have you ever been building a project using JSON, and managing the data format became too complex? Schema changes and type safety become a big pain. I will share one of my favorite solutions we used at Google, that many Scala programmers can appreciate: Protocol buffers. Protocol buffers are Google’s cross-platform language-agnostic mechanism for parsing and serializing structured data. With protocol buffers, you define your data schema, and the compiler generates parsers and serializers for your data in many different languages. ScalaPB is a library and code generator that brings the awesome power of protocol buffers to Scala and Spark. ScalaPB takes your data schema and generates case classes, along with parsers, serializers, and even lenses for convenient field updates within deeply nested structures. In this talk, I’ll give a gentle intro to protocol buffers and ScalaPB. I will show how you can use protocol buffers and ScalaPB as your project’s data exchange format to take the pain away from schema evolution and cross-team data sharing. We will do some live coding and build a Spark application that processes millions of protobufs!

avatar for Nadav Samet

Nadav Samet

CTO, TrueAccord
Nadav Samet is the CTO and co-founder of TrueAccord, a Scala-based Fintech startup. He started programming when he was five years old and has been passionate about it ever since. Before TrueAccord, he was working at Google, where he helped stabilizing GMail, and then went on to emerging... Read More →

Saturday November 12, 2016 9:50am - 10:30am PST
Off by One