Scalæ By the Bay

We will present Scio, a Scala API for Google Cloud Dataflow (incubated as Apache Beam). Apache Beam offers a simple, unified programming model for both batch and streaming data processing while Scio brings it much closer to other high level APIs many data engineers are familiar with, e.g. Spark and Scalding. We will cover design and implementation of the framework, including features like type safe BigQuery macros, REPL and serialization. There will also be a live coding demo.