Scalæ By the Bay

Due to privacy concerns and the nature of SAAS businesses, platforms like CRM systems often have to provide intelligent data-driven features that are built from many different unique, per-customer machine learnt models. In the case of Salesforce, this entails building hundreds of thousands of models tuned for as many distinctly different customers for any given data-driven application. In this talk I will describe our home grown scala and SparkML-based machine learning platform that has the following characteristics: - Automated feature engineering resulting in much quicker modeling turnarounds and higher accuracy than general purpose modeling libraries such as scikit-learn. - Automatic hyper-parameter optimization, feature selection and model selection resulting in a very good model for every specific customer of the product. - Modular workflows and transformations that complement systems like SparkML and KeystoneML. - Huge scale that enables training thousands of model per day. This talk will give the audience a good idea of which parts of the typical machine learning pipeline are easier to automate, and which are harder.