Deploying ML models to production (frequently and safely) - PYCON 2018

Editor's Notes

• #2: I’m David and here’s Ramsey, and we’re going to share about how you can deploy ML models to production frequently and safely. Note to self: A talk is more about telling a story around a topic Changing people's perspective Inspiring them to try something else and giving them the tools for that.” Empathize with audience. Don’t preach
• #6: Note: use “we”, rather than “you” Got an idea (e.g. NLP sentiment analysis). Follow a ML tutorial Built a model Asked to deploy. (click) “You want me to .. what?” Bombarded with questions. How do I deploy? How do I load new data? How do I call .predict() without hitting shift+enter? How do I vectorize user input strings before passing it to the model? We’re stumped. We don’t know where to start. We give up.
• #7: Before we go on, we want to take a quick temperature check
• #8: Bear this question in mind throughout the talk
• #10: Most of these are not ideas that Ramsey and I thought of. They are practices that smart these folks have thought of, and that have been tried and tested at our clients.
• #11: We built a sample app What it does Why we chose this stack / data source How you can use it To make this tangible, we’ve had to pick a stack. But focus on the patterns, and not our implementation
• #12: we built a demo so that we can have code to illustrate some points but we ran out of time So for the last few points, we'll talk abt concepts and how we would implement it
• #13: Just read the title. Don’t talk too much here.
• #14: Use fraud detection as an example. Share about tracer bullet idea here
• #15: In other programming languages / frameworks, when we build something, we can share a link on Twitter and the rest of the world can use it In ML, my experience === i/people just share screenshots of the loss curve (insert picture) or some object detection bounding boxes (insert pictures) This is the problem facing many of us today. We have tons of ML tutorials in local environment / jupyter notebooks, but very little / none about serving those models or continuous delivery/evolution of these models Until something is in production, it creates value for no one except ourselves
• #16: Model decay (our model can get stale / dangerous) Deploying frequently allows us to make iterative improvements to our model (training with new {data, hyperparameters, features}
• #17: cars, phones, ikea chairs go through multiple rounds of testing. Why should ML models be any different? The irony is that ML has already started to impact all of our lives, but testing and safety is something that we rarely talk about in ML
• #18: ML models affect decisions that impact lives… in real-time Safety is essential
• #20: Goal of today’s talk (in pictures)
• #21: “Ok, david - I’m sold on why this frequent and safe deployment thing is important. But what does it look like in practice?”
• #23: CI/CD pipeline - The main vehicle for everything we’re sharing today It’s all about feedback 30 seconds - quick overview of this. The model goes through different stages Each of them solves a different problem, which we’ll talk about next Generalizable approach: we can see it working for classifiers, regression models, deep learning models, NLP models, etc.
• #26: Snowflake Every dataset is unique, non-reproducible, hand-cleaned with TLC
• #28: Challenge Brittle glue code in ML Unit tests At lower levels, check edge cases, add more tests for all that At higher levels, check happy path and integration
• #31: Skip if people get CI pipeline
• #34: Deployment Provisioning Configuration Deploying your app
• #38: Tracer bullet Deploying a simple thing is easier than a complex thing Focus on deploying first. Focus on deployment pipeline. Don’t get distracted. We can come back to tuning models later
• #40: Benefits Monitoring === important source of feedback Find out when model are getting stale / dangerous LIME - Local Interpretable Model-Agnostic Explanations Caveat: Monitoring ML metrics can be challenging because labels take time to come
• #41: Training serving skew where the data seen at serving time differs in some way from the data used to train the model, leading to reduced prediction quality
• #44: Talk about just the first bullet
• #46: Pyception (Anaconda 2018 video) - a battle between data scientists and software engineers
• #48: Generalizable approach: we can see it working for classifiers, regression models, deep learning models, NLP models, etc.
• #52: TODO: add bitly link here