Human-in-the-Loop Machine Learning is a guide to optimizing the human and machine parts of your machine learning systems, to ensure that your data and models are correct, relevant, and cost-effective. 20-year machine learning veteran Robert Munro lays out strategies to get machines and humans working together efficiently, including building reliable user interfaces for data annotation, Active Learning strategies to sample for human feedback, and Transfer Learning. By the time you’re done, you’ll be able to design machine learning systems that automatically select the right data for humans to review and ensure that those annotations are accurate and useful. what's inside Active Learning to sample the right data for humans to annotate Annotation strategies to provide the optimal interface for human feedback Techniques to select the right people to annotate data and ensure quality control Supervised machine learning design and query strategies to support Human-in-the-Loop systems Advanced Adaptive Learning approaches that use machine learning to optimize each step in the Human-in-the-Loop process Real-world use cases from well-known data scientists