Evals for AI Engineers

Google Livros
Evals for AI Engineers

Registe-se ou faça Login para escrever uma crítica ou adicionar este item à sua coleção.

ISBN: 9798341660724
autor: Shreya Shankar / Hamel Husain
editora: O'Reilly Media, Incorporated
data de publicação: 2026 -10
idioma: Inglês
número de páginas: 225

/ 10

0 avaliações

Sem críticas suficientes
Requisitar ou comprar

Systematically Measuring and Improving AI Applications

Shreya Shankar / Hamel Husain   

visão geral

Stop using guesswork to find out how your AI applications are performing. Evals for AI Engineers equips you with the proven tools and processes required to systematically test, measure, and enhance the reliability of AI applications, especially those using LLMs. Written by AI engineers with extensive experience in real-world consulting (across 35+ AI products) and cutting-edge research, this practical resource will help you move from assumptions to robust, data-driven evaluation.
Ideal for software engineers, technical product managers, and technical leads, this hands-on guide dives into techniques like error analysis, synthetic data generation, automated LLM-as-a-judge systems, production monitoring, and cost optimization. You'll learn how to debug LLM behavior, design test suites based on synthetic and real data, and build data flywheels that improve over time.
Whether you're starting without user data or scaling a production system, you'll gain the skills to build AI you can trust--with processes that are repeatable, measurable, and aligned with real-world outcomes.
Run systematic error analyses to uncover, categorize, and prioritize failure modes Build, implement, and automate evaluation pipelines using code-based and LLM-based metrics Optimize AI performance and costs through smart evaluation and feedback loops Apply key principles and techniques for monitoring AI applications in production

comentários
Críticas
Notas