Models evaluations

Evaluating a machine learning model consists of computing its performance and behavior on a set of data called the Evaluation set. Model evaluations are the cornerstone of MLOps capabilities. They permit Drift analysis, Model Comparisons and automating retraining of models

• Concepts
  • When training
  • Subsequent evaluations
• Evaluating DSS models
  • Configuration of the evaluation recipe
    • Labels
    • Sampling
  • Limitations
• Evaluating Large Language models and Generative AI pipelines
  • Overview
  • Recipe configuration
    • Input dataset
    • Metrics
    • Custom metrics
  • LLM Evaluations
  • Comparisons
• Evaluating other models
  • Configuration of the standalone evaluation recipe
    • Labels
    • Sampling
• Analyzing evaluation results
  • The evaluations comparison
  • Model Evaluation details
  • Using evaluation labels
• Automating model evaluations and drift analysis
  • Metrics and Checks
  • Scenarios and feedback loop
  • Feedback loop