TLDR: 🎯 Accuracy is a lie when classes are imbalanced. Real ML evaluation uses precision (how many positives are actually positive), recall (how many actual positives we caught), F1 (their balance), and AUC-ROC (performance across all thresholds). The right metric depends on your cost function: optimize precision for spam filtering, recall for cancer screening.

---

📖 The 99% Accuracy Trap: Why Your "Perfect" Model is Failing

Your model reports 99% accuracy on a fraud detection dataset, but your bank is losing millions. What went wrong?

Here's the brutal reality: accuracy is meaningless when classes are imbalanced. In a dataset of 10,000 transactions with only 50 fraudulent ones (0.5%), a model that predicts "not fraud" for everything achieves 99.5% accuracy while catching zero fraud cases.

This isn't a hypothetical problem. In 2019, a major credit card company deployed a fraud detection model with 98.7% accuracy. After three months in production, it had missed $2.3 million in fraudulent transactions while flagging thousands of legitimate purchases. The model learned to predict the majority class and ignore the minority class that actually mattered.

The problem isn't the model architecture — it's using the wrong evaluation metric. Accuracy optimizes for overall correctness, but business problems optimize for specific outcomes. Catching fraud, diagnosing cancer, or filtering spam all require metrics that focus on the minority class performance.

This guide covers the essential evaluation metrics every ML practitioner needs: precision, recall, F1-score, AUC-ROC, and when to use each one. We'll work through a complete fraud detection example with scikit-learn to show how these metrics guide real decisions.

---

🔍 The Confusion Matrix: Your Model's Report Card

Every classification metric starts with the confusion matrix — a 2×2 table that breaks down where your model gets confused. Let's use our fraud detection example:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (
    confusion_matrix, classification_report, 
    roc_auc_score, roc_curve, precision_recall_curve,
    cross_val_score
)
import matplotlib.pyplot as plt
import seaborn as sns

# Generate imbalanced fraud detection dataset
np.random.seed(42)
n_samples = 10000
n_fraudulent = 200  # Only 2% fraud - realistic imbalance

# Normal transactions: lower amounts, standard patterns
normal_data = np.random.normal([50, 0.8, 10], [20, 0.3, 5], 
                               (n_samples - n_fraudulent, 3))
normal_labels = np.zeros(n_samples - n_fraudulent)

# Fraudulent transactions: higher amounts, unusual patterns  
fraud_data = np.random.normal([200, 0.2, 3], [100, 0.4, 2], 
                              (n_fraudulent, 3))
fraud_labels = np.ones(n_fraudulent)

# Combine datasets
X = np.vstack([normal_data, fraud_data])
y = np.hstack([normal_labels, fraud_labels])

# Feature names for clarity
feature_names = ['transaction_amount', 'user_reputation', 'time_since_last']
X_df = pd.DataFrame(X, columns=feature_names)

print(f"Dataset: {len(X)} transactions, {sum(y)} fraudulent ({100*sum(y)/len(y):.1f}%)")

The confusion matrix gives us four critical numbers:

graph TD
    A[Confusion Matrix] --> B[True Negative: 2870
Correctly predicted Normal]
    A --> C[False Positive: 32
Normal flagged as Fraud]
    A --> D[False Negative: 8
Fraud missed as Normal] 
    A --> E[True Positive: 52
Correctly caught Fraud]

    style D fill:#ffcccc
    style C fill:#fff2cc
    style B fill:#ccffcc
    style E fill:#ccffff

True Negatives (TN): Normal transactions correctly classified as normal True Positives (TP): Fraudulent transactions correctly caught
False Positives (FP): Normal transactions incorrectly flagged as fraud (Type I error) False Negatives (FN): Fraudulent transactions missed (Type II error)

In fraud detection, False Negatives are expensive (missed fraud costs money) while False Positives are annoying (legitimate users get blocked). Different business contexts flip this relationship.

---

⚙️ Precision vs Recall: The Fundamental Tradeoff

Precision and Recall capture the two sides of classification performance:

Precision = TP / (TP + FP) — "Of all cases I predicted positive, how many were actually positive?" Recall = TP / (TP + FN) — "Of all actual positive cases, how many did I catch?"

from sklearn.metrics import precision_score, recall_score

# Train and evaluate model
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)

y_pred = rf_model.predict(X_test)
y_pred_proba = rf_model.predict_proba(X_test)[:, 1]

precision = precision_score(y_test, y_pred)
recall = recall_score(y_test, y_pred)

print(f"Precision: {precision:.3f} ({precision*100:.1f}%)")
print(f"Recall: {recall:.3f} ({recall*100:.1f}%)")
print(f"\nInterpretation:")
print(f"- {precision*100:.1f}% of fraud alerts are actually fraud (precision)")
print(f"- {recall*100:.1f}% of actual fraud cases were caught (recall)")
print(f"- {(1-precision)*100:.1f}% of fraud alerts are false alarms")
print(f"- {(1-recall)*100:.1f}% of fraud cases were missed")

When to Optimize Each Metric

The precision-recall tradeoff defines your model's behavior:

Key insight: You can't optimize both precision and recall simultaneously. Lower thresholds catch more fraud (higher recall) but create more false alarms (lower precision). Higher thresholds reduce false alarms but miss more fraud cases.

---

🧠 F1-Score: The Harmonic Mean Balance

When you need to balance precision and recall, F1-Score provides a single metric:

F1 = 2 × (Precision × Recall) / (Precision + Recall)

F1-Score is the harmonic mean of precision and recall, which means it's dominated by the lower value. A model with 90% precision and 10% recall gets F1 = 0.18, not 0.50.

from sklearn.metrics import f1_score

f1 = f1_score(y_test, y_pred)
print(f"F1-Score: {f1:.3f}")

# Compare with arithmetic mean
arithmetic_mean = (precision + recall) / 2
print(f"Arithmetic mean: {arithmetic_mean:.3f}")
print(f"Harmonic mean (F1): {f1:.3f}")
print(f"F1 penalizes imbalance more heavily")

When to use F1-Score:

• You need a single metric for model selection
• Both precision and recall matter roughly equally
• You want to penalize extreme imbalances between precision and recall

---

🧠 Deep Dive: How Model Performance Metrics Actually Work

Understanding metrics beyond surface-level definitions requires diving into the mathematical foundations and computational trade-offs that affect real-world model deployment.

Internals: How scikit-learn Computes Metrics

Scikit-learn's metric implementations reveal important computational details that affect performance at scale:

import time

def manual_precision_recall(y_true, y_pred):
    """Manual implementation to show the core logic"""
    tp = np.sum((y_true == 1) & (y_pred == 1))
    fp = np.sum((y_true == 0) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == 0))

    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0

    return precision, recall

# Performance comparison on large dataset
n_samples = 1000000
y_true_large = np.random.choice([0, 1], n_samples, p=[0.98, 0.02])
y_pred_large = np.random.choice([0, 1], n_samples, p=[0.97, 0.03])

print(f"Large dataset ({n_samples:,} samples):")
print("sklearn uses optimized C implementations for speed")

Key implementation details:

• Memory efficiency: sklearn uses vectorized numpy operations
• Edge case handling: Automatic handling of divide-by-zero cases
• Multi-class support: One-vs-all and one-vs-one strategies built-in

Performance Analysis: Metric Computation Complexity

Understanding the computational cost of different metrics helps with production deployment decisions:

def analyze_metric_complexity(sample_sizes):
    """Analyze how metric computation scales with dataset size"""

    results = []
    for n in sample_sizes:
        y_true = np.random.choice([0, 1], n, p=[0.9, 0.1])
        y_pred = np.random.choice([0, 1], n, p=[0.88, 0.12])
        y_proba = np.random.random(n)

        # Time AUC-ROC computation (most expensive)
        start = time.time()
        auc_score = roc_auc_score(y_true, y_proba)
        auc_time = time.time() - start

        results.append({'n_samples': n, 'auc_time': auc_time})

    return pd.DataFrame(results)

# Test complexity scaling
sample_sizes = [1000, 10000, 100000]
complexity_df = analyze_metric_complexity(sample_sizes)

print("Time Complexity Summary:")
print("Precision/Recall/F1: O(n) - linear scan")  
print("AUC-ROC: O(n log n) - sorting required for curve")
print("Cross-validation: O(k × n) where k is number of folds")

Performance insights:

• Simple metrics (precision, recall, F1): O(n) time, O(1) space
• AUC-ROC/AUC-PR: O(n log n) due to sorting for curve computation
• Cross-validation: O(k × n) where k is number of folds

---

📊 AUC-ROC: Performance Across All Decision Thresholds

ROC (Receiver Operating Characteristic) curves plot True Positive Rate vs False Positive Rate across all possible thresholds. AUC-ROC is the area under this curve — a single number summarizing model performance.

from sklearn.metrics import roc_curve, auc

# Calculate ROC curve
fpr, tpr, thresholds = roc_curve(y_test, y_pred_proba)
auc_roc = auc(fpr, tpr)

print(f"AUC-ROC: {auc_roc:.3f}")

# Calculate precision-recall curve  
precision_vals, recall_vals, pr_thresholds = precision_recall_curve(
    y_test, y_pred_proba
)
auc_pr = auc(recall_vals, precision_vals)
print(f"AUC-PR: {auc_pr:.3f}")

ROC vs Precision-Recall Curves: When to Use Each

AUC-ROC interpretation:

• 0.5: Random classifier (no predictive power)
• 0.7-0.8: Decent model
• 0.8-0.9: Good model
• 0.9+: Excellent model

For imbalanced datasets like fraud detection, Precision-Recall curves are more informative because they focus on the minority class performance that actually matters for business outcomes.

---

🏗️ Advanced Model Evaluation Concepts

Beyond basic metrics, production ML systems require sophisticated evaluation approaches for complex scenarios.

Handling Multi-Class Classification

Multi-class problems extend binary metrics through averaging strategies:

from sklearn.datasets import make_classification

# Generate multi-class dataset
X_multi, y_multi = make_classification(
    n_samples=5000, n_features=10, n_classes=4, 
    n_informative=5, random_state=42
)

print("Multi-class F1 Scores:")
print("Macro-average (treat all classes equally)")
print("Micro-average (total TP/FP/FN)")
print("Weighted-average (by class frequency)")

Cross-Validation for Reliable Estimates

Single train-test splits can be misleading:

from sklearn.model_selection import cross_val_score, StratifiedKFold
from sklearn.metrics import make_scorer

# Stratified k-fold (maintains class balance)
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

# Cross-validate multiple metrics
cv_f1 = cross_val_score(rf_model, X, y, cv=skf, scoring='f1')
cv_precision = cross_val_score(rf_model, X, y, cv=skf, scoring='precision')
cv_recall = cross_val_score(rf_model, X, y, cv=skf, scoring='recall')

print(f"Stratified 5-Fold Cross-Validation:")
print(f"F1-Score: {cv_f1.mean():.3f} ± {cv_f1.std():.3f}")
print(f"Precision: {cv_precision.mean():.3f} ± {cv_precision.std():.3f}")
print(f"Recall: {cv_recall.mean():.3f} ± {cv_recall.std():.3f}")

Cross-Validation Best Practices:

1. Use Stratified K-Fold for classification to maintain class balance
2. 5-10 folds is typically sufficient
3. Report mean ± std to show variability

---

🌍 Real-World Model Evaluation Pipeline

Here's a complete evaluation pipeline that combines all these metrics:

def comprehensive_evaluation(model, X, y, threshold=0.5):
    """Complete model evaluation with cross-validation and multiple metrics"""

    # Stratified cross-validation setup
    skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)

    # Cross-validate multiple metrics
    cv_results = {
        'accuracy': cross_val_score(model, X, y, cv=skf, scoring='accuracy'),
        'f1': cross_val_score(model, X, y, cv=skf, scoring='f1'),
        'precision': cross_val_score(model, X, y, cv=skf, scoring='precision'),
        'recall': cross_val_score(model, X, y, cv=skf, scoring='recall'),
        'roc_auc': cross_val_score(model, X, y, cv=skf, scoring='roc_auc')
    }

    # Print cross-validation results
    print("5-Fold Cross-Validation Results:")
    print("=" * 40)
    for metric, scores in cv_results.items():
        print(f"{metric.capitalize():10}: {scores.mean():.3f} ± {scores.std():.3f}")

    return cv_results

# Run comprehensive evaluation
cv_results = comprehensive_evaluation(rf_model, X, y)

Production Model Monitoring

In production, monitor these metrics continuously:

def production_monitoring_metrics(y_true, y_pred, y_pred_proba):
    """Key metrics for production ML monitoring"""

    metrics = {
        'precision': precision_score(y_true, y_pred),
        'recall': recall_score(y_true, y_pred), 
        'f1': f1_score(y_true, y_pred),
        'auc_roc': roc_auc_score(y_true, y_pred_proba),
        'prediction_rate': y_pred.mean(),  # % of predictions that are positive
        'actual_rate': y_true.mean(),      # % of actuals that are positive
        'calibration_gap': abs(y_pred_proba.mean() - y_true.mean())
    }

    return metrics

# Example production monitoring
prod_metrics = production_monitoring_metrics(y_test, y_pred, y_pred_proba)
print("Production Monitoring Metrics:")
for metric, value in prod_metrics.items():
    print(f"{metric}: {value:.3f}")

Key monitoring alerts:

• Precision drop: More false alarms than expected
• Recall drop: Missing more positive cases
• Prediction rate drift: Model behavior changing

---

⚖️ Trade-offs & Failure Modes

Every evaluation metric involves trade-offs. Understanding where metrics fail helps you choose the right approach for your specific context.

Performance vs. Accuracy Trade-offs

Computational Cost vs. Metric Precision:

• Simple metrics (precision, recall) are fast but may miss nuances
• Complex metrics (AUC-ROC) provide more information but require more computation
• Cross-validation gives better estimates but multiplies computation by k-fold

Common Evaluation Failure Modes

When Metrics Mislead

F1-Score Limitations:

• Treats precision and recall equally (may not match business priorities)
• Harmonic mean can mask severe imbalances
• Not suitable when error costs are asymmetric

AUC-ROC Limitations:

• Optimistic on imbalanced datasets
• Doesn't reflect business decision thresholds
• Sensitive to class distribution changes

Cross-Validation Pitfalls:

• Time series data requires time-aware splits
• Small datasets lead to high variance estimates
• Stratification may not preserve important subgroup patterns

Remediation Strategies

---

📊 Visualizing Model Performance Trade-offs

Effective model evaluation requires clear visualizations:

def plot_evaluation_curves(y_test, y_pred_proba):
    """Plot ROC and Precision-Recall curves"""

    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

    # ROC Curve
    fpr, tpr, _ = roc_curve(y_test, y_pred_proba)
    auc_roc = auc(fpr, tpr)
    ax1.plot(fpr, tpr, label=f'ROC (AUC = {auc_roc:.3f})')
    ax1.plot([0, 1], [0, 1], 'k--', alpha=0.5)
    ax1.set_xlabel('False Positive Rate')
    ax1.set_ylabel('True Positive Rate')
    ax1.set_title('ROC Curve')
    ax1.legend()

    # Precision-Recall Curve
    precision_vals, recall_vals, _ = precision_recall_curve(y_test, y_pred_proba)
    auc_pr = auc(recall_vals, precision_vals)
    ax2.plot(recall_vals, precision_vals, label=f'PR (AUC = {auc_pr:.3f})')
    baseline = sum(y_test) / len(y_test)
    ax2.axhline(y=baseline, color='k', linestyle='--', alpha=0.5,
                label=f'Random ({baseline:.3f})')
    ax2.set_xlabel('Recall')
    ax2.set_ylabel('Precision')
    ax2.set_title('Precision-Recall Curve')
    ax2.legend()

    plt.tight_layout()
    plt.show()

# Generate evaluation curves
plot_evaluation_curves(y_test, y_pred_proba)

This visualization reveals:

• ROC curve: Overall discriminative ability
• PR curve: Performance on imbalanced data
• Baseline comparison: How much better than random

---

🧭 Decision Guide: Choosing the Right Metric

Your choice of evaluation metric should align with business objectives:

flowchart TD
    A[Classification Problem] --> B{Classes Balanced?}
    B -- Yes --> C[Use Accuracy + ROC-AUC]
    B -- No --> D[Use Precision/Recall + PR-AUC]

    D --> E{Error Cost Asymmetric?}
    E -- False Positives Costly --> F[Optimize Precision
📧 Spam Detection]
    E -- False Negatives Costly --> G[Optimize Recall
🏥 Cancer Screening]  
    E -- Both Matter --> H[Use F1-Score
⚖️ Content Moderation]

    C --> I[ROC-AUC for Model Selection]
    F --> J[High Precision Threshold]
    G --> K[Low Recall Threshold] 
    H --> L[Balanced F1 Threshold]

Quick decision framework:

1. Balanced classes → Accuracy + ROC-AUC
2. Imbalanced classes → Precision/Recall + PR-AUC
3. False positives costly → Optimize Precision
4. False negatives costly → Optimize Recall
5. Both errors matter → Balance with F1-Score

---

🧪 Practical Implementation: Complete Fraud Detection Pipeline

Here's a production-ready evaluation pipeline:

from datetime import datetime

class ModelEvaluator:
    """Production model evaluation with comprehensive metrics"""

    def __init__(self, model, threshold=0.5):
        self.model = model
        self.threshold = threshold
        self.evaluation_history = []

    def evaluate(self, X_test, y_test, dataset_name="test"):
        """Run comprehensive evaluation"""

        # Get predictions
        y_pred = self.model.predict(X_test)
        y_pred_proba = self.model.predict_proba(X_test)[:, 1]

        # Calculate all metrics
        metrics = {
            'timestamp': datetime.now(),
            'dataset': dataset_name,
            'n_samples': len(y_test),
            'positive_rate': y_test.mean(),
            'threshold': self.threshold,

            # Core metrics
            'accuracy': (y_pred == y_test).mean(),
            'precision': precision_score(y_test, y_pred),
            'recall': recall_score(y_test, y_pred),
            'f1': f1_score(y_test, y_pred),
            'auc_roc': roc_auc_score(y_test, y_pred_proba),

            # Business metrics  
            'false_positive_rate': (y_pred[y_test == 0] == 1).mean(),
            'false_negative_rate': (y_pred[y_test == 1] == 0).mean(),
            'prediction_rate': y_pred.mean()
        }

        # Store evaluation
        self.evaluation_history.append(metrics)
        return metrics

    def print_summary(self, metrics):
        """Print evaluation summary"""
        print(f"Model Evaluation - {metrics['dataset'].title()} Set")
        print("=" * 50)
        print(f"Samples: {metrics['n_samples']:,}")
        print(f"Positive Rate: {metrics['positive_rate']:.1%}")
        print()
        print("Classification Metrics:")
        print(f"  Accuracy:  {metrics['accuracy']:.3f}")
        print(f"  Precision: {metrics['precision']:.3f}")
        print(f"  Recall:    {metrics['recall']:.3f}")  
        print(f"  F1-Score:  {metrics['f1']:.3f}")
        print(f"  AUC-ROC:   {metrics['auc_roc']:.3f}")

# Use the evaluator
evaluator = ModelEvaluator(rf_model, threshold=0.5)
test_metrics = evaluator.evaluate(X_test, y_test, "holdout_test")
evaluator.print_summary(test_metrics)

This production pipeline:

• Tracks evaluation history for model monitoring
• Includes business metrics beyond standard ML metrics
• Provides clear summaries for stakeholder communication

---

📚 Lessons Learned: Real-World Evaluation Pitfalls

After evaluating hundreds of models in production, here are the critical lessons:

1. Accuracy is Almost Always the Wrong Metric

The problem: Accuracy optimizes for overall correctness, not business outcomes. The fix: Choose metrics based on the cost of different error types. Example: A 99% accurate model that misses all fraud is worthless.

2. Single Metrics Hide Important Trade-offs

The problem: F1-score can hide whether you're good at precision or recall. The fix: Always report precision and recall separately. Example: F1=0.7 could mean balanced 70%/70% or imbalanced 95%/54%.

3. Cross-Validation Prevents Overfitting to Test Sets

The problem: Iterating on a single test split leads to implicit overfitting. The fix: Use cross-validation for model selection, holdout for final evaluation. Example: Tuning 20 hyperparameters on the same test set invalidates your results.

4. Threshold Tuning is More Important Than Algorithm Choice

The problem: Default threshold=0.5 is rarely optimal for business problems. The fix: Tune thresholds based on precision-recall curves and business costs. Example: Moving from 0.5 to 0.3 threshold increased fraud detection by 23%.

5. Monitor Distribution Drift, Not Just Performance Metrics

The problem: Performance metrics lag behind data distribution changes. The fix: Monitor prediction rates, feature distributions, and calibration. Example: Model precision drops after feature distributions shift post-COVID.

---

📌 Summary & Key Takeaways

Model evaluation is about aligning metrics with business objectives, not maximizing scores.

Essential Takeaways:

1. 🎯 Accuracy lies — Use precision/recall for imbalanced problems
2. ⚖️ Understand tradeoffs — Precision vs recall based on error costs
3. 📊 Use multiple metrics — F1, AUC-ROC, AUC-PR for complete picture
4. 🔄 Cross-validate — 5-fold stratified CV for reliable estimates
5. 📈 Tune thresholds — Default 0.5 is rarely optimal
6. 🔍 Monitor production — Track drift in predictions and performance
7. 💼 Business context matters — Choose metrics based on real costs

Metric Selection Cheat Sheet:

• Balanced classes: Accuracy + ROC-AUC
• Imbalanced classes: Precision/Recall + PR-AUC
• High cost of false positives: Optimize Precision
• High cost of false negatives: Optimize Recall
• Need single metric: F1-Score or Business-specific metric

The Bottom Line:

Your model's 99% accuracy doesn't matter if it fails to achieve business objectives. The right evaluation metric depends on what you're optimizing for in the real world. Fraud detection optimizes for catching fraud (recall), spam filtering optimizes for avoiding false alarms (precision), and medical diagnosis optimizes for not missing cases (recall).

Choose your metrics wisely — they determine what your model learns to optimize for.

---

📝 Practice Quiz

Test your understanding of model evaluation metrics:

Question 1: A cancer screening model has 95% precision and 40% recall. What does this mean?

• A) The model is highly accurate overall
• B) 95% of positive predictions are correct, but 60% of cancer cases are missed
• C) The model has good precision but poor recall
• D) Both B and C

Question 2: When should you use AUC-ROC vs AUC-PR?

• A) ROC for balanced classes, PR for imbalanced classes
• B) ROC for binary classification, PR for multi-class
• C) Always use ROC as it's more standard
• D) PR is only for regression problems

Question 3: A fraud detection model achieves F1=0.6 with precision=0.9 and recall=0.45. How can you improve recall?

• A) Lower the decision threshold
• B) Raise the decision threshold
• C) Use more training data
• D) Switch to a different algorithm

Question 4: Why is cross-validation important for metric evaluation?

• A) It gives higher accuracy scores
• B) It prevents overfitting to the test set
• C) It's required for model deployment
• D) It automatically tunes hyperparameters

Question 5: You're building a spam filter where false positives (legitimate emails marked as spam) are very costly to users, but false negatives (spam getting through) are just annoying. How should you optimize your model and what threshold strategy should you use?

Explain your reasoning, including which metric to optimize, approximate threshold range, and how you would validate this approach in production.

Question 6: Your medical diagnosis model shows AUC-ROC=0.85 on balanced test data, but AUC-PR=0.45 on the same data with 5% disease prevalence. The model is being deployed in a population screening scenario.

• A) The ROC score indicates the model is good enough for deployment
• B) The low PR score suggests the model will have poor precision in production
• C) You should focus on the ROC score since it's higher
• D) Both scores are measuring the same thing, so average them

Correct Answer 1: D) Both B and C - High precision means 95% of positive predictions are cancer cases, but low recall (40%) means 60% of actual cancer cases are missed.

Correct Answer 2: A) ROC for balanced classes, PR for imbalanced classes - ROC can be misleading when the positive class is rare because it's influenced by the large number of true negatives.

Correct Answer 3: A) Lower the decision threshold - This will classify more cases as positive, increasing recall but likely decreasing precision.

Correct Answer 4: B) It prevents overfitting to the test set - Cross-validation provides more robust estimates by testing on multiple data splits.

Correct Answer 6: B) The low PR score suggests the model will have poor precision in production - With 5% disease prevalence, PR curves better reflect real-world performance than ROC curves.

---

🔗 Related Posts

• Machine Learning Fundamentals: A Beginner-Friendly Guide - Start with the basics of ML concepts and terminology
• Supervised Learning Algorithms: Deep Dive into Classification - Learn the algorithms behind classification models
• Machine Learning Fundamentals Learning Roadmap - Complete learning path for ML fundamentals
• Neural Networks Explained: From Neurons to Deep Learning - Understanding the models behind modern ML