What is nested cross-validation and how do I implement it?

What is nested cross-validation and how do I implement it?

Nested cross-validation is a technique used to assess the performance of a model while tuning its hyperparameters. It involves two layers of cross-validation: the inner loop for hyperparameter tuning and the outer loop for model evaluation.

Can you provide a code example for implementing nested cross-validation in Python?

Sure! Here's a simple example using scikit-learn:

from sklearn.model_selection import GridSearchCV, cross_val_score
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

# Load dataset
X, y = load_iris(return_X_y=True)

# Define the model and parameters for tuning
model = RandomForestClassifier()
param_grid = {'n_estimators': [10, 50, 100], 'max_depth': [None, 10, 20]}

# Inner loop for hyperparameter tuning
inner_cv = GridSearchCV(model, param_grid, cv=5)

# Outer loop for model evaluation
scores = cross_val_score(inner_cv, X, y, cv=5)
print('Mean cross-validated score:', scores.mean())

What are the advantages of using nested cross-validation?

Nested cross-validation provides a more reliable estimate of model performance, reduces the risk of overfitting, and ensures that hyperparameter tuning does not bias the evaluation.