What is k-fold cross-validation and how do I apply it?

What is k-fold cross-validation and how do I apply it?

K-fold cross-validation is a technique used to assess the performance of a model by dividing the dataset into 'k' subsets. The model is trained on 'k-1' subsets and tested on the remaining subset. This process is repeated 'k' times, with each subset used as the test set once.

How do I choose the value of k?

The value of k is typically chosen based on the size of your dataset. A common choice is 5 or 10, but for smaller datasets, you might use a value closer to the total number of samples.

Can you show me a code example for k-fold cross-validation?

Sure! Here's a simple example using Python's scikit-learn library:

from sklearn.model_selection import KFold
from sklearn.linear_model import LogisticRegression

kf = KFold(n_splits=5)
model = LogisticRegression()

for train_index, test_index in kf.split(X):
    X_train, X_test = X[train_index], X[test_index]
    y_train, y_test = y[train_index], y[test_index]
    model.fit(X_train, y_train)
    print(model.score(X_test, y_test))

What are the advantages of using k-fold cross-validation?

K-fold cross-validation provides a more reliable estimate of model performance compared to a single train-test split. It helps in reducing variance and ensures that every data point gets to be in both training and test sets.