Data leakage

In lesson 3.2 cross-validation gave an honest estimate of a model. Honest on one condition: nothing from the test part may touch training beforehand. When it does, we call it data leakage. The model looks great in validation and falls apart in real use. Kapoor and Narayanan collected other researchers’ reports of leakage in 294 scientific papers from 17 fields. Let us start with the cleanest example: data with nothing to learn at all.

Try this

• With the features picked before the split, move k from 5 to 50. How high does the score on pure noise go?
• With the same k, switch to picking inside validation. Where does the bar land, and where do the dots of the 20 new noise datasets?
• In the second part of the lab, find the leaking step in each of the 8 pipelines.

Once you have checked it in the lab, let us name what you see.

The idea in plain words

Data leakage is any information the model would not have at the moment of a real prediction that still reached its training or its evaluation. Validation is meant to play the future: data the model has never seen. Leakage spoils the play, so the score rises while the model is no better at all.

In the lab, among 1000 noise columns there are always some that fit the labels by chance. The best one has a correlation of 0.48 with the label, while about 0.13 is typical for noise. Picking on all 60 samples fits the columns to the samples that later play the test too. Cross-validation then checks the model on data that already shaped it, and shows 81.7%. Picking inside a fold sees only 48 training samples. Columns that fit those by chance say nothing about the 12 test samples, so the score falls to 43.3%, within a coin flip.

Kapoor and Narayanan sorted leakage into 8 types in three groups:

• no clean split: no separate test set, preprocessing fitted on all the data, feature selection on all the data, duplicates on both sides of the split;
• a feature you do not know at prediction time, for example days spent in hospital when the model should predict admission;
• a test set from another distribution than the use: a random split of time data, the same people in training and test, a test set chosen differently from the data the model will work on.

You will find each of them in the second part of the lab.

Jak to policzono

• Noise: 60 samples, 1000 columns from a normal distribution, 30 labels 0 and 30 labels 1 in random order. Generated by the browser with seed 1; the dots are 20 further datasets with other seeds.
• Model: the k columns with the largest correlation with the label (in absolute value), a nearest-centroid classifier, stratified 5-fold cross-validation (6 samples of each class in every fold).
• Biopsies in “Common mistake”: Breast Cancer Wisconsin (Diagnostic), UCI, licence CC BY 4.0, 569 biopsies. Leakage types: S. Kapoor, A. Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science”, Patterns, 2023.

Why it works

An evaluation is honest only when the test data shaped no decision about the model. Feature selection is a decision too, just like filling in missing values, scaling, choosing a threshold or picking the best of several models. If any of them saw the test labels, the test is no longer independent. The bigger the choice (10 columns out of 1000), the more chance fit passes into the score.

One noise dataset could come out like this by chance. But in none of the 20 new datasets in the lab does the leaky score at k = 10 drop below 81.7%. In the code below the same experiment runs on 50 noise datasets in scikit-learn: picking before the split gives 86.4% on average (lowest 73.3%), picking in each fold 50.0%.

Common mistake

True or false: “leakage is just the label hidden in the features”. False. In the lab no column contains the label, and validation still shows 81.7% on pure noise. What leaks is a decision made on all the data. On the biopsies, oversampling the malignant ones before cross-validation gives a recall of 97.1%, and oversampling only in the training data 92.1%, in 50 splits out of 50. The copies land on both sides of the split, so the model recognises biopsies it has already seen. Duplicates in the data work the same way, and a random split of time data puts the future into training.

Not every leak is equally dangerous. A scaler fitted on all biopsies changes the accuracy of a nearest-neighbour model by 0.08 points on average: it does not see the labels, so it carries very little. Feature selection and oversampling use the labels, so they carry a lot. The rule is one: fit every step that learns something from the data on the training part only.

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.model_selection import StratifiedKFold, cross_val_score, cross_val_predict
from sklearn.neighbors import NearestCentroid, KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import recall_score

def noise(seed):  # 60 samples x 1000 columns of pure noise, labels 30/30 in random order
    rng = np.random.default_rng(seed)
    X, y = rng.normal(size=(60, 1000)), rng.permutation(np.repeat([0, 1], 30))
    cv = StratifiedKFold(5, shuffle=True, random_state=seed)
    leak = cross_val_score(NearestCentroid(), SelectKBest(f_classif, k=10).fit_transform(X, y), y, cv=cv).mean()  # 10 columns picked on all 60
    honest = cross_val_score(make_pipeline(SelectKBest(f_classif, k=10), NearestCentroid()), X, y, cv=cv).mean()  # picked inside each fold
    return leak, honest
r = np.array([noise(s) for s in range(50)])
print(f"pure noise, 50 datasets: picked before the split {r[:, 0].mean():.1%} (lowest {r[:, 0].min():.1%}), inside CV {r[:, 1].mean():.1%}")

X, y = load_breast_cancer(return_X_y=True)
y = 1 - y  # 1 = malignant
knn = lambda: KNeighborsClassifier(1)
scale = []
for s in range(50):
    cv = StratifiedKFold(5, shuffle=True, random_state=s)
    before = cross_val_score(knn(), StandardScaler().fit_transform(X), y, cv=cv).mean()  # scaler fitted on all biopsies
    inside = cross_val_score(make_pipeline(StandardScaler(), knn()), X, y, cv=cv).mean()  # fitted on each training part
    scale.append(abs(before - inside))
print(f"breast cancer, 1-NN, 50 splits: scaling before the split changes accuracy by {np.mean(scale) * 100:.2f} points on average")
def oversample(rows, rng):  # copy random malignant rows until both classes are equal
    mal, ben = rows[y[rows] == 1], rows[y[rows] == 0]
    return np.r_[rows, rng.choice(mal, len(ben) - len(mal))]
rec = []
for s in range(50):
    rng, cv = np.random.default_rng(s), StratifiedKFold(5, shuffle=True, random_state=s)
    rows = oversample(np.arange(len(y)), rng)  # copies made BEFORE the split land in the test folds too
    before = recall_score(y[rows], cross_val_predict(make_pipeline(StandardScaler(), knn()), X[rows], y[rows], cv=cv))
    pred = np.empty(len(y))
    for tr, te in cv.split(X, y):  # copies made inside each training part only
        t = oversample(tr, rng)
        pred[te] = make_pipeline(StandardScaler(), knn()).fit(X[t], y[t]).predict(X[te])
    rec.append((before, recall_score(y, pred)))
rec = np.array(rec)
print(f"oversampling malignant before the split: recall {rec[:, 0].mean():.1%}, inside each training part: {rec[:, 1].mean():.1%}; higher before in {(rec[:, 0] > rec[:, 1]).sum()} of 50")

pure noise, 50 datasets: picked before the split 86.4% (lowest 73.3%), inside CV 50.0%
breast cancer, 1-NN, 50 splits: scaling before the split changes accuracy by 0.08 points on average
oversampling malignant before the split: recall 97.1%, inside each training part: 92.1%; higher before in 50 of 50