What is data leakage?

Definition

Leakage can make results appear stronger by exposing answers, future observations or other privileged signals. In historical evaluation, the inputs must reflect what could actually have been known at the specified cutoff.

Preventing leakage requires attention to data provenance, timestamps and how examples are selected or reused. A claimed cutoff is not independent proof that every source and preprocessing step respected it.

ELI5

Imagine practicing for a quiz while accidentally seeing part of the answer sheet. A high score would no longer measure how well you could answer without those extra clues.

For example, a model predicting tomorrow's missed payments should not receive information recorded after the payment was missed. Reviewers must check when every input became available, not just whether it appears in a table.

Can leakage occur in a historical test?

Yes. Future information or later corrections can enter the data unless the full input process respects the intended cutoff.

Does high accuracy prove there was no leakage?

No. Leakage can itself produce unusually strong results, so independence needs separate verification.

  1. The words Gemini Wins at Chess beside a flat blue knight above toppled pieces
    Gemini Wins the AI Chess Tournament
    No Hype AI18m 32s1 VIEW
  2. Portrait of Kristian Fagerlie on a black background with the blue and white headline "FORECASTS NEED EVIDENCE".
  3. Nick Heiner on the left against black beside the headline ‘BENCHMARKS VS REALITY’, blue comparison bars and converging lines.
  4. AI Prediction Battle beside one flat pair of arrows converging on a target
  5. Somya Rai and MD Sharique on either side of the blue-and-white headline “FRAUD HAS NEIGHBORS” against a black background.
  6. David Brumley in a blue top on black beside the headline ‘REAL VULNERABILITIES’ and a blue wireframe shield.
Definition card for data leakage: What is data leakage?

Data leakage occurs when information unavailable or inappropriate at the intended decision point enters a model's training, evaluation or prediction process.