Leakage check

Target: survived (classification) · 1309 rows · 13 features

Findings: 2 HIGH · 0 MEDIUM · 0 LOW

Severity Check Column What it means
HIGH missing_predicts_target (also: target_proxy) boat Just knowing whether boat is empty predicts the target (AUC 0.971). Values that get filled in only after the outcome look exactly like this.
HIGH one_sided_value body When body is filled in, the target is 0 in 100% of 121 rows (overall only 62%). A value that exists for only one outcome is usually recorded after the outcome.

How well each column predicts the target on its own

Out-of-fold AUC of a model that only sees one column (0.5 = useless, 1.0 = perfect).

Column AUC
boat 0.968
sex 0.748
fare 0.688
ticket 0.677
home.dest 0.662
pclass 0.654
cabin 0.620
parch 0.573
body 0.571
sibsp 0.570
age 0.568
embarked 0.568
name 0.499

Next steps

  1. For every HIGH column: confirm when it is recorded, then drop it (or rebuild it from past data only).
  2. Split before any fitting: scalers, imputers, encoders and feature selection go inside a Pipeline that is fit on the training part only.
  3. Use the right split: by time for forecasting, by group when one person/item has many rows.
  4. Run this check again and compare the scores before and after.