Kaggle S6E9, EV purchases

Nearly every point of AUC above about 0.942 comes from the generator's artifacts. A few thousand income and commute values are heavily over-produced, and the label rate at those values departs from the formula. So I modeled how the data was made rather than what it represents.

Kaggle Playground Series S6E9 asks for the probability that a person buys an electric vehicle, scored by ROC AUC. The train set (668,665 rows) and test set (286,571 rows) are synthetic, generated from a 10,000-row original that is itself synthetic: features drawn from a fixed random state, label from a probit formula on income, environmental concern, subsidy and range anxiety. Nearly every point of AUC above about 0.942 comes from the generator’s artifacts. A few thousand income and commute values are heavily over-produced, and the label rate at those values departs from the formula. So I modeled how the data was made rather than what it represents.

LightGBM on the 13 raw columns scores 0.9419 out of fold. Per-value frequency, lift against the original and novelty features take it to 0.9436, nested target encodings of income and commute at several bin widths to 0.9459, shallow column-subsampled trees with target encodings on every column to 0.9462, and 20 folds with encoding-seed bags and a logit blend of LightGBM, CatBoost and a 15-seed MLP to 0.9464. XGBoost was trained alongside and got zero weight. The last 0.0001 came from hill-climbing blend weights over out-of-fold prediction libraries other competitors had published; they are credited in the repo and not redistributed. The public leaderboard tracked my out-of-fold score within about 0.0001 throughout.

Every idea ran against a fixed reference with a paired DeLong test and went into experiments/results.csv, 192 rows, most of them negative: interaction encodings in every form, similarity to original rows, density windows, digit residues, pseudo-labeling, native categorical handling, deeper or tuned trees, neural-net value embeddings, segment-wise calibration and more, all within ±0.00005 or worse. The accept rule was a paired gain of at least +0.00008 with z ≥ 3, against a noise floor near 0.00003. To see what was left, I resampled labels from the model’s own calibrated probabilities: a perfect model scores 0.9459 ± 0.0003 on those, per cell of concern, subsidy and anxiety the real score matches that ceiling within 0.001, and a model trained to predict the blend’s errors explains none of them (R² = −0.003). The rest is label noise the generator put there.

The competition closed on 2026-09-30. The entry finished 287th of 3,575 teams (top 9%), with a private AUC of 0.94545 against 0.94602 for the winner. The pipeline is public under GPL-3.0 with a rerun playbook, dataset-fact tests, a leak-freeness test for the encodings and a synthetic end-to-end smoke test.