Kaggle

Notes on Kaggle competitions across computer vision, NLP, tabular data and time series – the tasks and the approaches that work.

This page collects notes on Kaggle competitions grouped by data type:

How Kaggle works

Kaggle hosts machine learning competitions in which companies and research groups publish a data set, an evaluation metric and a deadline. Participants submit predictions for a hidden test set and are ranked on a public leaderboard during the competition and on a private leaderboard at the end. Only the private leaderboard counts, which is why overfitting to the public scores is one of the most common ways to lose places on the final day.

Beyond competitions, the platform has three more progression tracks – notebooks, data sets and discussion – each with its own tiers from Novice to Grandmaster. Public notebooks are one of the best free learning resources in applied ML: strong baselines usually appear within the first weeks of a competition and are refined by the community until the deadline.

Computer vision

Cassava Leaf Disease Classification. Identifying sick plants from photos of cassava leaves. CNNs and vision transformers trained in PyTorch can be combined in a stacking ensemble, which tends to be more robust than any single architecture.
SIIM-ISIC Melanoma Classification. Training CNNs for skin lesion classification with a severe class imbalance. Pre-training on the full data with surrogate labels before fine-tuning on the target is covered in the pre-training post.
PetFinder Pawpularity Contest. Predicting the popularity of pets with potential adopters from a combination of image and tabular data, a good exercise in mixing CNN or transformer image features with metadata.
RANZCR Catheter and Line Position Challenge. Detecting catheter and tube positions on chest x-rays, where thin lines on large images make input resolution an important design choice.
Prostate Cancer Grade Assessment. Grading biopsies from gigapixel whole-slide images, where the key trick is tiling the slide and selecting informative patches.
SETI Breakthrough Listen. Finding anomalous signals in radio telescope spectrograms – a vision problem in disguise.
APTOS 2019 Blindness Detection. Grading diabetic retinopathy on retina photos; see the blindness detection post.
RSNA STR Pulmonary Embolism Detection. Classifying embolism in CT scans at study and image level. Training on TPUs made the large volumes tractable – see the PyTorch on TPU tutorial.

Natural language processing

BMS Molecular Translation. Translating images of chemical structures into InChI strings with CNN-LSTM encoder-decoders. Described in detail on the portfolio page.
CommonLit Readability Prize. Predicting the reading difficulty of short passages with fine-tuned transformers and custom pooling heads. The text readability post turns the models into an interactive app.

Tabular data

Google Analytics Customer Revenue Prediction. Forecasting future spending from session logs, with a target that is zero for almost every visitor.
IEEE-CIS Fraud Detection. Spotting fraudulent card transactions. The winning ingredient was reconstructing user identities from anonymised columns and aggregating features per user.
Home Credit Default Risk. Classifying risky loan applicants from several relational tables – a classic exercise in feature aggregation for gradient boosting.
OpenVaccine: COVID-19 mRNA Degradation Prediction. Sequence models (GRU and LSTM) predicting degradation rates at each base of an RNA molecule.
Instant Gratification. A synthetic data set where classical models such as quadratic discriminant analysis and Gaussian mixtures beat deep networks.
Mechanisms of Action Prediction. Multi-label classification of drug mechanisms from gene expression and cell viability data with shallow neural networks.

The test-time augmentation post shows one more trick that works on tabular data.

Time series

PLAsTiCC Astronomical Classification. Classifying astronomical objects by their signals – sparse, irregular light curves that call for careful feature extraction before any classifier is trained.
Riiid! Answer Correctness Prediction. Predicting whether a student answers the next question correctly, where fast incremental feature updates are as important as model accuracy.

Lessons that transfer to industry

Competitions are often dismissed as leaderboard chasing, but several habits carry straight over to production work:

Trust the validation scheme, not the leaderboard. A cross-validation split that mirrors how the test data was sampled is the single most valuable asset in any project.
Track every experiment. With hundreds of runs, a structured experiment tracker quickly beats spreadsheets – see tracking ML experiments with Neptune.
Diversity beats a single perfect model. Ensembles of different architectures, seeds and feature sets are more robust than one heavily tuned network.
Engineering unlocks modelling. Tricks like gradient accumulation or layer-wise learning rates make larger models trainable on modest hardware.

The main difference from industry lies elsewhere. A competition fixes the data, the metric and the deadline, while a real project has to negotiate all three: which data can legally be used, which metric reflects business value, and when a model is good enough to ship. The technical toolbox transfers well; the judgement about what to optimise has to be built on the job.