Research

Research notes on machine learning for credit risk, algorithmic fairness and demand forecasting.

These notes summarise research directions at the intersection of machine learning and business decision-making:

Sampling bias and reject inference in credit scoring

A credit scorecard is trained on past applications, but repayment behaviour is only observed for applicants who were accepted. Rejected applicants never receive a loan, so their labels are missing. The training data therefore covers only part of the population the model will later score, and this sampling bias distorts both model training and model evaluation.

Reject inference is the family of techniques that tries to recover information about rejected cases. Classic approaches include augmentation (re-weighting accepted cases), extrapolation (labelling rejects with a model trained on accepts) and parcelling. More recent work uses self-learning: the model iteratively labels the rejects it is most confident about and adds them to the training data, with separate regimes for labelling and for final training so that errors do not snowball.

Active learning offers a different angle. Instead of guessing labels, the lender deliberately approves a small number of rejected applications whose outcome would be most informative for the model. This costs money – some of those loans will default – but it gathers genuine labels from unexplored regions of the distribution.

Evaluation suffers from the same bias: a scorecard that looks accurate on accepted applicants can perform much worse on the full population. Bayesian evaluation frameworks extend standard metrics such as AUC to the biased setting by modelling the likely outcomes of rejected cases. Studies that validate these ideas on data from randomised trials, where a random share of applicants is approved, suggest that correcting evaluation often brings larger gains than correcting training.

Profit-driven feature selection

Modern lenders can buy dozens of data sources about an applicant, from bureau data to device information. Each extra feature adds acquisition and storage costs, regulatory burden and a risk of overfitting. Feature selection is traditionally treated as a statistical exercise – keep the variables that correlate with default – but the business objective is profit, not correlation.

Profit-oriented measures such as the Expected Maximum Profit (EMP) criterion translate a scorecard's ranking quality into the profit a lender can expect at the optimal cut-off. Wrapper methods can use EMP directly as a fitness function, so that the selected feature subset maximises profit rather than accuracy.

Framing selection as a multi-objective problem goes one step further. Evolutionary algorithms such as NSGA-II or multi-objective particle swarm optimisation search for a Pareto front of feature subsets that trade off predictive performance, the number of features and data acquisition costs. The decision-maker then picks a point on the front that fits the budget, instead of accepting a single "best" model.

Fairness in credit scoring

Algorithmic decisions in lending are subject to anti-discrimination law, and a model can discriminate even if protected attributes are excluded from its inputs. Group fairness criteria formalise what "fair" means:

Independence (demographic parity): acceptance rates are equal across groups.
Separation (equalised odds): error rates such as false positive and false negative rates are equal across groups.
Sufficiency (calibration or predictive parity): a given score means the same default risk in every group.

The criteria are mutually incompatible in general, so a lender has to choose. Separation is a natural candidate for credit scoring because it compares groups conditional on actual repayment behaviour. Fairness can be enforced by pre-processing the training data, by in-processing – adding a fairness penalty or adversary to the learning algorithm – or by post-processing the scores with group-specific thresholds.

Empirical benchmarks on credit data indicate that the cost of fairness is often moderate: discrimination can be reduced substantially while giving up only a small share of profit, with in-processing methods frequently offering the best balance. The R tutorial on algorithmic fairness shows how to compute these metrics in practice.

Probabilistic demand forecasting with graph neural networks

Retailers forecast demand for thousands of articles, and those articles interact: a promotion on one product cannibalises its substitutes and lifts its complements. Most forecasting models, however, produce independent predictions per article.

Graph neural networks (GNNs) can pass information between related articles. One promising design plugs a GNN encoder into DeepAR, an autoregressive recurrent model that outputs a full predictive distribution rather than a point forecast. Probabilistic forecasts are what inventory planning really needs: the safety stock depends on the upper quantiles of demand, not on its mean.

A practical obstacle is that the product graph is rarely known. Building it from attribute similarity – category, brand, price band, colour – avoids the need for a hand-crafted structure, and the resulting article embeddings turn out to be useful for other tasks such as recommendation and assortment planning. For a gradient boosting take on the same business problem, see the profit-driven demand forecasting post.

Recency and simple heuristics

For more than a century, managers have used a simple rule to find valuable customers: if a customer has not purchased for longer than a certain hiatus, treat them as inactive. This hiatus heuristic ignores almost all available data, yet studies across dozens of data sets – from retail to sports and weather – find that it frequently matches or beats complex stochastic and machine learning models.

The explanation is not small sample size but the dominant role that recency plays in many environments. The lesson generalises: before building an elaborate model, check how far a well-chosen simple signal can go. Sometimes less really is more.