Applying elastic net models to credit risk

It is the end of the reporting year and a bank’s credit risk models are recalibrated with new data. The discrimination looks fine. However, the story the model tells about itself does not. If a model’s predictive score is stable but the ranking of its most predictive features changes, then the model is subtly different. This instability in selecting which features are important or not is a significant problem for the independent validation function, and eventually, for a supervisor reviewing the model.

This is the kind of instability that has made elastic net models increasingly attractive to credit risk teams. Elastic net models keep the interpretability of a linear model while handling the correlated predictors that a logistic regression struggles with. For a lender that has to explain each coefficient to a validator, an auditor, and eventually a regulator, that combination matters.

It matters because lending is being pulled in two directions at once.

• More complex data. Teams are building on richer inputs: bureau variables, behavioural scores, affordability metrics, open-banking transaction feeds, and, increasingly, fields drawn from free-text or unstructured sources, the shift our article on the role of data in credit modelling argues matters more than the choice of algorithm.
• A higher governance burden. The bar for any model used in lending decisions continues to rise. Techniques that sit between classical logistic regression and black-box machine learning are getting a serious look.

When linear models wobble

The starting point is understanding where the instability begins. Logistic regression has been the baseline for decades, and with good reason. Each feature gets a single coefficient, and the sign is interpretable. The methodology slots into existing capital and impairment frameworks, and a developer, a validator, and a supervisor can all discuss the model in the same meeting.

What breaks the logic is modern data. A retail credit portfolio can easily generate several hundred candidate features, and more again where an automated method such as the deep feature synthesis we cover separately builds them from transaction data. Many are correlated because they genuinely capture overlapping aspects of credit quality.

Each feature tells you something slightly different. When predictors are correlated, the coefficients of each feature can change considerably. This is a well-known weakness of standard logistic regression, not an edge case.¹

Under most model risk management frameworks, and under SS1/23 from the Prudential Regulation Authority (PRA) in particular, feature stability is not a nice-to-have. Our guide to SS1/23 sets it out in full. When model inputs change from one run to the next, teams spend more time documenting and explaining those shifts.²

The overfitting antidote

Instability in selecting which features are important has two distinct causes. The first is overfitting. The model predicts accurately on data it was trained on, but less well on data it has never seen. The second is correlation between features. Closely related drivers force the model to split predictive power between them, and small changes in the data shift that split. Regularisation is the statistical answer to the first issue; elastic net is designed to help with the second.

A regularised model adds a constraint when trying to minimise the difference between its predictions and the actual values. It forces itself to find the best coefficients within a boundary. This is known as a penalty term. The penalty pulls the model away from perfectly fitting the dataset. It also stabilises coefficients across different training samples, which yields a more stable set of predictive features. Ridge and Lasso are two common regularisation techniques. They differ in the shape of the penalty and in how each treats correlated features.

Picture a room full of light bulbs, where each bulb is a feature and its brightness is how much the model relies on it. Some bulbs are grouped together and others are alone. The closer they are to each other, the more correlated the features are. Ridge is a dimmer wired to each group of bulbs. Turn the dimmer down and every bulb in a cluster gets dimmer together. Nothing ever goes fully off; the room just glows more softly. Lasso is the opposite. Each bulb has its own on/off switch. Lasso flicks the weaker ones off completely and leaves the rest on. The catch is that if two bulbs are right next to each other, Lasso picks one and switches the other off.

Ridge and Lasso each solve part of the problem, neither solves all of it. Faced with two strongly correlated features, Lasso picks one arbitrarily and zeroes the other. This choice can flip between samples. Ridge has the mirror flaw: it keeps everything, even features that carry very little signal.

Lasso and Ridge, blended

An elastic net model takes both penalties and lets the modeller blend them.³ Two parameters control the blend. The mixing parameter, α (alpha), sets the ratio between Lasso and Ridge. At α = 1, the model is pure Lasso. At α = 0, pure Ridge. At intermediate values, it combines both. The strength parameter, λ (lambda), controls how aggressive the shrinkage is overall: how hard the penalty pulls every coefficient towards zero.

The blend of the Lasso and Ridge penalties is a choice, and the choice has to be defended. The practical effect is grouped selection. A cluster of correlated features collectively carrying signal is retained as a group and shrunk jointly, rather than having all but one member zeroed arbitrarily.³

To return to our light bulbs, elastic net models give you both on/off switches and dimmers. Isolated bulbs that are weak can be switched off (not just dimmed) and grouped bulbs that are bright are dimmed together (not just picking one to keep on). So Ridge dims every bulb, Lasso flicks switches, and elastic net models do both.

Explaining the model to a sceptic

Defending the choice of the blending and shrinkage parameters is where the governance work begins. A regulated credit model has to be explainable in three directions at once:

1. To the validation team: the colleagues reviewing the model developer’s work before anyone else sees it.
2. To the model risk committee: the body that approves the methodology and signs off on material changes.
3. To the supervisor: the person or team that may later review the model in a regulatory context.

Each audience has a different tolerance for complexity, but the underlying question is the same: can the model’s behaviour be described in a few clear sentences that survive scrutiny?

Elastic net models tend to score better on explainability than a gradient-boosted tree or a neural network, the families our guide to machine learning in credit risk modelling works through one by one. They remain linear models. Each feature maps to a single coefficient. A developer can point to a positive coefficient on debt-service-to-income and explain, in one sentence, that higher leverage increases predicted default probability. That narrative is what the committee asks for and what a supervisory review expects in writing.⁴

The interpretability is not without its caveats. Defending α = 0.6 rather than α = 0.4 requires evidence from multiple training samples. This is a reasonable cost if it results in a model that a validator can approve.

That is not a guarantee the modelling decisions are right. It is a guarantee that they are arguable.

The cost of more accuracy

Interpretability matters even more when the alternative is a tree ensemble or a neural network. XGBoost, LightGBM, and CatBoost routinely offer a material uplift in discrimination on complex retail portfolios by capturing non-linear interactions between features. The accuracy gain is real. The interpretability cost is real too.

Tree ensembles are harder to explain. The standard workaround is to compute SHAP (SHapley Additive exPlanations) values⁵ or partial dependence plots, which are themselves a layer that can be challenged in validation. For example, a supervisor reviewing a capital model application is entitled to ask whether the SHAP values reflect the model’s actual behaviour in the tail. The answer is not always obvious, and what was meant as an explanation becomes another thing to defend.⁶

Neural networks push the trade-off further. They can exploit very large datasets and capture interactions no linear model would anticipate. The black-box behaviour means the cost of validation tends to be high.

For a retail portfolio with a small, well-understood feature set, the extra uplift from a tree-based model may rarely justify the additional supervisory validation burden. On the other hand, for a larger portfolio with thousands of features, the use of tree-based models or neural networks could be worth the scrutiny. Elastic net models cover the middle ground where the choice is not obvious.

A model worth considering

The goal in credit risk modelling is not to abandon classical methods but to improve them. Techniques that handle richer data while staying explainable will earn their place. Elastic net models are one of those improvements. They need a bit more fine-tuning. But for linear models that introduce much-needed stability during recalibration, they could shine a valuable light.

Frequently asked questions