Deep feature synthesis for credit risk scorecards

A modeller building an acquisition scorecard for a personal-loan portfolio has the raw material most lenders now envy: months of current-account transactions for each applicant, every payment, transfer, and merchant tagged and timestamped. The problem is that a scorecard cannot read a transaction log. It reads features, one row per customer, each a number that says something about risk. Turning the log into those numbers means deciding that "average inflow over the last three months" or "count of gambling merchants in the last thirty days" might predict default. That part of model development has always been slow, manual, and driven by whoever on the team has the best intuition about the portfolio.

Deep feature synthesis (DFS) is an algorithm that automates that step. Introduced by two MIT researchers in 2015 and packaged as the open-source Featuretools library,¹ it generates candidate features from relational data by traversing the joins that already connect a lender’s tables (customers to their accounts, accounts to their transactions) and stacking simple operations along the way. It matters for credit risk because the raw material of a modern lender is exactly the kind of relational, timestamped data the algorithm was built for, and because, handled correctly, it expands what a scorecard can see without forcing the modeller to surrender the interpretability that regulated lending demands.

That last point is what makes the technique worth a practitioner’s attention now. A 2025 account from Revolut’s credit-management data science team describes deep feature synthesis running in production against transaction data,² and the striking thing is not that its features went into a black box. They went into a weight-of-evidence logistic regression, the most conventional scorecard structure there is.

The bottleneck is the features

The case for automating feature engineering starts with an honest account of where modelling time actually goes. Fitting a logistic regression is minutes of work. Deciding what to fit it on is weeks. When the inputs are relational (an applicant links to accounts, accounts link to transactions, transactions link to merchants), the space of plausible features is enormous, and a human explores only the corner of it their experience points them towards. The original research framed this directly: feature engineering for relational and human-behavioural data "remains iterative, human-intuition driven, and challenging, and hence, time consuming".

The ambition of deep feature synthesis was to show that a machine could search that space credibly. The two MIT researchers built their algorithm into an end-to-end pipeline and entered it into three data science competitions against 906 human teams, beating 615 of them and finishing ahead of a majority of competitors in two of the three. In the third competition the pipeline reached 94% of the best competitor’s score, and its strongest single result, in one of the other two, was 85.6% of teams beaten with 95.7% of the top submission’s score.

The headline is not that automation won outright; it is that automated feature generation reached the neighbourhood of expert human performance on relational prediction problems. For a credit risk team, that is the interesting threshold. It means the automation is good enough to widen the feature search, not that it should be trusted to run unsupervised.

Following the data to a feature

The mechanics are simpler than the name suggests. Deep feature synthesis starts with a lender’s existing tables and the joins that already connect them: customers to their accounts, accounts to their transactions. Against that structure the algorithm applies three kinds of operation. Entity features (efeat) are calculations within a single table, such as turning a transaction timestamp into the hour of the day or the day of the week.

Direct features (dfeat) are values pulled across a link from a related table, carrying an account-level attribute down onto each of its transactions, so that (for example) each transaction inherits the credit limit or opening date of the account it belongs to. Relational features (rfeat) are aggregations that summarise many related rows into one: the count, sum, mean, minimum, or maximum of a customer’s transactions.

The word "deep" refers to what happens when these operations are stacked. The algorithm can take the hour of each transaction (an entity feature), then take the most common hour across a customer’s transactions (a relational feature on top of it), and keep going, following relationships outward and composing operations until it reaches a set limit. That limit is a single parameter, a maximum depth, which the paper writes as d and Featuretools exposes as max_depth. It is the main dial the modeller turns. Turn it up and the features become more expressive; turn it up further and the number of them grows combinatorially.

A shallow run on a modest set of tables can already produce hundreds of candidate features. This is the algorithm’s power and its hazard in one property: it will happily hand a modeller more features than they could ever have written by hand, and far more than belong in a scorecard.

The published algorithm does not prune during generation. It enumerates every composition up to max_depth and defers value judgement to the selection stage described below. There is no branch-and-bound step that watches which transformations are showing promise and expands only those; the algorithm has no view of predictive value while it is stacking operations. This exhaustive-then-select design is what forces the downstream discipline the next section covers. Value-guided variants have been explored in the wider Featuretools ecosystem, but neither the original paper nor the Revolut pipeline uses one.

DFS enables scalable domain-agnostic feature generation, often producing richer representations than manual feature engineering.Spinella and Krisciunas, 2025

Taming the candidate explosion

A pile of thousands of machine-generated features is not a model, and treating it as one is how automated feature engineering earns its bad reputation. The Revolut pipeline is instructive precisely because most of its design effort goes into discipline after generation, not the generation itself.

The pipeline applies a weight-of-evidence transformation to every continuous feature. Each feature is first coarse-binned into a small number of ranges using a decision-tree criterion, with each bin required to hold a minimum share of the population (around 5%) and adjacent bins merged where their default rates are too close or the trend reverses. Every bin is then replaced by a single number: the log-odds of default within that bin relative to the population. This has two useful properties. It linearises the feature’s relationship with the log-odds of default, which is exactly what a logistic regression wants. And it captures a non-linear risk profile at the level of the variable rather than the model, so a linear estimator can still ride the shape of the underlying risk.

Selection is where the candidate explosion is finally controlled. Rather than rank features by their standalone predictive power and hope the top ones combine well, the pipeline uses marginal information value (MIV), a forward-selection procedure that asks a sharper question: given the features already in the model, how much new information does this candidate add? Features are added one at a time by their marginal contribution, and the process stops when performance on a held-out sample plateaus or the best remaining candidate falls below a threshold (around 2%), with a correlation cap (in the region of 40% to 60%) applied to keep the survivors from being near-duplicates of each other.

Matching the ensemble, keeping the glass box

The reason to go to this trouble is that it lets a lender have the predictive reach of transaction data and a model it can still explain. Our guide to machine learning in credit risk modelling sets that trade-off out across model families.

The comparison measure is the Gini coefficient, the standard rank-ordering statistic for a scorecard. On Revolut’s own benchmarks, a weight-of-evidence logistic regression built on synthesised, MIV-selected features matched or beat gradient-boosted tree ensembles in four of five acquisition portfolios: 58% against 54% on one personal-loans portfolio, and a level 70% against 70% on a credit-card portfolio, with the tree ensemble preferred only for a model built on alternative data. As a reference point, a scorecard is generally expected to clear a Gini coefficient of about 50%, so these are working models, not toy examples.

The explanation offered is the one every scorecard modeller will recognise: because the weight-of-evidence step has already absorbed each feature’s non-linearity, a linear model can match a non-linear one while remaining a transparent sum of coefficients. The synthesised features do the heavy lifting; the model stays legible.

The commercial case is made in the same paper through a personal-loans rollout, and it is worth stating in the firm’s own numbers rather than in the abstract.

The uplift came mainly from a higher offer take-up rate rather than a looser approval rate, and it was achieved while delinquency fell, with disbursed loan amounts up 22.71% for customers the previous model would also have accepted. The rollout was run as a three-month A/B test, customers scored at random by the new or the incumbent model, so the effect is measured against a genuine control rather than inferred from a before-and-after comparison. For a technique whose critics reach first for the word "overfitting", a controlled test that holds risk flat while lifting volume is the more persuasive evidence.

What a validator can trace

The governance argument is where the "automated but transparent" framing has to hold up, because a model validator does not care that the features were clever; they care that the features can be traced. Deep feature synthesis, handled this way, gives them three things to hold onto.

• Lineage: each synthesised feature has a definition, a known path of operations over a known set of tables, so where it came from is reconstructable rather than opaque.
• A familiar selection framework: marginal information value sits inside the information-value machinery that scorecard validators already know how to challenge.
• A readable model: the final estimator is a logistic regression whose coefficients can be read directly.

The automation sits upstream, in feature construction; the object that reaches the validation report stays the conventional scorecard. That division is the one supervisory guidance points toward rather than away from. The European Banking Authority (EBA) recommends that institutions "avoid unnecessary complexity in the modelling approach if it is not justified by a significant improvement in the predictive capacities". Among the complexities it names is using unstructured data where conventional data offers similar predictive capacity.³ A pipeline that puts the complexity into feature construction and leaves the estimator simpler passes that test rather than straining against it. The same paper, as a preliminary view it says does not bind the EBA, lists handling unstructured data among the areas where machine learning might prove beneficial in meeting some prudential requirements, because it would expand the data sets available for parameter estimation. The EBA’s August 2023 follow-up report carried the complexity recommendation forward but did not repeat that list.

That does not make the residual risks disappear, and the honest version of the argument names them. Generating features by the thousand raises a multiple-comparisons problem that a correlation cap and out-of-time testing only partly address, and the human-in-the-loop review remains the real control against features that are statistically strong but conceptually unsound.

That review has a place in the UK framework rather than sitting outside it. The Prudential Regulation Authority (PRA) sets out its model risk expectations in supervisory statement SS1/23, and under Principle 1 a firm’s assessment of model complexity may consider the use of unstructured data alongside interpretability, explainability and transparency, and the potential for designer or data bias.⁴ The same pipeline can be run continuously as a monitoring tool, recomputing the marginal information value of the full feature set against a delinquency target to flag when new features would improve an existing scorecard. That is a useful discipline, and also a reminder that an automated feature factory needs automated oversight to match.

This is also where a UK modeller should read the technique against the model risk regime rather than in isolation. SS1/23, effective 17 May 2024, expects firms to identify and manage model risk across the lifecycle regardless of the modelling technique, which means an automated feature pipeline is in scope as much as the model it feeds.

Feature stability over time is the question of whether a synthesised feature keeps its meaning as the portfolio drifts. It becomes a monitoring obligation, not a modelling nicety. Where the final model is not itself transparent, post-hoc tools such as SHAP (SHapley Additive exPlanations) values are the usual fallback, but the appeal of the pipeline described here is that it largely avoids needing them.

The practical implication for a credit risk team is narrow and worth stating plainly. Deep feature synthesis is best treated as an upstream discipline that widens what a scorecard can see, not as permission to hand the model itself to something a validator cannot read. The modeller’s job does not disappear under automation; it moves. Less time spent hand-crafting individual covariates, more spent governing the pipeline that generates them and defending the handful that survive selection. That is a trade most credit risk functions should be glad to make.

Frequently asked questions