Sharpening residential mortgage default assessment
Pop quiz. Two different parts of the country. Two houses. Two mortgages. Same credit score. Identical LTV. Which loan has a higher likelihood of defaulting? The answer, unsurprisingly, is: ‘It depends’. It depends on the local economic conditions. It depends on how house prices move in those areas. The area where house prices decline more will have a higher likelihood of the loan defaulting. The loan in the area with more jobs, better wages, and higher growth prospects is less likely to default.
A typical assessment of mortgage loan defaults requires the borrower’s credit score, loan-to-value (LTV), and some facet of individual financial capacity. An individual’s financial capacity changes with economic conditions. And house price movements are also linked to local economic conditions. Which brings up the simple question: Can incorporating local economic conditions improve loan-level mortgage default assessment? The answer: Yes!
We compared two models to predict loan-level defaults. The Champion model incorporated the borrower’s credit score and LTV as predictors. The Challenger model added HousingIQ’s Housing Market Vitality Indicators (HMVI) corresponding to the property’s metro housing market as an additional predictor. The result? Incorporating HMVI as a predictor creates a statistically and economically valuable improvement in default predictions.
Quantifying the advantage
The Challenger model’s Kolmogorov Smirnov statistic is nearly ten percentage points higher than the Champion model’s statistic value of 45.5%. So? By incorporating HMVI, the Challenger model is able to discriminate better between defaults and non-defaults.
Based on the ROC Curve, the Challenger model performs better than the Champion at each business choice of sensitivity and specificity. Based on the corresponding Area Under the Curve (AUC) metric, the Challenger model outperforms the Champion model by over 4 percentage points. So? Incorporating HMVI as a predictor increases the probability of correctly risk ranking any two loans by over four percentage points.
When dealing with credit, investor risk appetite dictates a level of acceptable defaults. A false alarm—incorrectly flagging a loan as one that will default—represents a lost business opportunity. When evaluating default risk models, we prefer models that not only correctly identify defaults but also do not raise false alarms. The Precision-Recall Curve and its associated AUC represent this concept.
The Challenger model outperforms the Champion model by over 50%. So? The Challenger model is more correct. By incorporating HMVI as a predictor, the likelihood that a loan that was predicted to default actually defaults increases by over 50%.
At HousingIQ, we prefer the metric Diagnostic Odds Ratio. Borrowed from the medical world, it measures the effectiveness of a medical test. Its value ranges from 0 to infinity with values greater than one corresponding to greater discriminatory power. In the vocabulary of an investor, the Diagnostic Odds Ratio summarizes how well a model preempts bad decisions by correctly identifying defaults and how much it allows good decisions by not raising false alarms.
The Challenger model outperforms the Champion model by a factor of 5. So? By incorporating HMVI, there are far fewer false alarms and unpleasant surprises.
Conclusion
Housing Market Vitality is an additional predictor of residential mortgage defaults that yields statistically and practically significant improvements. In the instant analysis, incorporating HMVI as a predictor in addition to the usual credit score and LTV attributes yielded substantial improvements based on multiple yardsticks – KS Statistic, ROC, Precision-Recall, and Diagnostic Odds Ratio. Since the HMVI are updated monthly, similar analysis can provide ongoing portfolio surveillance and prevent surprises.
Contact us to learn how we can sharpen your decisioning and portfolio monitoring capabilities.
Methodology TL;DR
We examined a portfolio of over 19,000 residential re-performing loans (RPL) acquired during 2013-14. The median credit score upon acquisition was 681 and the median LTV was 98%. Within 18 months of acquisition, 3.1% of the loans had defaulted. We define 90-days delinquency as the default event.
We fit a gradient boosting tree model to predict defaults. The Champion model used credit score and LTV as predictors and the Challenger model used two additional predictors—the metro area Housing Market Vitality Indicator (HMVI) upon acquisition and the corresponding trailing twelve month change in the HMVI value.
The model was tuned using repeated cross-validation to avoid overfitting and the log loss metric was used to select between competing models. We prefer the log loss metric as it heavily penalizes mistakes.
Download the data underlying this analysis.