A CECL Validation That Accomplished More Than Just Identifying Issues

The Challenge

The transition to CECL life of loan reserve requirements has introduced new complexities to the modeling process required for forecasting reserves. The biggest challenge in calculating a CECL reserve is moving beyond the historical one-year forecast period used in incurred loss ALLL to the longer life-of-loan estimate under CECL. This typically requires estimating losses for a reasonable and supportable forecast period (typically 1-3 years) that necessitates some level of quantitative analysis followed by a reversion to historical averages.

Many of the models we have reviewed and validated over the past few years have included a regression modeling component where correlations between historical losses and economic variables are established in order to predict losses based on forecasts for those same economic variables for a specified time period. While some institutions develop these regression models internally, many institutions we have worked with are faced with resource limitations and outsource the modeling to third-party model vendors. This can create modeling challenges as the institutions are typically constrained by the standardized approach to the regression modeling offered by these vendors. 

Our Approach

VBC endeavors to provide customized validation services that are fully regulatory compliant and to also offer valuable insights to maximize the ROI of your models as both a risk management and business decision tool. As part of our validation process, we evaluate the vendor model testing and diagnostics and, when relevant and possible (depending on data availability), provide alternative, or challenger models to correct for potential issues.

In a recent validation of a vendor Ordinary Least Squares (OLS) regression model, it was noted that the vendor testing identified the presence of multicollinearity, heteroscedasticity and autocorrelation, three of the more important test areas for OLS models, in many of the loan segments. As part of our validation process, we created alternative, or challenger models to correct for these issues. The challenger models considered three alternatives, a weighted least squares estimator that corrects for heteroscedasticity, a Cochrane-Orcutt estimator that corrects for autocorrelation and a Cochrane-Orcutt estimator applied to a model which included a lagged value of the dependent variable. Focusing on the root mean square error (RMSE) and mean absolute percent error (MAPE), the two models that corrected for autocorrelation showed the greatest improvement compared to the vendor model. 

The Result

The institution was able to implement the results of our alternative models into their overall modeling process to ensure a more conceptually sound process, and using VBC’s model results actually resulted in a lower reserve forecast.    

This is one example of VBC providing custom model validation solutions that are much more than a regulatory check box exercise.  Our mission is to position your financial institution for success by engaging with you in a collaborative partnership, enriching you with valuable expertise and useful data and insights, and empowering you to make impactful decisions. 

Learn more about VBC’s Model Validations service.