Helping Model Risk Management with Machine Learning

Stephen Hsu, SVP, Model Risk Management, Pacific Western Bank

Stephen Hsu, SVP, Model Risk Management, Pacific Western Bank

Based on SR 11-7, the guiding principle for managing model risk is to provide effective challenge of models. Most of the time, the effective change is provided through model validations, model annual reviews, or other forms of independent reviews. The scope of the effective challenge is vast, covering from modeling data, conceptual soundness, choice of methodology, outcome analysis to benchmarking, etc. In order to cover all of the required aspects, it is not surprising that it takes a lot of human resources (internally or externally) and time to provide comprehensive and well-thought-out effective challenges. As a result, model risk managers in the industry started to explore whether there is a more efficient approach to accomplish these tasks and one of the approaches considered is using machine learning.

"Machine learning is powerful, but to be helpful and safe, the user needs to understand how this machine learning engine works"

Machine learning can help to improve efficiency of various tasks in model risk management. It potentially reduces the time of some of the laborious tasks by model risk managers or increases the likelihood to observe some implicit pattern of modeling data. Intuitively, there are some areas that machine learning is able to help model risk management. For example, machine learning can be used to identify whether the segmentation or pooling of the modeling data is reasonable. Another example is on model variable selection. Variable selection is a time-consuming process given that there are various functional forms and lags that could be chosen from. Machine learning can help to identify the importance of each of the risk drivers that potentially could enter the model. And last but not least, model risk managers can use machine learning to build benchmark models that are different from champion models, and then compare and contrast their model behaviors and outputs. Using machine learning to create a benchmark model is considered as a reasonable and sometimes an efficient approach. All of the examples above are some use cases that model risk management team can build out during the course of various model validations or model reviews.

Machine learning is powerful, but to be helpful and safe, the user needs to understand how this machine learning engine works. What are some of the situations that we should be cautious about using machine learning? For example, the user cannot explain the results from machine learning analyses, which lead the analysis to a complete black box. It is important from model risk management perspective that outcome from any model needs to be explainable. Without transparency to model output from machine learning, it is difficult to evaluate the reasonableness of the model. Another example is that the modeling data fed into the machine learning analysis is potentially biased, which lead to biased outcome. Machine learning model learns the pattern from the modeling data and as a result makes decision based on the patterns learned. It’s important to understand the data so that there is no unintended consequence from the model.

In a nutshell, machine learning can potentially help model risk management to be more efficient and assist model risk management team to be more knowledgeable in their work. We expect to observe more and more use cases in the future that apply machine learning in various tasks in model risk management.

Check out: Top Risk Management Solution Companies