9March 2019In the current competitive environment, future business strategies will depend both on thoughtful data and analytical strategies· Assess which variables contribute to the outcome being modeled as well as the stability and business sense of those relationships.· Conduct statistical testing of the model fit as well as testing of the sensitivity to different populations or scenarios to understand under what circumstances the model performs.· Benchmark the candidate model against alternatives, including different model methods and model design choices as well as the selection of different predictive variables.· Evaluate the model performance across different time periods, populations and/or scenarios to ensure that the model can perform in production across a range of circumstances.· Test the data and model code in production to ensure that the coding is correct and that data flows retain the data integrity.In production, the model will require monitoring to ensure that it continues to provide reliable results. This monitoring will provide valuable insights into potential model improvements as well as insights for management into shifts in the customer population or transaction patterns. This testing should include:· Statistical tests of the model's fit and tests the model's performance against expectations. · Key assumptions, such as the customer profile, customer behavior patterns and demand, economic or market environment, or other changes in external influencers should be monitored as material changes in these factors can render the model less reliable. · As a backup, a benchmark model may be run in parallel with ongoing improvements to eventually replace the champion model. In addition to ongoing testing to monitor the model performance and robustness, the storage of production data, as well as model results is required for reporting and future analysis. If this is not prioritized, the cost will be that sufficient data may not be maintained for advancements in analytics that provide the necessary competitive advantage.In the current competitive environment, future business strategies will depend both on thoughtful data and analytical strategies. These are often imbedded in new technology platforms to streamline and optimize business processes. It is the responsibility of executives to work with risk management and technology groups to ensure that these platforms fully perform as advertised without unintended consequences. For this reason, functions such as Model Risk Management are strategic partners in innovation.
< Page 8 | Page 10 >