In order to verify and preserve the integrity of a risk management process, it's essential that the process be reviewed and validated periodically.

To be meaningful, a review should encompass the entire process, from data collection and cleansing, to examination of the final results.

The questions which must be answered affirmatively in the validation process are outlined below:

• Was the data utilized in the analysis consistent and of sufficient quality to reliably support the results?
• Were all assumptions defensible?
• Were all relevant risks identified and incorporated into the analysis?
• Did the measurement techniques benefit from the latest scientific refinements and discoveries?
• Is there sufficient buy-in by the relevant parties who have an interest in the risk management process?
• Are those in charge of carrying out the process sufficiently empowered?
• Is the process being used in practise?
• Was there any unjustified or suspicious interference in the process?
• Is there a mechanism in place to capture important feedback about the system's successes and failures (strengths and weaknesses)?
• Does the process meet regulatory guidelines (if relevant)?
• Do the final results meet basic intuition tests? That is, do they make sense?

If the existing risk management system fails any of these questions, the relevant portions of the system must be revisited. This may necessitate recalculation or recalibration of models and approaches.

It's also critically important to ask what can be gleaned from examination of the results. The point of a risk management process is to create infrastructure enabling consistent examinination of sources of uncertainty, leading to better understanding of that uncertainty. Even though the outputs of the system may be imperfect (as will invariably be the case), it's critically important to make the most of any educational opportunity the results allow.

Proper validation should also ensure that there is a procedure for ongoing surveillance and renewal. The latter refers to a mechanism for periodic reviews of the entire validation process, ensuring it benefits from new and better data, more advanced calculation methodologies, and lessons learned.

Return to Top