Author: Kevin Stewart
This question was posed to a discussion group and it got me thinking how do you grade an investigation?
The overall success will be whether the solution actually prevents recurrence of the problem. One definition of Root Cause Analysis is: “A structured process used to understand the causes of past events for the purpose of preventing recurrence.” So a reasonable assessment of the quality of the analysis would be to determine whether the RCA addressed the problem it set out to fix by ensuring that it never happens again (this may be a lengthy process to prove if the MTBF of the problem is 5 years, or has only happened once). 
Are there some other tangibles that can help you assess the quality of an RCA? RCAs use some sort of process to accomplish their task. If this is the case then it would stand to reason that there will be some things you can look for in order to gauge the quality of the process followed. While this is no guarantee of a correct analysis, ensuring that due diligence was followed in the process would lend more credibility to the solutions.
What are some of these criteria by which you can judge an analysis?
- Are the cause statements ‘binary’? By this we mean unambiguous or explicit. A few words only and precise language use without vague adjectives like “poor” since they can be very subjective.
- Are the causes void of conjunctions? If they have conjunctions there may be multiple causes in the statement. Words such as: and, if, or, but, because.
- Is there valid evidence for each cause? If causes don't have evidence they may not belong in the analysis or worse yet solutions may be tied to them and be ineffective.
- Does each cause path have a valid reason for stopping that makes sense? It is easy to stop too soon and is sometimes obvious. For example, if a cause of “no PM” has no cause for it so that the branch stops, it would seem that an analyst in most cases would want to know why there was no PM.
- Does the structure of the chart meet the process being used? If it is a principle-based process then it should be easy to check the causal elements to verify that they satisfy those principles. These might be causal logic checks or space time logic checks or others that were associated with the particular process.
- Is the chart or analysis completed? Does it have a lot of unfinished branches or questions that need to be answered or action items to complete?
- Is the chart or analysis completed? Does it have a lot of unfinished branches or questions that need to be answered or action items to complete?
- Are the solutions SMART (Specific, Measurable, Actionable, Relevant, and Timely)? Or do they include words like: investigate, review, analyze, gather, contact, observe, verify, etc.
- Do the solutions meet a set of criteria against which they can be judged?
- Do the solutions address specific causes or are they general in nature? Even though they may be identified against specific causes if they don’t directly address those causes then it may still be a guess.
- If there is a report, is it well written, short, specific and cover just the basics that an executive would be interested in? Information such as cost, time to implement, when will it be completed, a brief causal description and solutions that will solve the identified problem are the requisites.
These are some of the things that I currently look at when I review the projects submitted by clients. I’d be interested to know about other things that may be added to the list.
