FEATURED BLOG

Author: Kevin Stewart

Audit is defined in the Merriam-Webster dictionary as:  “a methodical examination and review.” When we talk about auditing your Root Cause Analysis (RCA) investigations, we mean just that -- a methodical examination and review. This is easier said than done, especially without some sort of standard to gauge against. If we establish a standard by which we gauge the quality of an RCA, the audit then becomes a simple matter of checking the RCA against the accepted standard and then determining how well it meets that standard. This post is all about helping you establish a standard, and we’ll even give you a free score sheet template to get you started.

Could you have the worst-looking RCA in the world and meet none of the criteria, but have an effective solution that: 

• prevents reoccurrence,
• meets our goals and objectives,
• is within our control, and
• doesn’t cause other problems?

Sure, and it is hard to argue with success. I doubt anyone would say: “Even though this solution will prevent the problem from recurring, it comes from an RCA that doesn’t meet our stringent, high-quality metrics so we can’t use it.” This scenario is entirely possible, though the odds of it are unlikely. If we have a set of measures to check an RCA against to ensure it meets some quality standards, the probability of an effective solution coming from that RCA is greatly increased. 

So what characteristics of an RCA are important?

Here are some questions to consider: 

(If you need a refresher on some of these points, I’ve included the relevant page numbers from the eBook “RealityCharting™: Seven Steps to Effective Problem-Solving and Strategies for Personal Success” by Dean L. Gano.)

• Do the causes pass the noun-verb test? (page 83)

• Do the causes have a lot of unnecessary words or descriptors?
• Do the causal elements pass all logic tests? (page 108)
  • Space-Time Logic Check 
    • Do the causes of this effect exist at the same time? 
    • Do the causes of this effect exist in the same place?
  • Causal Logic Check
    • If you remove this cause, will the effect still exist?
      If the answer to this question is no, then the cause is necessary for the causal relationship and should stay on the chart. If the answer is yes, it should be removed or repositioned.
• Are there any rule violations? If so, what are they and do they pass the minimum standards? Rules to be included are:
  • Are any of the cause boxes empty?
  • Are there any unconnected causes floating around in the chart?
  • Has each cause been identified as an action or condition?
  • Does each effect meet the 2nd principle (causes exist in an infinite continuum – there is an action and a condition for each effect)?
  • Have all conjunctions been eliminated? Remember that “and” is often interpreted to mean “caused,” which can leave too much room for misunderstanding and error. (pages 67-68)
  • Does each cause have the appropriate supporting evidence to justify its inclusion in the chart?
  • Does each branch have some type of stop identified for it? Below are the five potential stops: (pages 88-89)
    • Question Mark – more information needed; an Action Item is created.
    • Desired Condition – there is no need to keep asking why.
    • Lack of Control – something over which you or your organization have no control, for example “laws of physics.”
    • New Primary Effect – a separate analysis is required.
    • Other Cause Paths More Productive – continuing down this path would be a waste of time.
• Does the solution matrix fall into a typical mix such as:

• Have the solutions been judged against a standardized set of criteria with standard ranges to minimize the possibility of favorite solutions being chosen? (page 118-120)
• Has each solution been assigned to a team member and given a due date?
• Does the chart meet all of the four principles of causation? (page 36)
  • Causes and effects are the same thing.
  • Causes exist in an infinite continuum.
  • Each effect has at least two causes in the form of actions and conditions.
  • An effect exists only if its causes exist in the same space and time frame.
• Does the problem definition establish a clear dollar value significance that will let management make informed choices and approvals?
  • If a dollar value is not appropriate (safety near miss or potential fatality) does the problem definition establish a significant value?
• Have all of the action items been resolved? (Action items can include areas where more information is needed, there are evidence issues, or any manually entered items need to be resolved and deleted.)

The next step in developing an audit is to generate a checklist that your RCA will be gauged against.

This list can come from the items above, your own list, or a combination of the two. Once you have a list of items to audit against, you need to generate a ratings scale. This can be a pass/fail situation or a scale that gives a rating from 0 to 5 for each item. This can allow you to give partial credit for some items that may not quite meet the full standard.

Develop a score sheet with each item listed and a place to put a score for each one. Don’t forget to leave a space for notes from the reviewer to explain the reasons for partial credit. It’s handy to add some guidelines with each item to give the reviewer a gauge to score the item against. A sample of such guidelines might look like:

0 = Does not exist
1 = Some are in place but not correct
2 = Many are in place and some are correct
3 = All are in place but only some are correct
4 = All are in place and most are correct
5 = All are in place and correct

With guidelines like these easily available as a reference on your score sheet, it helps ensure consistency in the scoring, especially if multiple people will be scoring an RCA.

Now all you have to do is review an RCA against your list, score it, and have some sort of minimum for passing.

This will ensure that each RCA is measured against a consistent standard that can be repeated by multiple people, though there will always be differences if multiple people are auditing RCAs. Differences can be minimized by either having only one person doing the audit or calibrating the audit, or by bringing all personnel together and scoring several as group so that all auditors understand the scoring nuances.

While I’ve provided a pretty thorough list of what to check for when auditing an RCA, my experience is that an RCA can meet all of the requirements above and still have some issues. The biggest one is that the logic may be correct but the causes may not, so the RCA can pass the tests but it won’t actually fix the issue. The fact is, humans are involved and we make mistakes. Sometimes the errors can be caused by inexperienced investigators that need more practice. Other reasons for error are some of the filters that we talk about, such as time constraints, preconceived notions or biases, language issues, etc. This means that there is still a component that needs to be reviewed by someone for general integrity and for things that a computer just can’t look for. This person can be an external corporate person, a contractor, or an internal resource.

RealityCharting™ has tools that are available to the reviewer to assist them in critiquing the analysis such as rules check, action item report, causal element view, and most importantly there is a dashboard.

FREE Instant Download

To assist you in creating your RCA score sheet, we’re offering a free template.