Enabling Your Risk Analysts to Succeed: Trust but Clarify

January 13, 2020  Tim Wynkoop

Adopting a quantified cyber risk management program in your organization can seem like a bit of a leap of faith. While we all want a world with more rigor and defensibility baked into our risk management programs – how can you trust your teams to produce risk analyses that are accurate and useful when at the end of the day they often rely on the knowledge and expertise of their colleagues?

The first thing to note is you’re not in this alone. Through the RiskLens Academy, your team can receive FAIR™ training taught by accredited trainers either live or online. The introductory course, FAIR Analysis Fundamentals, is great for all levels of the organization from leadership to the subject matter experts who will be providing the estimates that feed into the RiskLens platform to generate FAIR-based analyses.  It provides the knowledge necessary to know and apply consistent risk terminology, use various measurement concepts to select scenarios for analysis and estimate risk factors using probability distributions, understand and interpret the results of a FAIR analysis, and more.

If you’re looking for a little more know-how from the analysts who will be performing the FAIR analyses, the RiskLens Academy also offers the FAIR Analyst Learning Path, which is an advanced training course designed to take participants with a foundational understanding of FAIR to the next level with four advanced courses, each covering one phase of the risk analysis process.

Using the information gained from training, as well as the hands-on guidance provided by the RiskLens Professional Services team to our customers, your organization will be empowered to conduct FAIR quantitative risk analyses that enable risk-based decision making.

Now, even with all the training in the world, if your analyst is given a bad estimate it can result in inaccurate results. Often the cause of the bad estimate is not a lack of knowledge of the subject matter expert’s part, but rather a miscommunication during the data gathering session. In order to avoid such miscommunications, we have a piece of advice for any quantitative risk management program: Trust but Clarify.

Trust but Clarify: 3 Ways to Avoid Bad Estimates in FAIR Analysis 

At the end of day, we need to rely on historical information, industry data, and the knowledge and expertise of the subject matter experts in the organization in order to produce accurate estimates for FAIR analysis. In order to ensure those estimates are as accurate as possible, keep these three things in mind:

Keep it in Context – What Is the Purpose of the Analysis?

In data gathering sessions, always be sure to use context specific questions to ensure the subject matter expert knows exactly what you are talking about. For example, a question related to the threat event frequency of a database breach might look something like “how many times in a given year do you think a malicious external actor will attempt to compromise the PII contained in XYZ database?.”  The question makes clear the specific scope – asset, threat, and effect – you are interested in knowing more about.

Also – always make sure the SME knows the context of the “big picture” – why are you conducting this analysis in the first place? If they know it is related to a specific control improvement, for example, they may be able to help inform you what information is or is not relevant based on the implementation of that control.

Learn more: How To Scope A Risk Analysis Using FAIR

Gut Check – Do the Numbers Make Sense?

Once the estimate has been given, take a moment to apply it to the analysis logically: Does the number(s) make sense? If the end result is a value that is 10x the annual revenue of the organization, there is a good chance there was an inaccurate assumption at some point.  Sometime doing some simple math can help.

One common cause of the miscommunication is a difference in measurement – make sure the SME is aware that frequency is measured in a given year (i.e. how many events do we think will occur in the next year?) whereas magnitude is measured on a per event basis (i.e. how much will it cost each time this event happens?)

Learn more: Secrets to Gathering Good Data for a Risk Analysis

When in Doubt – Rely on Your Training

Before ever conducting a FAIR risk analysis, the analyst should be thoroughly trained on all things FAIR, including calibrated estimation and the quality assurance process. It is never too far into your FAIR journey to revisit those concepts and make sure they are being applied to the analysis. If a SME provides an estimate on the fly that appears to lack rigor, try using the equivalent bet test (spinner game) to help them calibrate it to a more useful degree of precision.

And always remember – once all of the estimates have been gathered it is important to conduct a Q/A session with all of the SMEs so that they can do their gut check as well.

Learn more: Download the (One-Page) RiskLens FAIR Analysis QA Guide