Six Good Reasons Half of Cyber Risk Quantification (CRQ) Programs May Fail

April 4, 2023  James Graham


Frustration - 6 Reasons CRQ Programs May FailIn a recent press release, Top Eight Cybersecurity Predictions for 2023-2024, Gartner, the leading tech consultancy, included a very provocative set of statistics around cyber risk quantification (CRQ) and how cybersecurity leaders will fare with this approach in the next 2-3 years:

“By 2025, 50% of cybersecurity leaders will have tried, unsuccessfully, to use cyber risk quantification to drive enterprise decision making.”

Author James Graham is Vice President, Marketing for RiskLens

“Gartner research indicates that 62% of cyber risk quantification adopters cite soft gains in credibility and cyber risk awareness, but only 36% have achieved action-based results, including reducing risk, saving money or actual decision influence. Security leaders should focus firepower on quantification that decision makers ask for, instead of producing self-directed analyses they have to persuade the business to care about.”

Despite the negative connotations of that prediction, Gartner’s analysis is essentially that in the next few years, security leaders have a 50-50 chance at making CRQ work for their organizations.  As the leader in the cyber risk quantification space, and having helped hundreds of organizations successfully adopt, implement and operationalize CRQ with Factor Analysis of Information Risk (FAIR™), this statistic is not surprising.

The main challenge with CRQ is that it fundamentally changes an organization’s perspectives on how to measure and manage cybersecurity risk. It’s not something that comes instantly to any organization or cybersecurity culture, and we find that the 50% of organizations who take the right approach to CRQ find more success than the 50% who do not. 

While we wait for additional research details and industry reactions around Gartner’s latest prediction, we have some thoughts about why that 50% of CISOs have…and will fail at implementing CRQ:

1. They are not using a defensible model

In the growing CRQ marketplace, models are everything. Quantifying risk requires a basis for the Value-at-Risk calculation, and there are currently a number of ways that CRQ vendors help their clients measure cyber risk. Some other providers adhere to the FAIR model like we do, but others offer proprietary models or simple scores as the basis for making million-dollar cybersecurity decisions. The thing is, the board is going to ask you about those numbers eventually, and without the proper model to explain your results, anything less than an open, defensible cyber risk model cannot withstand this scrutiny. Read more about FAIR.

2. They are still experimenting

Forrester Cyber Risk Quantification Landscape 2022 CoverThe recent Forrester CRQ Vendor Landscape Report Q4 2022 is a wealth of information about how to gauge the current vendor landscape (including RiskLens).  In this report, the authors note that “Firms in this market also report customers making the transition from experimentation to a more programmatic approach to CRQ, particularly in high-maturity industries such as financial services and critical infrastructure.” For many organizations, CRQ is still being tested and evaluated through small “experiments,” including small-batch analyses with little to no definition of goals or desired results. If CRQ is meant to drive long-lasting results and changes in cybersecurity culture, the experiments must stop at some point and make way for deeper capabilities driven by a well-governed set of processes and outputs. Read More in Forrester’s Cyber Risk Quantification Landscape Report Q4 2022.

3. They are quantifying for quantification’s sake

CRQ Buyers Guide Cover - EmailWe believe the second half of Gartner’s prediction is actually great guidance for the failing 50%. Many organizations start out with CRQ without understanding that CRQ is a means to an end. Success itself may not have anything to do with the process of quantifying risk, but the business reason for quantifying risk in the first place.  In the FAIR Institute’s popular publication, “Understanding Cyber Risk Quantification: The Buyer’s Guide,” Jack Jones, creator of the FAIR model, explains: “Measuring risk isn’t the goal. Risk measurement, whether quantitative or qualitative, is performed so that decisions can be well-informed.” The goal of CRQ, then, is to provide actionable information about probable loss events with real impact on the business to help stakeholders make better-informed decisions.. Read more of Jack’s guidance in Understanding Cyber Risk Quantification: The Buyer’s Guide.

4. They have data-related challenges

Benchmark - Insider Misuse 2For many organizations struggling to implement CRQ, data is a real challenge.  Ultimately, CRQ calculations (including FAIR) rely upon good data to avoid the “garbage in, garbage out” problem. Some organizations have particular difficulties identifying or gathering the right data, and don’t know they can get started more easily using benchmark data, like that available in the free RiskLens My Cyber Risk Benchmark tool, which offers industry-specific data ranges to help organizations understand their top risk themes, including probable frequency and potential financial loss by threat category. RiskLens also provides additional benchmark data points in its annual Cybersecurity Risk Report.

5. They do not have any resources for CRQ

Establishing an enterprise-wide programmatic approach is the end-game of most RiskLens customers, but they don’t all start out that way.  Cybersecurity funding is often a zero-sum game, and sometimes there simply aren’t enough resources to start big.  In these cases, other approaches must be considered in the short term, where the program champion can show the value of CRQ to their teams and stakeholders, until a deeper scope can be properly resourced.  Given our unmatched expertise in CRQ reporting, we often work with under-resourced organizations using our RiskLens Pro Managed Service, which gives CISOs all the benefits of CRQ reporting and decision-making, without the need for in-house resources or expertise.  This approach has given many a resource-strapped organization just the right amount of data and quantified assessment to make the case for a bigger program. Read more about our RiskLens Pro managed service and our Executive Board Reporting Service use case on the site.

6. They are not part of a community

FAIRCON19 Audience View 2Finally, much of the success or failure of a CRQ program comes down to support, including being able to surround yourself with others who have experienced the same challenges and overcome the same hurdles toward building a program. We recommend that anyone implementing a CRQ program of any size or maturity consider joining the FAIR Institute, a 14,000-plus-member nonprofit community of cybersecurity and risk-management professionals from across the globe.  The FAIR Institute is a welcoming and information-rich organization dedicated to helping its members learn more about CRQ and FAIR, and they hold an annual conference that brings these members together for some of the most informative and inspirational presentations and content in the space today. Basic membership is free.

In conclusion, there are likely myriad reasons that half of the CRQ programs cited by Gartner have failed and will continue to fail.  I’ve only mentioned a few here in the hopes that someone recognizes their own challenge and sees a potential fix in the resources I’ve outlined. To me, the bright spot in Gartner’s stat really is that if 50% of CISOs are failing at CRQ, there has to be another 50% that are succeeding. The goal of RiskLens is to meet as much of that other half as possible, so we can get started in changing this stat for the better.