The vendor assessment teams we’ve had the opportunity to work with are great. More often than not they are forward thinking people who have a passion for digging through what seems like endless amounts of questions and documents all in the effort to nail down the risk stemming from their third parties.
Yet I’ve noticed a glaring shortfall of many vendor assessment programs when it comes to assessment process and the model by which they assign their “risk ratings”. Regrettably what I’ve experienced, whether it be a homegrown assessment process, or something based off of the Standardized Information Gathering (SIG) questionnaire, is that there is very rarely a model that underpins the assessment process. Unlike the model that underpins FAIR, which clearly outlines how risk is derived, many vendor assessment teams lack any such process when assigning their vendor assessment risk ratings. The consequences are as follows:
Assessments are subjective rather than objective
Subjectivity and objectivity live on a spectrum. The goal of any assessment process should be to drive more objectivity into our assessments, as opposed to allowing our own subjective and biased experiences to dictate the outcome.
Assessments are more likely to be inconsistent
Without a sound, repeatable process by which to assess vendors, each assigned rating is more likely to be inconsistent from one vendor to the next, as well as one analyst to the next.
Can’t normalize data, nor benchmark
The above statement has two further implications. The above means that normalizing risk ratings over the population of vendors becomes especially difficult if each vendor is assessed using a different model – mental or formal.
Can't stand behind recommendations
More importantly, it also means that the vendor assessment team will have an especially difficult time standing behind their recommendations and conclusions. By having a model, like the RiskLens Bayesian models that underpin our Cyber Risk Maturity and Third Party applications, assessments are:
- More objective in their approach, by factoring in only those items that assesses an organization’s risk posture, internal or external.
- More consistent from analysis to analysis, as well as analyst to analyst, as each assessment utilizes the same frameworks regardless of vendor or assessor.
- Recommendations and conclusions are more defensible, as the underlying model and assessment process is repeatable and well vetted.