If you search for “risk assessment” on the Internet, you will quickly find plenty of presentations and templates for carrying out risk assessments. They have become ubiquitous and it is impossible to imagine a multitude of technical and commercial issues without them. What is surprising, however, is the fact that the quality and results of most risk assessments tend to be mediocre to poor. Why is that? And what can be done about it?
What are the typical applications for risk assessments?
In principle, there are many possible applications. A risk assessment is generally useful if you think that an initiative, product or project may entail certain risks that you want to make transparent. A closer look at the subject matter revealed four principal types of situations where risk assessments are used:
- decisions under uncertainty
- choosing between risks/options
- a more precise understanding of a specific risk
- in the process of finding an appropriate risk treatment
These four types could also be understood as stages in a process that develops from “still rather diffuse considerations” to “very concrete problem solving” and could offer different support in each phase. The very nature of the matter therefore makes it necessary to differentiate the objectives and application of risk assessments.
What are typical errors in the context of a risk assessment?
However, most people make mistakes during risk assessments – the tricky thing is that many of them are made unconsciously, for example by simply using templates from the internet without having analysed the problem in detail. The problem is that these mistakes can have far-reaching consequences. From the author’s point of view, typical mistakes can be roughly divided into three categories:
Definitional sources of error: It starts with the question of what stage you are actually at in the process mentioned, including the question of a common definition of risks and the desired goal? It is possible that the starting point has already been set incorrectly and consequently the result either meets different expectations or is not accepted. This is accompanied by discrepancies in relevance and possibly the attempt to assess risks that are not risks.
Procedural sources of error: The actual implementation appears simple at first glance, as there are only three phases according to ISO 31000: Identification, analysis and evaluation. As is often the case, the devil is in the detail here: for example
a) the selection of contributors,
b) the choice of method and
c) the scale for assessment and presentation as a matrix
to name just three critical elements – have a very serious influence on the result. Renowned scientists have proven, for example, that the majority of us are not born statisticians and therefore the use of probabilities – even if it is very widespread – is nonsense in the vast majority of cases, or at best leads to distorted or misleading assessment results. This manifests itself time and again, for example, in the incorrect use of so-called risk matrices.
Cognitive sources of error: The third group of error potentials comprises cognitive aspects that have a strong impact on the process, depending on the situation. To put it bluntly: our heads sometimes play tricks on us. The following three effects are a highly abbreviated selection of findings by Professors Gigerenzer, Kahneman and Tversky, who have spent years intensively studying these issues. They significantly influence the entire judgement process and therefore the result.
- Illusion of certainty: People tend to seek certainty. Sometimes this goes so far that a “precise risk value” (even if it is questionably created or possibly even wrong) is preferred to an approximate amount range (even if this range is much more realistic).
- Anchor effect: During a group discussion on risk assessment, it can happen that someone consciously or unconsciously sets a so-called “anchor” by proposing a value X. This is often followed by a discussion on adjusting the value instead of questioning it completely or setting a completely different value against it. As a result, there is often only a discussion about adjusting the value instead of questioning the value completely or setting a completely different one against it.
- Framing effect: This effect describes the fact that theoretically the same problems should be decided identically by the same people under the same framework conditions, even if they are formulated slightly differently. However, this is precisely not the case: even slight deviations in the problem descriptions (i.e. the “frame”) can lead to completely contradictory or opposing decisions.
What makes a good risk assessment?
Traditionally, the combination of identification, analysis and evaluation is mentioned, as can also be found in ISO 31000. However, these three steps are not nearly precise enough. The 2019 version of the “ISO 31010 Risk Assessment Techniques” standard can be used here. The standard not only describes all phases and steps of a risk assessment in detail, but also lists – as the name suggests – a variety of techniques that can be used or even combined. It is also important to define a precise objective and the designated recipients: who should make which decisions based on the assessment?
With regard to some explicitly mentioned weaknesses in risk assessments, the following tips could be particularly helpful:
- With regard to some of the weaknesses in risk assessments that have been explicitly addressed, the following tips in particular could be helpful:
- Make sure that the chosen approach does justice to the actual complexity of the facts. This also includes selecting a ‘suitable tool / method’ for the assessment.
- Since probabilities are highly susceptible to error in many cases, it is recommended to use frequencies instead.
- Ordinal scales should be avoided wherever possible and replaced by numerical scales. This will then allow calculations to be made at a later stage.
- Work and evaluate in ranges, as exact values (are not possible, but) only fit under special conditions and thus offer a false sense of security.
- Take a closer look at cause-and-effect chains to distinguish real risks from drivers. Risks can be assessed, drivers can be controlled.
- If necessary, consult an expert. Make sure that the assumptions and framework conditions of the risk assessment are well documented for later traceability.
Conclusion: Beware of false security
Carrying out risk assessments is a learning process and if you take at least the majority of the points mentioned into account, the quality of your next risk assessments will improve significantly.
ensec provides you with professional support for training courses and risk assessments.
The article was first published on the RiskNET portal. Source: https://www.risknet.de/themen/risknews/risk-assessments-zwischen-hype-und-performance/
Managed SASE
Managed Infrastructure
Analytics as a Service
Information Security as Service
Security Strategy
Security Architecture
Technology Consulting
GRC
Security Awareness
Cyber Resilience
Network Security
Content Security
Application Security
Cloud Security
Infrastructure Security
OT Security
Endpoint Security
Security Analytics
Team
Working at ensec
Facts and Figures
