Introduction

This blogpost, based on a forthcoming article, emphasizes the need for reliable decision support in today’s cyber sphere, where decision makers must benefit from assessments and advice based upon differing points of view. Devil’s advocacy, which criticizes established positions, and offers alternative perspectives to a given argument based upon the same inputs, is one instrument to try to achieve this. Israeli decision makers have been able to use this instrument since the Yom Kippur war of 1973, when a Devil’s Advocate office was established within Israeli military intelligence.

Enter the Devil’s Advocate

A Devil’s Advocate is anyone with a dissenting view (authentic or non-authentic) that takes a contrary position. Devil’s advocacy is an established contrarian technique within the growing research body of Structured Analytic Techniques (SAT). Its aim is to test the validity of propositions by seeking to prove the opposite of the challenged view. It serves as a check on groupthink, which otherwise would dismiss contradictory evidence leading to poor assessments. Dissent – even when it is wrong – stimulates divergent thinking and the consideration of alternatives, which ultimately improves the quality of decisions.

Devil’s advocacy also helps tackle the neglected, but fundamental problem of the so-called ‘Alpha and Beta Chance.’ The alpha-chance is the chance of incorrectly concluding that there is a significant relationship between phenomena. The beta-chance is the chance that a relationship between phenomena is overlooked. The alpha-chance is usually placed at 5%, i.e. evidence of the relationship between described phenomena is only accepted if observed in at least 95 out of 100 instances. The beta-chance is usually positioned between 20-90%, which indicates the likelihood of missing weak, but existing, relationships.

These percentages are inacceptable in many risk calculations and assessments, including war. Here, the beta-chance should be put lower (in order not to miss a weak, but vital relationship), and the alpha-chance higher (making a relationship significant sooner). An analytic technique such as devil’s advocacy appears to minimize the margin of error when it comes to the Alpha-Chance, while maximizing the chance of discovering significant relationships (Beta Chance).

The aftermath of Yom Kippur

Yom Kippur had shown poor comprehension on all executive levels despite the wealth of intelligence available. As a result, decision making suffered badly. The shock of the Yom Kippur war spurred the creation within Israeli military intelligence of the Mahleket Bakara (Department of Control), or Ipcha Mistabra. The concept of Devil’s Advocacy was introduced for quality assurance when producing intelligence assessments. The new office sought to stimulate openness, allowing dissenting opinions to be voiced in accordance with the agency’s slogan: ‘Freedom of opinion, discipline in action.’

In several instances the contradictory opinion of the Israeli Devil’s Advocate has made an impact. One of the best known concerns the 2006 Lebanon war against Hezbollah, when an Israeli naval vessel sailing off the coast of Lebanon was hit by a missile killing four people. The ship’s anti-missile system had not been activated since navy intelligence denied that Hezbollah possessed this kind of capability. The devil’s advocate had argued – contrary to the position of naval intelligence – that reports showed that Hezbollah possessed Iranian surface-to-sea missiles.

Does the Israeli experience with Devil’s advocacy support its efficacy? The available sources (e.g. 1, 2). Show that institutional groupthink, organizational obstacles and clashing egos have not disappeared, but are partly contained due to the existence of this form of ‘control’. As a safeguard against group think, the Devil’s Advocate has instilled an atmosphere of accountability within the analytical process. Analysts have to argue their analysis and be prepared to deal with critique, making it more difficult for individuals to act as a ‘single point of failure’. Overall, the contradictory stance of the Devil’s Advocate has served as a check on organizational tunnel vision. Decision makers can be provided with additional points of view, which reduces information gaps and makes their choices more robust.

Devil’s Advocacy on the cyber horizon?

What Devil’s Advocate lessons are in store for the cyber future? Many cyber concepts are new, their possible consequences unknown, and (technological) development is moving at break neck speed. The information gap is increasing, which makes it more difficult to balance Alpha and Beta chance considerations and reach informed decisions.

In practice, decision makers will always be imperfectly informed, either about the outcomes that will occur (prospective uncertainty) or about what has transpired (retrospective uncertainty). However, cyber developments have multiplied this problem exponentially. Controlling the current information deluge requires self-regulation through institutionalized feedback mechanisms such as devil's advocacy. This minimizes the discrepancy between the organisation’s goal(s) and its performance by feeding back information to the decision makers.

Successful use of a Devil’s Advocate demands unrestricted access to source material. This effectively equals a paralyzing data deluge for any Devil’s Advocate. The available data is of such magnitude today, that it can only be handled ‘industrially’ through automated processing, i.e. software algorithms.

Consequently, devil’s advocacy within the cyber domain will have to understand software and hardware processes and concentrate, first of all, on algorithm reviews. These reviews can serve as a check on correlations that hide themselves within algorithms. Correlation is often mistaken for causation, although it only implies probability and even strong correlations might be coincidence. It should be realized that - by definition – an analysis based on statistical probabilities will always produce false positives (criminalizing innocent people) and false negatives (unnoticed security risks).

Conclusion

The need for decision support in today’s cyber sphere appears obvious. The (cyber) decision maker can use a differing point of view, a clear voice that can act as an intellectual sparring partner. Devil’s Advocacy can test the validity of cyber propositions and prevent cognitive pitfalls. It can also challenge algorithmic decisions and prevent them from being taken at face value. By changing perspective, battling group think, assigning Alpha and Beta chance problems their proper place, and reducing the information gap, the quality of cyber assessments will improve and thereby the cyber decision making process overall.