“Everybody supports evaluations but nobody wants to be evaluated”, states a counterterrorism expert on one of those Chatham House Rule conferences. Reflecting on the question How can we evaluate counterterrorism measures? unveils the questions, doubts and in some cases perhaps double attitudes of terrorism fighters towards evaluation. Here some takes on that: “Of course it would be great. But is it really possible? Nobody really ever did it, it is impossible. How to measure success of counterterrorism policies? Is the absence of an attack success? And does an attack automatically imply that counterterrorism failed?” If you want to embark on the path of counterterrorism evaluation, you have to realize that there are no blueprints waiting for implementation. To start with, you have to be very clear about the scope of your evaluation. What measures are you specifically looking at and are the definitions of the indicators you want to analyse clear enough? In getting an evaluation on track, it can help to focus on what the measures were specifically aimed at. Were they supposed to eliminate a threat or “only” to reduce a risk? But how to move on? An important suggestion brought forward is to combine qualitative and quantitative measures in a multidisciplinary approach and to be very precise regarding the focus of your evaluation.

From an international perspective it could be useful to take the universal antiterrorism instruments as point of departure. An evaluation could check how many of those often legal instruments are implemented. The higher the score the better. Although an interesting approach on a global level, counting the score of abstract instruments implemented might not tell enough about the working of extended counterterrorism policies of many western democracies.

Why not bringing in evaluation specialists? There is a lot of expertise on evaluation of all kind of policies and practices. From development aid to running a hospital or steel factory. One objection is that counterterrorism is quite complex and unique. But is that not true for any sector? Another objection is that you need some kind of expertise on counterterrorism in order to perform an evaluation. That would be indeed helpful and can be fixed quite easily. But why are there so few evaluations? Is it because of the problems of secrecy attached to the realm of counterterrorism? Or because counterterrorism is like terrorism also about politics and communication and therefore sensitive?

Fact is that the call for counterterrorism evaluation will stay with us and is likely to get louder once the experiences of the most recent attacks get blurred in public memories. There are already some experiences with evaluation of counterterrorism. In the Netherlands for example the first decade of counterterrorism after 9/11 has been evaluated in 2011, however, by the government itself. The next evaluation is due in 2016. Given the mixed nature of counterterrorism - combining prevention and repression - any future evaluation will have to struggle with the uncertainty about how to measure prevention. Measuring prevention seems impossible but measuring specific interventions in the field of prevention might offer some concrete clues on how to go forward. Another potential fallacy is to get lost in evaluating the economic costs. Of course a critical view on the spending related to counterterrorism is essential. But it remains quite difficult to work out how to count success from an economic perspective. It might be even more difficult to evaluate the non-economic costs like public values and human rights involved. How much of which value do you want to limit to remain safe? Isn’t it in the end that everybody supports evaluations but nobody really knows how to do it?