When to Fire a Cannon and When to Use a Precision Knife in Behavioral Design

When designing and testing a behaviorally informed intervention, there are two extreme approaches: (1) Precision Knife Approach and (2) Firing a Cannon.

In the Precision Knife Approach we design a simple intervention that uses only one or two features that vary (independent variables). We subsequently run an experiment (Randomized Control Trial – RCT) to investigate each feature’s effect on the target behavior (Dependent variable).

The Precision Knife Approach is rooted in rigorous academic research. In order to conduct proper (experimental) research, scientists need to investigate the effect on the target behavior (Dependent variable) of each feature that is manipulated (independent variable) and, if more than one, their interaction effect(s).

The advantage of using a Precision Knife Approach is that you get to know how each feature in your intervention works. You know which features used together generate positive interaction effects (i.e. 1+1 > 2) and which features used together generate negative interaction effects (i.e. 1+1 < 2). 

The downside of the Precision Knife Approach is that it faces behavioral designers with a choice between simplistic interventions (i.e. one or two features) that can easily be tested and complex interventions (i.e. 4 and more features) that are incredibly difficult to test.

The difficulty of testing complex interventions (using the Precision Knife Approach) comes from how a correct experimental research design is done. If we have an intervention based on one feature, then we need two experimental conditions (test cells): Control and Intervention. Once we introduce another feature in the intervention the number of test cells doubles. If we introduce a third feature it doubles again (from 4 to 8) and so on.

Having such hyper-complex research designs is impractical for many reasons including costs of designing different variants of the intervention, acquiring a large enough sample to “fill in” all test cells etc.

The other extreme approach is the Firing a Cannon. In a nutshell, this means that when designing the behaviorally informed intervention, you put everything (reasonable) in it and, subsequently, test the entire intervention against a control (do nothing) or / and against the current material used.  

From the point of view of scientific research methodology this is really sloppy. Moreover, it comes with the risk of generating negative interaction effects (1 + 1 < 2).

From a design / practical point of view, the Firing a Cannon Approach is highly useful because behavioral design has the main goal of improving an existing situation through cost-effective and subtle means (interventions). Finding the best – most effective – intervention can be a later goal.

Moreover, the Firing a Cannon Approach requires fewer resources and smaller samples to test the effectiveness of the intervention.

Another reason for which the Firing a Cannon Approach is advantageous is the increased chances of actually getting things done or proving the worth of behavioral design.

Imagine that you go to a (prospective) client or beneficiary with a complex intervention and an extremely complicated RCT design (such as in the Precision Knife Approach). Because most people are scared of complex things, there’s a good chance that the proposition will be rejected.

Imagine that you go to a (prospective) client or beneficiary with a simple intervention using one or two features and the proposition is accepted. You implement the intervention and run the RCT. You find nothing – the intervention doesn’t work. Subsequently you meet with the beneficiary (client) and present the non-results and ask to run another try, this time using different features (tools) in the intervention. Although this is perfectly correct from a methodological perspective, (real) people are not eager to keep investing in things that don’t produce (desired) results.

In the early stages of the behavioral design process (after the research), the Firing a Cannon Approach is superior to the Precision Knife Approach. In the beginning it is important to show that cost-effective behavioral interventions produce results that are equivalent or superior to what is happening at the current stage.

If the project allows for refinement of behavioral interventions, it is possible to use the Precision Knife Approach to fine-tune the materials used.

For more on Behavioral Design take a look at www.naumof.com