Guidance for writing a systematic review for inclusion in the crime reduction toolkit.
What to include
Systematic reviews must follow specific criteria to be included in the crime reduction toolkit, including for interventions, type of review and the research outcomes.
Intervention
The systematic review must focus on a single intervention or approach.
The intervention under review must have a crime reduction or prevention focus and outcome measures.
Crime is defined as any behaviour prosecutable by law. These interventions could be aimed at offenders, communities or places. The intervention may be delivered in any setting or country.
Review type
Systematic reviews (such as meta-analyses, narrative reviews, integrative reviews, realist reviews and meta-reviews) should include:
- at least two studies
- an explicit method section explaining how the review was conducted
- sources searched and search terms used
Outcome
The research must include a quantitative measure of crime reduction or prevention.
These outcomes can include data collected at any point during the criminal justice process, including but not limited to:
- calls for service (response to emergency service calls)
- recorded crime (official police records including local, regional and national data)
- victimisation (estimates or self-reported)
- arrests (official records)
- charge or release data (official records)
- remand or bail records (official records)
- conviction (official records)
- sentencing (official records)
- imprisonment (official records)
General measures of crime reduction or prevention are also appropriate, including:
- reoffending (general)
- antisocial behaviour
- self-reported offending
Follow the EMMIE framework
Systematic reviews that meet the inclusion criteria are assessed using the EMMIE framework (effect, mechanism, moderators, implementation and economic cost). Use the guidance to ensure your systematic review meets the criteria for each part of the framework.
Effect – impact on crime
The systematic review needs to include a meta-analysis, other quantitative method of synthesis or narrative summary of quantitative studies.
- For meta-analyses, report overall pooled effect sizes for any findings as well as effect size for any subgroup analysis undertaken.
- Report the analytic methods clearly, including confidence intervals and standard errors wherever possible.
- Clearly state the number of participants in treatment and control groups.
- Clearly state the individual study effects covered by the review.
- Be clear about whether or not findings are statistically significant.
- Be clear about whether any weighting was used when calculating the mean effect size.
- Report on all bias assessments in primary studies considered as part of the review.
Mechanism – how it works
Clearly state the theory of change, including an explanation of how the intervention is presumed to work and impact on the outcome, and the theoretical and empirical evidence that justifies this.
- How is the intervention presumed to work?
- What is needed to make it work?
- What are the anticipated impacts on crime or on behaviour?
Clearly state data being used to test the mechanism or theory of change and whether this analysis was planned as part of the original aims of the review or whether the data led to post-hoc analysis.
Moderators – where it works
Moderator analysis is the analysis of factors that are considered causally relevant to the activation of the mechanisms through which the intervention is expected to work. These are factors that are a pre-existing condition (for example, the age or gender of participants) rather than a factor introduced by the researchers or implementers (like data period).
- Aim to pre-specify any theoretically or empirically derived factors and be clear when this has been done.
- Report any moderator analysis alongside supporting effect sizes.
- Report any literature that has been used to support the notion that the factors being tested are causally significant.
- Report where it has been possible to consult with practitioners or other relevant bodies about the contextual factors which might be causally relevant.
- Report any data enabling moderators (for example, collection of times and places that interventions were implemented or any environmental variables), and any sub-group analysis clearly.
Implementation – how to do it
- Describe the key stages involved in implementing the intervention. As much information as possible about the stages of implementation and the different people or groups that need to be involved is helpful. This also relates to the context, recipients of the intervention and components of the intervention itself.
- Report any challenges and resolutions as well as factors which would aid implementation.
- Clearly identify any key components of successful implementation and how best to ensure they happen.
- Clearly state the key components that are necessary in order to replicate the intervention elsewhere.
Economic Cost – how much it costs
- Quantify the resources necessary for the intervention (for example, 100 hours of staff time or 200 items of hardware).
- Quantify the outputs from the intervention (for example, the number of items installed).
- As far as possible, estimate the direct and indirect financial cost of implementation (overall and by subgroup where appropriate), taking into consideration the cost of any unintended effects.
- As far as possible, estimate the cost-effectiveness of the intervention (for example, the estimated cost of reducing one crime versus the estimated cost of delivering one output), taking into consideration the cost of any unintended effects.
- Clearly state any sources used to estimate the cost of crime.
- Where appropriate, identify the stakeholders involved in an intervention and estimate how the costs and benefits are distributed across those stakeholders.
For more information contact [email protected]