Randomised controlled trials, gold standard or fool’s gold?

< Back to search results

The role of experimental methods in voluntary sector impact assessment.

• Format Texts
• Language/s English
• Target Audience Researchers
• EBM Stage 0 - Why EBM?
• Duration >15 mins
• Difficulty Advanced

Key Concepts addressed

• 2-1a Comparison groups should be similar
• 2-1c Comparison groups should be treated equally
• 2-1d People should not know which treatment they get
• 2-1g All should be followed up
• 2-1h Peoples' outcomes should be analyzed in their original groups
• 3-2c Are the treatments practical in your setting?
• 1-2e Comparisons are needed to identify treatment effects
• 1-2f Consider all of the relevant fair comparisons

Details

The role of experimental methods in voluntary sector impact assessment.

Summary

RCTs can be powerful, but may not always be an appropriate methodology for assessing the impact of social programmes. And their practical application in the UK voluntary sector is, currently at least, limited.

The resources and skills needed to do a quality RCT, combined with ethical issues and the need for additional context and process data to understand the results, can limit their relevance to many voluntary sector organisations. Other methods, while perhaps not being gold standard, may still produce very useful results, and it’s important to remember that when choosing evaluation design it’s about what methods are appropriate to the evaluation questions and the intervention being studied.

The debate must continue as to the role of quasi-experimental methods in impact evaluation. Further, using qualitative methods in impact assessment to construct a counterfactual needs more exploration. More on this from CES soon, in our upcoming article on the use of qualitative approaches to assess impact.

Read more…