Diagnostic tests
Resources for teaching LR etc
What does a “significant difference” between treatments mean?
< Back to search results- Format Audio, Texts
- Language/s English
- Target Audience Self-directed learning
- EBM Stage 3 - Appraising evidence, 4 - Decision making
- Difficulty Introductory
Key Concepts addressed
- 2-3g Statistical significance is not the same as importance
- 2-3f Confidence intervals should be reported
- 3-2a Do the outcomes measured matter to you?
Details
What does a “significant difference” between treatments mean?
Robert42 16:22pm Fri 07 Mar 2014
The wording here is not inaccurate but it is unfortunate. Saying that a statistically significant result is unlikely to have occurred by chance implies causality of the treatment being tested. That isn't true. A statistically significant difference is simply one where the measurement system (including sample size, measurement scale, etc.) was capable of detecting a difference (with a defined level of reliability). Just because a difference is detectable, doesn't make it important, or unlikely.
Douglas_Badenoch 17:30pm Fri 07 Mar 2014
@Douglas_Badenoch Hi Robert Thanks for your comment, and important clarification. I think here the authors made a trade-off between precision and getting the message across. There is a similar comment on the previous page (hit Previous above to see it), to which Paul Glasziou, one of the co-authors, gave a reply which is much better than any I could hope to muster. Please check it out and do let us know if you feel it merits further discussion. cheers Douglas