Lies, Damned Lies, and Statistics

PROGRAM DESCRIPTION: This Mini Expert Series will show how flaws in study design and statistical analysis may lead to false conclusions from research studies. The two experts use a point-counterpoint, light-hearted debate to present literature and their misleading conclusions, with rebuttals that expose the key fallacies upon which the analyses were based. An interactive didactic review of the concept follows each debate. This series will help providers and researchers interpret the literature that impacts clinical practice, with rotating topics each year.

Examples of specific topics that have been covered include:
1. Multiple hypothesis testing. Multiple hypothesis testing, when not handled correctly, commonly leads to false positive findings that later may be proven false with further data.
2. Regression to the mean. Apparently dramatic effects may be artificial if the baseline measures were selected from among the outliers.
3. Control groups. Exaggerated treatment effects may result from the absence of appropriately selected control groups.
4. Historical controls. Secular trends can compromise comparisons.
5. Blinding. Unconscious bias affects data collection, and blinding controls for it.
6. Confounding. False associations can occur when important confounding variables are not accounted in the analysis.
7. Intention to treat. The benefits of randomization persist only if this analytic method is employed.
8. Stage migration. Improved diagnostic testing can up-stage the disease stage at presentation, which makes treatment artificially appear more effective.
9. Equivalence trials. To be valid, these trials must be powered adequately to rule out a difference between the new treatment and the standard treatment, and the standard treatment must be proven effective a priori.