Randomised controlled trials (RCTs) are considered one of the most rigorous and scientific methodologies to determine whether a cause-effect relationship exists between treatment and outcome, allowing researchers to exclude the possibility that the association was caused by an alternative factor (Sibbald & Roland, 1998).
The random allocation to intervention groups, or randomisation, renders the groups comparable both in terms of known and unknown factors, facilitates blinding, and is also supposedly beneficial in terms of allowing the statistical analysis of the results by ensuring their statistical significance (Stang, 2011)—even if this statistical significance can sometimes be too fragile to draw definitive conclusions from a randomised study (Walsh et al., 2014).
This fragility can be mitigated by performing systematic reviews and meta-analyses of existing evidence from RCTs to draw a more robust conclusion when examining the comparative efficacy of psychotherapeutic interventions for specific mental health conditions. They must also be designed rigorously and conducted carefully to ensure the validity of the findings (Greco et al., 2013).
Beside randomisation, RCTs must have the following design characteristics: intervention groups are of representative sample size; both clinicians and patients are blind to which treatment is given until the completion and the study; identical intervention are delivered for all intervention groups except for the experimental treatment (Chalmers et al., 1981).
These design characteristics address many shortcomings of other study methods—for example, observational studies, where all the variables are not under the control of the researcher, are by essence subjective, and controlled case studies and cohort studies are subject to bias—but are not always applicable to study the effectiveness of psychotherapy. For example, it is impossible to double-blind the study of cognitive-behavioral therapy (CBT) because the patients are actively involved in the therapy (Berger, 2015).
In the field of psychotherapy, the extreme rigour of RCTs can in fact become a limitation, not accounting for the complex range of individual differences and the multifaceted nature of mental health disorders (Shean, 2014). While RCTs are the gold standard to prove the efficacy of a particular treatment delivered in a controlled environment in comparison to a placebo treatment, the results are often challenging to generalise, and do not provide support in choosing the most efficient treatment in a real-world setting (Tunis et al., 2003).
In other words, not all treatments proven to be efficacious through a RCT would necessarily improve a clinician’s ability to help patients if they were introduced into the practice settings to treat the population for which they were designed, and a RCT cannot prove that a treatment would improve mental health care in actual practice (Essock et al., 2003).
To study the effectiveness of psychotherapies, it is required to question some assumptions that are usually present in RCT studies of other therapies: that most patients can be treated for a single disorder; that disorders can be treated independently of personality factors; and that disorders are highly malleable. In fact, comorbidities are the norm in psychopathologies, and mental health disorders are highly connected to personality factors, resistant to change, and prone to relapse, which makes the study of their long-term effectiveness challenging (Westen et al., 2004).
An additional difficulty in using RCTs to study the effectiveness of psychotherapies stems from ethical concerns from the medical community, including the risks inherent in receiving the experimental treatment rather than the treatment-as-usual, and the lack of data proving the efficiency of the current treatment needed to ethically justify a clinical trial rather than a direct treatment (Stines & Feeny, 2008). Once a psychotherapeutic treatment has shown some effectiveness in clinical practice, it can indeed prove difficult or even impossible to recruit clinicians for RCTs, because of the perceived lack of viable alternative treatment and ethical doubts about the randomisation process (Fairhurst & Dowrick, 1996).
RCTs may not be the only valid design to evaluate the efficacy of psychotherapies, and researchers are advocating for a more pragmatic approach that addresses the current shortcomings discussed above. Observational studies are incredibly seen as complementary to RCTs, and new forms of data collection and therapeutic treatment delivery methods facilitated by connected devices and electronic health records may provide researchers and clinicians with a broader repertoire of research tools to study the efficacy and the effectiveness of psychotherapies (Bothwell et al., 2016).
References:
Berger, D. (2015). Double-blinding and bias in medication and cognitive-behavioral therapy trials for major depressive disorder. F1000Research, 4.
Bothwell, L. E., Greene, J. A., Podolsky, S. H., & Jones, D. S. (2016). Assessing the gold standard—lessons from the history of RCTs.
Chalmers, T. C., Smith Jr, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., & Ambroz, A. (1981). A method for assessing the quality of a randomized control trial. Controlled clinical trials, 2(1), 31-49.
Essock, S. M., Drake, R. E., Frank, R. G., & McGuire, T. G. (2003). Randomized controlled trials in evidence-based mental health care: getting the right answer to the right question. Schizophrenia Bulletin, 29(1), 115-123.
Fairhurst, K., & Dowrick, C. (1996). Problems with recruitment in a randomized controlled trial of counselling in general practice: causes and implications. Journal of health services research & policy, 1(2), 77-80.
Greco, T., Zangrillo, A., Biondi-Zoccai, G., & Landoni, G. (2013). Meta-analysis: pitfalls and hints. Heart, lung and vessels, 5(4), 219.
Shean, G. (2014). Limitations of randomized control designs in psychotherapy research. Advances in Psychiatry, 2014.
Sibbald, B., & Roland, M. (1998). Understanding controlled trials. Why are randomised controlled trials important?. BMJ: British Medical Journal, 316(7126), 201.
Stang, A. (2011). Randomized controlled trials—an indispensable part of clinical research. Deutsches Ärzteblatt International, 108(39), 661.
Stines, L. R., & Feeny, N. C. (2008). Unique ethical concerns in clinical trials comparing psychosocial and psychopharmacological interventions. Ethics & behavior, 18(2-3), 234-246.
Tunis, S. R., Stryer, D. B., & Clancy, C. M. (2003). Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. Jama, 290(12), 1624-1632.
Walsh, M., Srinathan, S. K., McAuley, D. F., Mrkobrada, M., Levine, O., Ribic, C., … & Thabane, L. (2014). The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index. Journal of clinical epidemiology, 67(6), 622-628.
Westen, D., Novotny, C. M., & Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: assumptions, findings, and reporting in controlled clinical trials. Psychological bulletin, 130(4), 631.