Clinical Trials for Rare Diseases: Problems and Perspectives, part 2

Part 2, methodological challenges.

Clinical studies in the field of rare diseases (RD) face a number of methodological problems. In some cases, traditional parallel-group randomized controlled trials with generally accepted rates of type I and type II errors (0.05 and 0.20) are difficult or even impossible to conduct.

  • Small sample size limits the statistical power of the study. While large treatment effects have a good chance to detected, slight or moderate changes can hardly reach the statistical significance threshold of 0.05.  

  • Study samples in RD are often not only constrained, but also heterogeneous. Heterogeneity arises from poor knowledge on RD pathogenesis, improper diagnostics, or soft inclusion criteria established to recruit more participants. Both sample paucity and heterogeneity enhance the degree of random imbalance in covariates between the groups. As a result, such trials are prone to bias, and the quality of evidence produced is in jeopardy.

  • In the case of severe diseases with shortened lifespan, using placebo can be unethical. Therefore, a trial design should allow all patients to be exposed to an active treatment.

What are possible solutions?

  1. When there is no way to enroll more individuals, relaxing type I and type II error rates could be considered acceptable. For example, if RD is a serious condition with unmet medical need, a treatment having a probability of false positive trial results of 0.10 instead of 0.05 may be better than no treatment at all.

Ioannidis et al. developed a model of optimal choice of type I and type II errors according to the available sample size and the plausible effect sizes1. The equations reported can be used to maximize the chances of correct inferences and minimize the chances of wrong inferences when the available sample size is fixed.

However, care should be taken in applying this approach. Trials in RD are time and resource consuming and may never be repeated. Therefore, erroneous conclusions can accumulate in clinical practice without a chance of being refuted.

  1. Selection of more sensitive outcome measures can be of great help. This requires the development of new clinical rating scales to better quantify disease activity/progression, which is impossible without deeper insights into RD pathophysiology. Furthermore, according to current understanding, outcomes should be not only more reflective of the disease characteristics, but also more patient-centered. In other words, clinical trials should focus on outcomes which are important for patients and their families. Thus, involvement of multiple stakeholders would be beneficial, and those who suffer from RD should be placed in the heart of the decision about the best end points for each trial2.

  2. If a conventional clinical trial design is infeasible, one can consider alternative design options as well as more complex and flexible statistical techniques. Such approaches can mitigate the effects of clinical heterogeneity, extract maximal possible information from small groups of individuals, and avoid or minimize the use of placebo. All the alternative methods have their own inherent advantages and drawbacks and are applicable only under certain circumstances.

Detailed description of these approaches can be found in the recently published reviews3,4. Abrahamyan et al.4 have also provided a conceptual framework for selection of appropriate design depending on various factors related to the study.

Examples of advantages and disadvantages of several alternative clinical trial designs.

Design

Main benefits

Main limitations

Within-patient designs

Crossover design

N-of-1 trial

  • All participants receive active treatment

  • Resistant to clinical heterogeneity

  • Small sample size can be appropriate (in n-of-1 trials, only one patient is needed)

  • Condition must be stable over time

  • Only rapid and reversible effects can be studied

  • Vulnerable to carry-over and period effects

  • Highly sensitive to drop-outs

  • N-of-1 trials: limited amount of evidence is produced

Adaptive designs

Response-adaptive randomization

  • More patients can be allocated to more effective treatment

  • Vulnerable to selection bias

  • Requires rapid responses

Sequential trial

  • Information about futility, benefit and harm can be obtained earlier

  • Small sample size can be appropriate

  • Treatment effect must occur quickly relative to recruitment

  • Increased risk of bias

  • Complexity

Other

Enriched enrollment randomized withdrawal trial

  • Shorter time of exposure to placebo

  • Can improve trial efficiency

  • Treatment effect cannot be generalized  to all patients with the studied RD

  • Suitable for stable diseases or conditions with slow evolution

  • Vulnerable to carry-over effects

  • Complexity

Randomized placebo phase trial

  • All participants eventually receive active treatment

  • Applicable for treatments that may produce a lasting remission or response

  • Higher risk of drop-outs

  • Complexity

Alternative analysis

Bayesian analysis

  • Flexible and not constrained by traditional type I error rate

  • Relevant prior information can be incorporated in the study

  • Small sample size can be appropriate

  • Prior probability distribution may be uncertain

  • Increased risk of bias

  • Can be unfamiliar to regulatory authorities and peer reviewers

  • Complexity

Information was obtained from the reviews3,4

Novel methods for the design and analysis of RD clinical trials are now being proposed. The Integrated Design and Analysis of small population group trials (IDeAl) project5, funded by the Seventh Framework Programme of the EU, was finished in 2017. The results reported by the IDeAl consortium6 include:

  • a new methodology for the selection of the best practice randomization procedure and subsequent analysis for a small population clinical trial taking possible bias into account

  • statistical methods to adapt the significance level and allow confirmatory decision-making in clinical trials with vulnerable, small populations

  • new methods for sample size calculation, type 1 error control, model averaging and parameter precision in small populations group trials within non-linear mixed effects modeling

References

[1] Ioannidis, J. P. A., Hozo, I. & Djulbegovic, B. Optimal type I and type II error pairs when the available sample size is fixed. J. Clin. Epidemiol. 66, 903–910.e2 (2013).
http://dx.doi.org/10.1016/j.jclinepi.2013.03.002

[2] International Rare Disease Research Consortium. Patient-Centered Outcome Measures in the Field of Rare Diseases. Accessed 27 July 2018.
http://www.irdirc.org/wp-content/uploads/2017/12/PCOM_Post-Workshop_Report_Final.pdf

[3] Tudur Smith, C., Williamson, P. R. & Beresford, M. W. Methodology of clinical trials for rare diseases. Best Pract. Res. Clin. Rheumatol. 28, 247–262 (2014).
http://dx.doi.org/10.1016/j.berh.2014.03.004

[4] Abrahamyan, L. et al. Alternative designs for clinical trials in rare diseases. Am. J. Med. Genet. Part C Semin. Med. Genet. 172, 313–331 (2016).
http://dx.doi.org/10.1002/ajmg.c.31533

[5] Integrated Design and Analysis of small population group trials (IDEAL). Accessed 27 July 2018.
http://www.ideal.rwth-aachen.de/

[6] Final Report Summary – IDEAL. Accessed 27 July 2018.
https://cordis.europa.eu/result/rcn/203187_en.html

Image: