Secondary endpoints cannot be validly analyzed if the primary endpoint does not demonstrate clear statistical significance

doi:10.1016/S0197-2456(97)00075-5

Controlled Clinical Trials

Volume 18, Issue 6, December 1997, Pages 550-556

https://doi.org/10.1016/S0197-2456(97)00075-5 Get rights and content

Abstract

There is lack of consensus surrounding the interpretation of observed treatment effects for secondary clinical endpoints when the primary endpoint for which the clinical trial was initially designed does not meet the objective of a demonstrated effect. We provide some arguments to support caution in making inferences for secondary endpoints in this situation. We examine the definitions of primary and secondary endpoints within the context of a hypothesis-testing framework for multiple endpoints, and we address the relationship of the correlation structure of these endpoints and the statistical adjustments needed to preserve experiment-wise type I error for a valid inference. We also address the hypothesis-testing framework and the estimation framework for valid inference, focusing on the interpretation of p-values associated with differentially powered hypothesis tests for each endpoint to detect an important clinical effect. We point out the limitations on the strength of evidence (and quantification of uncertainty) for a secondary endpoint effect that can be derived from only one study and introduce the likelihood of replication of the finding in another study of identical size and design as a useful concept to guide this interpretation.

References (5)

CPMP Working Party and Efficacy of Medicinal Products
Note for guidance: biostatistical methodology in clinical trials in applications for marketing authorization for medical products
Stat Med
(1995)
T Capizzi et al.
Testing the hypothesis that matters for multiple primary endpoints
Drug Info J
(1996)

There are more references available in the full text version of this article.

Cited by (98)

Editorial Commentary: Knee Lateral Extra-articular Tenodesis Procedures: Appropriate Study Design Is Essential for Reducing Conflicting Findings and Unnecessary Controversy in the Orthopaedic Literature
2024, Arthroscopy - Journal of Arthroscopic and Related Surgery
Knee lateral extra-articular tenodesis procedures (LEAPs) reduce graft rupture rates when performed at the time of anterior cruciate ligament (ACL) reconstruction. However, in the setting of revision ACL reconstruction, LEAPs are less studied and remain controversial. Many studies support combined procedures (ACL + LEAP), yet others do not. When the literature comprises small patient cohorts and short follow-up periods, conflicting results often arise. The controversy surrounding them may be unnecessarily generated by the publication of low-quality studies. Future studies should focus on adequate power; appropriate design and methodology, including matching or randomization to account for potential confounding factors; proper statistical analyses; and avoidance of spin bias.
A randomized, sham-controlled, quintuple-blinded trial to evaluate the NET device as an alternative to medication for promoting opioid abstinence
2022, Contemporary Clinical Trials Communications
There is an unmet need for non-medication approaches to illicit opioid discontinuation and relapse prevention. The NET (NeuroElectric Therapy) Device is a non-invasive, battery-operated, portable, re-useable device designed to deliver bilateral transcranial transcutaneous alternating current electrical stimulation, and is intended to treat opioid use disorder (OUD) without medication. The device is a CE-marked Class IIa, non-significant risk, investigational medical device.
This prospective trial (NRC021) tests the hypothesis that the NET Device provides safe and effective neurostimulation treatment for persons with OUD who express a desire to be opioid abstinent without medications for opioid use disorder (MOUD).
NRC021 is a randomized, parallel-group, sham-controlled, quintuple-blinded, single-site study. Persons with OUD entering a residential treatment facility for opioid detoxification are assigned to active or sham treatment (n = 50/group). Group assignment is stratified on presence of any current non-opioid substance use disorder and by sex. The biostatistician maintains the blinding so that the study sponsor, principal investigator, research assistants, treatment staff, and participants remain blinded. Following discharge from the inpatient facility, participants are assessed once weekly over 12 weeks for substance use (using timeline followback interview and video assessment of observed oral fluid sample provision and testing). The primary efficacy endpoint is each participant's overall percentage of weekly abstinence from illicit opioid use without use of MOUD. The secondary efficacy endpoint is each participant's percentage of non-opioid drug-free weeks. Safety outcomes are also measured.
NRC021 is designed to assess the efficacy of a novel non-medication treatment for OUD.
ClinicalTrials.gov with the identifier NCT04916600.
Primary and Secondary Outcome Reporting in Randomized Trials: JACC State-of-the-Art Review
2021, Journal of the American College of Cardiology
Consensus as to best practices for the selection, reporting, and interpretation of primary and secondary outcomes of randomized controlled trials is lacking. We reviewed the strategies adopted in publications of randomized controlled trials (RCTs) for the analysis, presentation, and interpretation of efficacy outcomes from a survey of all cardiovascular RCTs published in the New England Journal of Medicine, Lancet, and the Journal of the American Medical Association during 2019. We focus on the choice of primary outcomes, the variety of approaches to selecting secondary outcomes, the options sometimes used to control type I error, and the common practice to not correct for multiple testing in reporting secondary outcomes. We comment on current practice across journals in the reporting of P values and also how conclusions in trial reports frequently adhere to an undue reliance on P < 0.05 as a basis for positive claims of treatment efficacy. We conclude with recommendations for how future RCT reports could best select, report, and interpret their findings on primary and secondary outcomes.
Interpretation of clinical endpoints in trials of acute myeloid leukemia
2018, Leukemia Research
Treatment regimens for acute myeloid leukemia (AML) have remained largely unchanged until recently. Molecular advances have opened the door to targeted therapies, many of which are in late-phase clinical trials. As new therapeutic opportunities arise, it is appropriate to review key aspects of clinical trial design, statistical interpretation of outcomes, and methods of data reporting. Complete remission and overall survival (OS) are common primary endpoints in early-phase AML clinical trials. OS and event-free survival are frequent primary endpoints in phase 3 trials. Clinical trials are designed to address the primary endpoint using prespecified α and power levels. Interpretation of additional endpoints (eg, secondary endpoints and subgroup analyses) must be viewed in light of a trial’s statistical design. Furthermore, variations in reporting of endpoints must be considered in order to understand trial outcomes. Time-to-event endpoints are typically reported using Kaplan-Meier curves, which are visually informative. Statistical data derived from these curves can be complex, and a variety of factors may impact interpretation. The purpose of this review is to discuss the nuances of common AML trial endpoints and their data presentation to better inform evaluation and understanding of clinical trial data.
Systematic literature review of health-related quality of life in locally-advanced non-small cell lung cancer: Has it yet become state-of-the-art?
2017, Critical Reviews in Oncology/Hematology
Citation Excerpt :
That the effect only becomes evident when pooling the two studies, which on their own did not reach statistical significance for HRQoL parameters, suggests that a higher statistical power was needed to make HRQoL effects evident. Statistical power analyses are typically based on the primary endpoint (Neill, 1997). Therefore, the potential effect of therapy on HRQoL may possibly be underestimated.
Lung cancer and its treatment have an important impact on the patients’ health-related quality-of-life (HRQoL). A systematic literature review of prospective clinical studies published since 2005 and measuring HRQoL in patients with locally-advanced non-small cell lung cancer (LA- NSCLC) was performed. Besides reviewing the HRQoL impact of LA-NSCLC treatment, it critically examined the frequency, methodology and quality of HRQoL data collection and analysis in LA-NSCLC clinical studies. Out of 814 potentially eligible publications, only 27 (representing 19 individual studies) met the inclusion criteria. Eight studies documented an impact on HRQoL. Large variability in use of HRQoL instruments, statistical analysis and methodological quality was observed. Reporting of HRQoL data lacks standardization, but recent initiatives establishing recommendations to standardize the analysis and reporting of HRQoL in cancer trials are expected to address these issues. Overall, more research is needed to evaluate the treatment impact on HRQoL in both clinical trials and daily care.
Impact of Induction Therapy on Delayed Graft Function Following Kidney Transplantation in Mated Kidneys
2017, Transplantation Proceedings
Delayed graft function (DGF) is defined as the need for dialysis within 1 week of transplantation and occurs in 20%–50% of deceased-donor kidney transplant recipients. Although recovery from DGF often occurs within a few days, many cases may take weeks to months before the transplant function begins. The delay in function increases the complexity of recipient care, makes the diagnosis of acute rejection more difficult, prolongs length of stay, and increases hospital costs. Although several authors have proposed nomograms to predict DGF, there is no identifiable strategy to ameliorate it, except for the possible use of a specific type of induction therapy called Thymoglobulin.
In this retrospective analysis we included 407 subjects, of which 76 were mated (left and right kidney transplanted at Montefiore from the same donor). We used conditional logistic regression analysis while adjusting for the mated kidneys. We adjusted for age, gender, and race a priori, as well as cold ischemia time.
There was a 36% decrease in odds of DGF when Thymoglobulin was used as induction when compared with basiliximab in mated kidneys 0.64 (0.10–4.05) (odds ratio [OR] with 95% confidence interval [CI]).
Thymoglobulin did have a protective effect in these data when analyzed in mated kidneys, however, we need a larger amount of data to concretely conclude this effect.

View all citing articles on Scopus

View full text

Secondary endpoints cannot be validly analyzed if the primary endpoint does not demonstrate clear statistical significance

Abstract

Note for guidance: biostatistical methodology in clinical trials in applications for marketing authorization for medical products

Stat Med

Testing the hypothesis that matters for multiple primary endpoints

Drug Info J