Elsevier

The Lancet

Volume 361, Issue 9360, 8 March 2003, Pages 865-872
The Lancet

Review
Problems of reporting genetic associations with complex outcomes

https://doi.org/10.1016/S0140-6736(03)12715-8Get rights and content

Summary

Inability to replicate many results has led to increasing scepticism about the value of simple association study designs for detection of genetic variants contributing to common complex traits. Much attention has been drawn to the problems that might, in theory, bedevil this approach, including confounding from population structure, misclassification of outcome, and allelic heterogeneity. Other researchers have argued that absence of replication may indicate true heterogeneity in gene-disease associations. We suggest that the most important factors underlying inability to replicate these associations are publication bias, failure to attribute results to chance, and inadequate sample sizes, problems that are all rectifiable. Without changes to present practice, we risk wastage of scientific effort and rejection of a potentially useful research strategy.

Section snippets

Is the association study strategy futile?

Can any association study ever be expected to detect genetic determinants of a complex disease? Several researchers have argued that attempts to locate disease genes must account for the complexity of biological and social pathways to disease and the context dependency of the effect of many risk factors.16, 17 In short, the assertion here is that genes do not generally act in a simple additive manner, but through complex networks involving gene-gene and gene-environment interactions, so that

Why are reports of associations between genotype and outcome so frequently inconsistent?

In principle, it is possible to measure associations between genotype and outcome more reliably than in classic epidemiological studies, because biases that are unavoidable in epidemiological studies can be largely eliminated when studying genetic associations. The exposure (genotype) can be measured after disease onset without systematic error. Selection bias in sampling of cases or controls is less serious in genetic association studies than in studies of environmental exposures because it is

Failure to exclude chance as an explanation in some studies, and publication bias

We suggest that failure to exclude chance is the most likely explanation for difficulty in replication of reports of genetic associations with complex diseases. For most diseases of interest, hundreds of known genes are possible candidates, and in most of these genes, dozens of polymorphisms are known or can be easily identified by screening of the gene. Around the world, thousands of such polymorphisms are tested for disease associations every week. Even if none of these genotypes is

Possible approaches to exclusion of results seen by chance

Adoption of more stringent criteria for exclusion of chance as an explanation for noted associations between genotype and outcome will reduce this problem.2, 3 But how stringent should the criteria be?

Reporting subgroup analyses

Even the simplest genetic association study can be analysed in several different ways—for instance, by testing the effects of alleles, genotypes, or haplotypes. Additional analyses by subgroup—eg, age, sex, categories of phenotype—can be undertaken. Subgroup analyses are typically done when no robust main effect is seen, to guarantee publication of some apparently positive finding. Since the number of possible subgroup analyses that can be undertaken is large, significant results obtained in

Replication studies

The recommendation that large case-control banks are needed to test for associations is set against the present reality that most researchers have only small collections available to them. In an attempt to deal with the limitations imposed by these small sample sizes, replication of associations before declaration of evidence as convincing is gaining increasing acceptance.32 However, some clarity about the role of replication studies is needed. These studies do have advantages, in that biases

Keeping publication bias to a minimum

For clinical trials, publication bias is kept to a minimum by prospective registers and systematic reviews.56 For genetic association studies, a different approach is needed, because most negative results will never even reach conference proceedings, protocol papers will not usually have been published to suggest that the study is ongoing, and an effective mechanism for establishment of prospective registers of proposed analyses is not feasible. To propose that journals should publish all

Summary of data in systematic reviews

Even if publication bias can be reduced, and stringent criteria are adopted for declaration of significance, the problem will remain of how to collate this profusion of data to extract useful information. For any given genotype-disease association, considerable effort is needed to assemble all available studies.1 Some databases containing reviews of genetic variants and reported associations with various outcomes already exist: for instance OMIM (Online Mendelian Inheritance in Man)57 and

Search strategy

We searched the PubMed database for articles on the methodology of genetic association studies for complex traits or diseases and for linkage disequilibrium mapping. References to population stratification, allelic heterogeneity, allelic spectrum, and sample size were sought. Additionally, we retrieved review papers in key journals by hand-searching our files. For each of the disease-specific examples, tailored searches were done with PubMed. Articles were selected that showed key points or

References (58)

  • ClaytonD et al.

    Epidemiological methods for studying genes and environmental factors in complex diseases

    Lancet

    (2001)
  • PritchardJK et al.

    Association mapping in structured populations

    Am J Hum Genet

    (2000)
  • KeavneyB et al.

    Large-scale test of hypothesised associations between the angiotensin-converting-enzyme insertion/deletion polymorphism and myocardial infarction in about 5000 cases and 6000 controls

    Lancet

    (2000)
  • GoringHH et al.

    Large upward bias in estimation of locus-specific effects from genomewide scans

    Am J Hum Genet

    (2001)
  • IoannidisJP et al.

    Replication validity of genetic association studies

    Nat Genet

    (2001)
  • DahlmanI et al.

    Parameters for reliable results in genetic association studies in common disease

    Nat Genet

    (2002)
  • Editorial. Freely associating

    Nat Genet

    (1999)
  • CasariG et al.

    Association of the alpha-adducin locus with essential hypertension

    Hypertension

    (1995)
  • CastellanoM et al.

    Alpha-adducin gene polymorphism and cardiovascular phenotypes in a general population

    J Hypertens

    (1997)
  • KamitaniA et al.

    Human alpha-adducin gene, blood pressure, and sodium metabolism

    Hypertension

    (1998)
  • KatoN et al.

    Lack of association between the alpha-adducin locus and essential hypertension in the Japanese population

    Hypertension

    (1998)
  • CardonLR et al.

    Association study designs for complex diseases

    Nat Rev Genet

    (2001)
  • GambaroG et al.

    Association studies of genetic polymorphisms and complex disease

    Lancet

    (2000)
  • StrohmanR

    Maneuvering in the complex path from genotype to phenotype

    Science

    (2002)
  • HoltzmanNA

    Putting the search for genes in perspective

    Int J Health Serv

    (2001)
  • CollinsA et al.

    Trials of the beta model for complex inheritance

    Proc Natl Acad Sci USA

    (1996)
  • WacholderS et al.

    Counterpoint: bias from population stratification is not a major threat to the validity of conclusions from epidemiological studies of common polymorphisms and cancer

    Cancer Epidemiol Biomarkers Prev

    (2002)
  • VielandVJ

    The replication requirement

    Nat Genet

    (2001)
  • Cavalli-SforzaLL et al.

    The history and geography of human genes

    (1994)
  • Cited by (1025)

    • Open Science Practices in Psychiatric Genetics: A Primer

      2024, Biological Psychiatry Global Open Science
    • The generalizability crisis

      2022, Behavioral and Brain Sciences
    View all citing articles on Scopus
    View full text