Article Text
PDF
Abstract
Background The clinical need to determine the presence of epidermal growth factor receptor (EGFR) gene mutations in non-small-cell lung cancers (NSCLC) in order to make informed decisions for patient treatment has seen the widespread introduction of EGFR molecular testing in many laboratories. To ensure high-quality molecular testing and allow laboratories to externally measure the standard of the service, an external quality assessment (EQA) scheme was provided to assess the whole testing process.
Methods Formalin-fixed paraffin-embedded NSCLC tumour sections were distributed to laboratories for routine EGFR molecular testing, and the genotyping accuracy, interpretation of the result and clerical accuracy of the report were independently assessed.
Results Three rounds of assessment have identified many genotyping errors and have highlighted the need for external assessment and education in many testing laboratories. The main issues raised were the importance of accurate genotyping, including the use of common mutation nomenclature, clear unambiguous interpretation of the result, the impact of tumour sample assessment regarding amount of tumour being analysed and the heterogeneity of the sample on the molecular test result.
Conclusions Improvements in all these areas were observed during the progression of the three EQA rounds, however, continuous unacceptably high genotyping error rates demonstrate the clear need for continual external assessment and education in this field.
- Eqa
- Lung Cancer
- Molecular Pathology
- Quality Assurance
- Egfr
Statistics from Altmetric.com
Introduction
Members of the receptor tyrosine kinases (RTK) family of genes are commonly altered in cancer through somatic mutation, and it has been widely demonstrated that these alterations are key events in tumour formation and progression.1 In non-small-cell lung cancer (NSCLC), specific mutations in the epidermal growth factor receptor (EGFR) gene produce RTK-variants capable of ligand-independent signalling. Tumour cells possessing these activating mutations in their EGFR gene are driven by constitutively activated signalling pathways, which confer several of the hallmarks of cancer.2 Targeting of RTKs in NSCLC by the use of small molecule tyrosine kinase inhibitors (TKIs), such as gefitinib (IRESSA), was first reported in the late 1990s with only very limited success (reviewed in3), and it required the analysis of clinical trial data, and in particular, those produced by the IPASS trial (IRESSA Pan-Asia Study) to bring to light the correlation between certain activating EGFR mutations and response to TKIs.4–7 With this, the model of targeted therapies provides another step towards becoming a major force in the fight to improve cancer patient outcomes.8
Laboratories have always had a major role in the diagnostic process, and in turn, in guiding clinical treatment decisions, but now, more than ever before, the results generated in the laboratory are becoming crucial for successful patient treatment. It is clear that the role of the laboratory in correctly detecting genetic alterations in tumours which predict sensitivity to targeted therapies has a central role, and is rapidly becoming more widespread as more targets are identified in an increasingly wide variety of tumour types.9
The task of molecular testing in cancer has largely devolved into two types of laboratories; those based in regional genetic centres and those in cellular pathology departments. Both have faced challenges, albeit subtly different ones, in delivering an effective service. Scientists working in genetics have gained considerable experience and expertise in molecular-based testing and its modalities, but are not extensively trained in oncology. While those working in cellular pathology have great expertise in cancer diagnostics, they have little or no prior experience in molecular genetics and/or molecular techniques. Successful delivery has, therefore, often required cross-discipline networking.
In this context, external quality assessment (EQA) as an aid to measuring laboratory performance is a key factor.10–13 An EQA scheme for the mutation analysis of the EGFR gene in NSCLC was introduced in pilot form in 2010 by the UK National External Quality Assurance Services (UK NEQAS) as a collaboration between UK NEQAS for Molecular Genetics and UK NEQAS for immunocytochemistry and in situ hybridisation (ICC and ISH), this reflected the aforementioned distribution of testing provision in the health-science community. The EQA scheme format was based on that devised for the UK NEQAS schemes for mutation testing of the KRAS gene in colorectal cancer, and the KIT gene in gastrointestinal stromal tumours,12 ,13 and was open to UK and non-UK-based laboratories. The standard of the reports submitted for the 2010 EGFR pilot EQA indicated a need for education and close monitoring of EGFR molecular testing and, therefore, two further rounds of EQA were provided in 2011.
Methods
Scheme organisation
The EQA was administered using the UK NEQAS for Molecular Genetics website, whereby each participating laboratory registered for the EQAs, submitted EQA returns, accessed their laboratory scores and the scheme publications via their own website account. This also provided each laboratory with a record of participation and level of performance as well as feedback from EQA assessors. Participation was open to all interested parties. Each participant was assigned a unique laboratory identity number in order to maintain impartiality of assessment. Only UK NEQAS scheme staff had access to laboratory codes.
The UK NEQAS Molecular Pathology EQA Working Group was responsible for the delivery of the EQA service and was advised by the UK NEQAS Steering Committees for Molecular Genetics and Cellular Pathology Techniques to ensure both the molecular genetics and histopathology aspects of the testing were addressed and the needs met.
Scheme format
In order to provide a true measurement of the quality of service a laboratory is providing, it is important for the complete process to be assessed. Therefore, NSCLC tumour samples were distributed to participants as formalin-fixed paraffin-embedded (FFPE) tissue sections. For each EQA round, three clinical case scenarios with corresponding NSCLC tumour samples were distributed to each participating laboratory. Participants requested their sample preference; either 2×5 µm rolled FFPE tumour sections, 2×5 µm rolled FFPE tumour sections plus one slide-mounted FFPE tumour section, or 3×5 µm slide-mounted tumour sections. All requests were met.
Participants were required to follow routine testing protocols (4 weeks was allocated for testing), then submit a full interpretative report which was assessed for genotyping accuracy, interpretation of the result and clerical accuracy of the report.
All returns were scored independently by 3–4 assessors against peer-ratified marking criteria. The assessment teams comprised of senior clinical scientists/biomedical scientists/consultant clinicians from pathology, histopathology and molecular genetics disciplines who had extensive experience in EGFR mutation testing in NSCLC. Each category (genotyping, interpretation and clerical accuracy) was scored from a total of 2.00 marks. Deductions were made for the reporting of incorrect genotypes and clerical errors (such as incorrect patient name/date of birth). The EGFR molecular test returns were also assessed for the interpretative content of each report. The team of assessors agreed on a consensus, marking criteria to be followed after each EQA round had closed, and all participant reports had been reviewed. The criteria outlined the key interpretative comments which should be present in each report. These comments were tailored to the EGFR genotype, percentage of tumour content present and the clinical case scenarios supplied. Marking was lenient with respect to actual wording, and if the correct interpretation was portrayed then full marks were allocated.
Participants received individual score reports with feedback from the assessors, and a scheme report was published for each EQA round summarising the data gathered and detailing issues arising from that run.
Validation
The EQA material was subjected to independent validation by two laboratories prior to distribution. In order to monitor the heterogeneity of the EQA material, the first and last sets of tumour sections prepared from each tumour block were validated. Furthermore, the validating laboratories also participated in the EQA and were sent sections from the centre of the tumour blocks. Extra samples were included in the validation to ensure that the validating laboratories did not know the expected EQA results. Table 1 summarises the genotypes and the estimated tumour burden for the EQA samples distributed.
Summary of the EQA material; genotypes and estimated tumour burden
Validating Laboratory 1 (All Wales Molecular Genetics Laboratory, Cardiff) tested for changes in the EGFR gene TK domain by gene fragment length analysis and pyrosequencing. Validating Laboratory 2 (Department of Cellular Pathology, Queen Elizabeth Hospital Birmingham) tested exons 18, 19, 20, 21 of EGFR gene known to contain clinically relevant mutations by real-time PCR using the Qiagen (DxS) kit (ARMS-Scorpions).
Results
There was an increase in the number of participants from 2010 (27 laboratories), 2011 run 1 (47 laboratories) to 2011 run 2 (49 laboratories).
The proportion of molecular genetic laboratories to histopathology-based laboratories in the UK is displayed in figure 1. The most recent EQA runs (ie, 2011) show an increase in the number of histopathology participants resulting in an equal spilt between the types of laboratories providing EGFR molecular services. Laboratory discipline was not collected for non-UK participants, therefore, only UK laboratory data is displayed.
Summary of the participant laboratory type (UK only).
In both EQA years, the most requested sample type was rolled FFPE tumour sections along with one slide-mounted section. For the 2011 EQA runs, there was a reduction in the number of participants requesting rolled sections only, and the amount of requests for slide-mounted samples increased (figure 2). The breakdown of sample requests by UK laboratory type for the 2011 run 1 EQA is displayed in figure 3. The majority of genetic-based laboratories requested rolled tumour sections. The majority of the histopathology-based participants requested a slide-mounted tumour section to be supplied along with the rolled sections.
Graph displaying the sample requirements of all participants for the 2010 and 2011 run 1 external quality assessment distributions.
Graph displaying the participant laboratory sample requirements for the 2011 run 1 external quality assessment (UK only).
A range of genotypes was distributed for each EQA run including: exon 19 deletion, point mutations and samples with no mutation detected in the validated regions of the EGFR gene (table 2). Twenty-four percent of laboratories (6/25 laboratories submitting EQA returns) reported at least one genotyping error in the 2010 EQA run (table 2). The errors reported were: (1) reporting the presence of a second mutation c.2576C >A, p.(Ala859Thr) in case 2010-1 by Sanger sequencing, as well as correctly identifying the deletion mutation; (2) false positive result reported (deletion in exon 19 detected in case 2010-2) using the Qiagen (DxS) kit (ARMS-Scorpions), (unfortunately, these two laboratories failed to provide any further information regarding the cause of the errors); (3) incorrect detection of c.2573T >G, p.(Leu858Arg), and failure to detect the exon 19 using the Qiagen (DxS) kit (ARMS-Scorpions) in case 2010-1 (the version of the kit used did not include the c.2369C >T, p.(Thr790Met) mutation and, therefore, the data was shifted and the results were misinterpreted. The laboratory has updated the kit used; (4) false positive result reported, c.2156G >C, p.(Gly719X) detected using the Qiagen (DxS) kit (ARMS-Scorpions) in case 2010-2 (the laboratory stated that this mutation was at a low level, and as they have not been able to reproduce the result, they suspect it was a contamination issue); (5) incorrect mutation detected in case 2010-3 due to a sample swap in the laboratory and (6) failure to detect the c.2582A >T p.(Leu861Gln) mutation in case 2010-3 caused by a transcription error when reporting the results.
Summary of the error rate in the three rounds of EQA
The 2011 EQA runs detected a decrease in the number of laboratories reporting genotyping errors (table 2). The first 2011 EQA run had a genotyping error rate of 6.7% (3/45 laboratories submitting EQA returns) with three laboratories each reporting one genotyping error: (1) one failed to detect the deletion in exon 19 by fragment length analysis in case 2011-2. This laboratory has not provided any further information to the scheme as to the cause of the error, however, 10 other laboratories, plus the validating laboratories, detected the deletion by fragment length analysis; (2) one failed to detect the point mutation c.2573T >G, p.(Leu858Arg) in case 2011-3 by Sanger sequencing (the mutation was detected at a low level in one sequencing read, but the result was not repeatable, therefore, the presence of the mutation was not reported. This may be due to a low level of mutation being present in the tumour sections being tested and, therefore, at the limit of detection by direct sequencing), and (3) the third reported the presence of the correct point mutation in case 2011-3, but also detected a second mutation (deletion in exon 19) (this laboratory did not state the methodology used in the report, but since confirmed that they were using the Qiagen (DxS) kit (ARMS-Scorpions), and also that they had been experiencing the presence of low-level contamination. The testing had been repeated without detecting the deletion).
The second 2011 run had a genotyping error rate of 6.4% (3/47 laboratories submitting EQA returns). There were three laboratories, each reporting one or more critical genotyping errors all of which were due to a transcription error in the reporting process: (1) failed to detect the exon 18 mutation in case 2011-6 (when the data was reviewed, the mutation was present, but this had not been reported when initially tested); (2) incorrect mutation reported for case 2011-6, c.2303G >T, p.(Ser768Ile) reported instead of c.2156G >C, p.(Gly719Ala) (the correct mutation was detected by the test but the results were incorrectly reported, as the report was written using out-of-date documentation) and (3) switching of test results and, therefore, incorrect reporting of cases 2011-4 and 2011-5 (again the results were switched when reporting the test results).
There was a range in methodology used by participating laboratories (figure 4), and many used a combination of two or more. The 2011 EQA runs demonstrated that the most commonly performed method at that time was ARMS-based kits with many laboratories also performing direct Sanger sequencing.
Graph displaying the different methodologies used by participating laboratories in 2011 runs 1 and 2.
There was a wide variation in the degree of interpretation provided on the EQA reports. Some reports did not give any interpretation of the results and stated the genotype only. Many others did link the result obtained back to the clinical case scenario provided, and gave an indication of the patient's predicted response to TKIs. Many participants omitted to state the method of testing, the sensitivity of the method used, provide an indication of the mutations screened, use Human Genome Variation Society (HGVS) mutation nomenclature when reporting a mutation (http://www.hgvs.org/mutnomen/), give a relevant reference sequence when reporting a mutation, and whether or not the tumour content of the sample had been assessed. These omissions resulted in marks being deducted in the interpretation category. Due to the high number of genotyping errors in the 2010 EQA, the interpretation aspect of the EQA was marked very leniently. As a result, the 2010 scheme had a mean interpretation score of 1.75 (out of 2.00). However, as the genotyping had improved with the 2011 EQA the interpretation was then marked more harshly in order to promote good laboratory reporting. The first EQA run in 2011 resulted in a mean interpretation score of 1.44 (out of 2.00), and using the same marking criteria a significant improvement was observed in 2011 run 2 (1.68).
To mimic the variability in the amount of tumour often present in samples received for EGFR molecular testing, the amount of tumour present in each set of samples used ranged from 30% to 80%. Many of those participants receiving slide-mounted sections provided the estimated tumour content of each sample on their report. Figure 5 shows the variation in the estimates reported.
Graphs displaying the range of estimated tumour content reported by participants for the three external quality assessment (EQA) samples distributed for 2011 run 2. The names provided are the EQA mock names used for the scheme and are not related to any known patient. The solid line(s) indicate the estimated tumour content obtained during sample validation prior to distribution.
Discussion
The provision of a pilot external quality assurance scheme by UK NEQAS for EGFR testing in NSCLC has demonstrated marked improvement in the quality of returns received over a relatively short period. In the first round (2010), the high number of genotyping errors by participating UK and non-UK laboratories was of significant concern; all errors were followed-up and investigated with the participating laboratories. There was no correlation between genotyping errors and any particular methodology used. Many were false positive results, and would have resulted in a patient being incorrectly recommended for treatment with an EGFR-TKI. Clinical trials data has demonstrated that EGFR wild-type patients do worse on EGFR-TKIs than with doublet chemotherapy. Follow-up with participating laboratories showed that false positive results were due to both analytical and data transcription errors. To avoid false positives, it is crucial that adequate internal quality control measures are employed, including appropriate sample and data transfer checks by competent staff. Analytical false positives are avoided by the careful internal control of molecular assays, and clear understanding of the technologies used and their sensitivity and risks. It is clearly unacceptable for laboratories providing this service to be making any errors that could directly lead to inappropriate patient treatment, and therefore, the UK NEQAS scheme for EGFR will impose further rounds of EQA and intervene with poorly performing UK laboratories making genotyping errors in the future.
During the three rounds of EQA, a marked improvement in the quality of reporting was also demonstrated. A wide variation in reporting style and interpretation was experienced in the 2010 scheme leading to leniency in the marking of the Interpretation section. The standardisation and clarity of clinical reporting of molecular results is essential for the many health professionals who are likely to encounter these reports, and need to act upon them. Combinations of EGFR mutations will be detected, and EGFR gene variants of uncertain significance will be encountered. The number of biomarkers (molecular or otherwise) assayed in NSCLC will increase. As the complexity of clinical investigations increases, the need for standardisation will become more important. The UK NEQAS Molecular Pathology EQA Working Group agreed on the following recommendations for the interpretation and reporting of EGFR results: mutations should be reported using HGVS nomenclature (http://www.hgvs.org/mutnomen/) and be expressed at both the nucleotide and protein levels. As EGFR-activating mutations can confer sensitivity or resistance to EGFR-TKIs, the type of mutation (where there is evidence) should be stated. The analytical methodology should be referenced, to include the mutations or regions of the gene which have been analysed, and the sensitivity of analysis, as this is particularly important for wild-type samples.
The heterogeneous nature of tumour samples is well documented14 and challenges the delivery of molecular diagnostic services. If a patient sample has a particularly low tumour load (or low-percentage tumour nuclei) it risks a mutation being missed should it be present—a false negative. In the context of EGFR analysis and patient treatment, the outcome would be that a potentially EGFR mutation-positive patient would be recommended for doublet chemotherapy and miss the opportunity of beneficial TKI-therapy. The risk of false negatives may be reduced by sample assessment by a histopathologist to select samples with sufficient tumour content for analysis, and designate regions of tumour samples for macro- or microdissection prior to DNA extraction, or the use of ultrasensitive molecular techniques that can detect mutations at levels of approximately 1%. The tumour load of the extracted DNA sample should always be considered alongside the sensitivity of the molecular assay employed. Samples were selected for the EQA runs which should not present problems due to low tumour percentage. Despite this, the returned estimates of tumour content for 2011 run 2 (figure 5) show a considerable range for the same distributed sample, albeit different sections from the same block. This may also reflect differences in practice for estimation by histopathologists, where in some cases, the volume of the tumour cellularity versus the background would be assessed instead of the proportion of tumour nuclei in respect to normal cells. This is a key concept for accurate molecular diagnostics of tissue samples, since the analyses are performed in DNA extracted from both tumour cells and surrounding lymphocytes, stromal and epithelial cells, all of which have different volumes yet practically identical amount of DNA to the neoplastic cells. For the analysis and reporting of EGFR tests, it was recommended that the sample should undergo histopathological assessment for tumour load, and that this should be included in the clinical report. This is particularly important information for wild-type samples to confirm that sufficient tumour material had been available for analysis. It was therefore encouraging to observe that the proportion of laboratories requesting a slide for histopathological assessment increased during the three EQA rounds.
Although only a snapshot of a laboratory's annual activity, the periodic assessment of a laboratory's performance in both the analytical and clinical reporting elements of EGFR service provision is critical. Through UK NEQAS, participating laboratories are assessed three times per year, thus rapidly identifying any service problems. The UK NEQAS Molecular Pathology EQA Working Group provides a scheme report on each round of EGFR EQA, and is also available for support and advice.
Conclusions
The genotyping errors and variability in reporting format highlights the critical requirement for EQA in the molecular pathology discipline. It has been demonstrated that participation in EQA improves the standard of laboratory testing and provides laboratories with educational help and support in order to develop a better service to their service users and ultimately the patient.
Take home messages
-
The UK External Quality Assessment Service (UK NEQAS) scheme for epidermal growth factor receptor in non-small-cell lung cancer has shown a high incidence of genotyping errors which implies a direct impact on patient care.
-
There is a wide variability in the format of reporting molecular pathology testing results, and there is a need to standardise in order to ensure the correct patient receives the correct result.
-
The UK NEQAS scheme has demonstrated that participation in EQA improves the standard of testing.
Acknowledgments
We thank all the laboratories for their participation in the UK NEQAS scheme for EGFR testing in NSCLC, and the UK NEQAS for Molecular Pathology Specialist Advisory Group for their support and guidance. We thank the scheme assessors for all their work, and we also thank the Department of Cellular Pathology, Queen Elizabeth Hospital, Birmingham, and the All Wales Molecular Genetics Laboratory, University Hospital of Wales, Cardiff, for help with sample preparation and validation.
references
Footnotes
-
Contributors ZCD, CC, AD and RB had the idea for the article, planned, conducted and reported the work. ZCD, CC, RB, NB, BOS, LPL, DGC, SP, AD and PT were involved in sample sourcing, validation, distribution and assessment.
-
Funding The 2010 run of the UK NEQAS EGFR EQA scheme was supported by a grant from the UK Clinical Pathology Accreditation Trust (0161). All other EQA runs were self-funded by participant laboratories. No funding parties had any input into scheme design, assessment or access to any specific participant data.
-
Competing interests ZCD has received educational grants from Astra Zeneca, Qiagen and Roche. ZCD, RB and DGC have consulted for Astra Zeneca and Roche. AD has consulted for Roche.
-
Ethics approval The paper reports on the standard of testing and reporting by participant laboratories following the analysis of formalin-fixed tumour tissue samples for specific mutations. The tumours were derived from human patients, but were anonymised at source. The genotyping was performed as part of a quality assurance programme and not as a scientific study. External quality assurance purposes are covered within the Human Tissue Act as ‘quality control purposes’.
-
Provenance and peer review Not commissioned; externally peer reviewed.