Abstract
Forced spirometry is pivotal for diagnosis and management of respiratory diseases, but its use in primary care is suboptimal. The aim of the present study was to assess a web-based application aiming at fostering high-quality spirometry in primary care.
This was a randomised controlled trial with 12 intervention primary care units (PCi) and six control units (PCc) studied over 12 months. All 34 naïve nurses (PCi and PCc) received identical training. The PCi units had access to educational material and remote expert support. Quality of spirometry and usability of the web application were assessed.
We included 4,581 patients (3,383 PCi and 1,198 PCc). At baseline, quality was similar (PCi 71% and PCc 67% high-quality tests). During the study, PCi showed higher percentage (71.5%) of high-quality tests than PCc (59.5%) (p<0.0001). PCi had 73% more chance of high-quality performance than PCc. The web application was better for assessing quality of testing than the automatic feedback provided by the spirometer. Healthcare professionals’ satisfaction and usability were high.
The web-based remote support for primary care by specialists generated a sustained positive impact on quality of testing. The study expands the potential of primary care for diagnosis and management of patients with pulmonary diseases.
Forced spirometry (FS) is viewed as a first-line test for clinical assessment of patients with respiratory symptoms. Because of its high applicability and information content [1, 2], FS plays a pivotal role in the diagnosis and follow-up of chronic obstructive respiratory diseases [1–4]. It is of note that relevant clinical guidelines indicate the need for a widespread use of spirometry in primary care for early detection and appropriate management of asthma and chronic obstructive pulmonary disease (COPD). There is, however, a great deal of controversy [5–8] regarding the quality of the tests performed in primary care by nonexpert professionals, resulting in suboptimal deployment of FS. Consequently, effective training of healthcare professionals ensuring high quality of FS in primary care is crucial to generate reliable results preventing unnecessary test duplications across the healthcare system.
Quality of FS strongly depends on adherence to international recommendations [9, 10]. The American Thoracic Society (ATS)/European Respiratory Society (ERS) documents establish well-defined quality control criteria for both equipment and tests, but they do not include indications on strategies to ensure sustained quality assurance in clinical settings wherein nonexpert professionals are likely to perform the tests. Previous experiences in remote support of FS [11–14] seem to indicate both feasibility and positive outcomes, but none of them shows scalability or potential for generalisation.
The current randomised controlled study carried out in five areas of Spain throughout a 1-yr follow-up period examines efficacy, acceptability and usefulness of a web-based application [15] providing remote assistance to nonexpert professionals for both quality assurance and support to interpretation of the tests.
MATERIAL AND METHODS
The research was carried out from 2007 to 2008 in five different areas of Spain that were organised for the study purposes as independent nodes located in: Extremadura (south-western region of Spain), Basque Country (north of Spain) and three nodes in Catalonia (north-eastern area of Spain). At baseline, a survey on available resources to perform FS and the perceived need of the test in primary care was administered to the general practitioners (GPs) participating in the study [16].
Each node (fig. 1) had a reference centre (lung function laboratory) from a tertiary hospital with a specialised lung function professional playing the role of coordinator of the primary care units of the node. They were responsible for blindly scoring (from A (best score) to F (worse score)) (table 1) all spirometric tests performed in the area for both intervention and control groups [17].
Structure of each node. The 12 intervention primary care units (PCi) had a bidirectional communication with the lung function laboratory playing a role as a support centre, whereas the six control primary care units (PCc) only transferred information to the support centre without any feedback. The five nodes were Bilbao (two PCi and two PCc), Cáceres (two PCi and one PCc), Vic (three PCi and one PCc), Badalona (two PCi and one PCc) and Barcelona (three PCi and one PCc).
In each primary care unit, patients eligible for the study were selected among those with respiratory symptoms that required testing based on the GP’s criteria, without any restriction related to age, sex or clinical status. Forced spirometry was carried out by naïve nurses. No information on clinical status of the patients was used for the purposes of the study.
The 18 primary care centres included in the trial were randomly allocated, within each node, either to one of 12 intervention primary care centres (PCi) or six control primary care centres (PCc). Up to 34 nurses, five coordinators, three telecommunication engineers and ∼150 GPs participated in the study.
The research was approved by the Ethical Committee of the Hospital Clínic i Provincial de Barcelona (Barcelona, Spain) and the corresponding ethical committee of each participating node. Patient consent was not required for the present study.
The study protocol included a 2-day training course for all nurses of the two groups (PCi and PCc) using a methodology close to that applied by the National Institute for Occupational Safety and Health (Centers for Disease Control, Atlanta, GA, USA) [18]. The training course was carried out in each node at the beginning of the study. At the end of the training, all participants had performed several FS manoeuvres and had participated in discussions on standardisation of FS [10].
The nurses of the intervention group (PCi) were instructed in the management of the website and were able to access it during the whole study period. The educational content was specifically designed to empower the healthcare professionals to perform high-quality testing. It included a description of the spirometers used in the study, international recommendations on FS and educational videos.
The application provided a forum facilitating communication among healthcare professionals (GPs and nurses) and with the node’s coordinator. The nurses were able to generate specific questions to the coordinator related to quality or interpretation of the test and they received regular individualised feedback from the coordinator regarding the quality of the spirometries loaded into the system.
The coordinator of each node was also responsible for the evaluation of each test loaded into the system, following the classification described in table 1, and generated, on a weekly basis, a report addressed to each PCi nurse including information on several aspects of quality control of the tests analysed, namely: repeatability of the manoeuvres, characteristics of the curves, and checks for starting (back-extrapolation) and ending (expiratory time) of those manoeuvres accepted by the primary care professionals. The quality assessment was based on visual analysis of both flow–volume and volume–time manoeuvres. All node coordinators were instructed to follow strictly identical criteria for grading the tests throughout the study period. There was a general supervision of the node coordinators’ tasks performed by F. Burgos, assessing for homogeneity of the coordinators grading criteria.
In contrast, the healthcare professionals included in the control group did not have access to the web application. The 2-day face-to-face training course was the only support provided to them throughout the entire study period.
Technical setting
We used two types of spirometric systems conforming to the ATS/ERS recommendations [9, 10]. In all cases, the system was connected to a personal computer. In two out of the five nodes, we used a disposable and pre-calibrated pneumotachograph-based spirometer (Datospir 110; Sibelmed, Barcelona, Spain), whereas in the three remaining nodes, an ultrasound transit time-based spirometer (EasyOne; ndd Medical Technologies, Barcelona, Spain) was used. The FS equipment had the original software without any modification except for the potential to export the data from all tests in XML (Extensible Markup Language) format. Briefly, each node used the same type of spirometer independently of being an intervention or control centre. In all cases, the nurses were instructed to use the automatic quality messages generated by the equipment. Although the two systems had a build-in capacity to generate automatic messages, only those of the EasyOne spirometer were explicitly for the users. Consequently, the comparison between remote reviewer and automatic feedback was only reported for those nodes using EasyOne spirometer.
The application tested in the current study is one of the modules of the information and communication technology platform used to support management of chronic patients [15, 19]. Such a platform provided traceability of all the actions taken during the follow-up period. A VeriSign™ Trust Node (Symantec™, Sunnyvale, CA, USA) security system was used to ensure confidentiality of encrypted data shared through the Internet. After the end of the follow-up, we assessed acceptability of the web-based quality control programme by the GPs involved in the study. Usability of the web application was also assessed (Software Usability Measurement Inventory (SUMI); University College Cork, Cork, Ireland) [20] by the nurses that performed the tests.
Data analysis
Characteristics of the sample are presented as n (%) for categorical variables or mean±sd for continuous variables (since all of them followed normal distributions). Comparisons of sociodemographic and lung function variables between the intervention and control group were made using Chi-squared or ANOVA tests, as appropriate. Effects of the intervention in the quality of the spirometry were tested by comparing the percentage of quality grade A and B spirometries between PCi and PCc, both at each month and during the whole study period, using the Chi-squared test. Additionally, multivariate logistic regression analyses were built with quality of spirometry as the outcome and intervention as the main exposure, adjusting for differences between PCi and PCc subjects. Data analysis was conducted using Stata 10.1 (StataCorp, College Station, TX, USA). A p-value <0.05 was considered statistically significant.
RESULTS
Study groups
We examined 4,581 subjects whose main anthropometric characteristics, age and lung function results are displayed in table 2. Each subject had been scheduled only once for a visit to a primary care clinic and FS was performed following the criteria of the GP.
We observed that subjects in the intervention group were slightly older and moderately taller than those in the control group. Mean forced expiratory volume in 1 s % predicted was moderately abnormal with no differences between groups. In contrast, forced vital capacity was within the reference interval, but slightly lower in controls than in the intervention group.
The main results of the self-administered baseline questionnaire [16] to assess the status of FS in primary care are displayed in table 3. It was answered by 146 (99%) GPs from the 18 primary care centres participating in the study.
Effects of the intervention
After the first quarter, monthly percentages of high-quality FS manoeuvres were significantly and consistently higher in the intervention than in the control group (fig. 2). The intervention group presented an average of 71.5% high-quality spirometries throughout the whole study period, with no differences between months 1 and 12. In contrast, the control group showed a lower mean percentage (59.5%) (p<0.001) of high-quality tests during the whole study period with a statistically significant fall between month 1 (67%) and month 12 (62%) (p=0.011). Throughout the study, the difference in percentage of high-quality tests between intervention and control groups increased from 4%-points at month 1 up to 16%-points at month 12 (p<0.05). No differences between groups were seen in score C (table 1). However, while the proportion of tests within the lowest score (F) increased from 9.3 to 16.2% in the control group, we observed a decrease in the intervention group, from 15.2 to 5.2%. The results of the logistic regression analysis indicated that the effects of the intervention remained after adjusting for baseline differences (age, lung function and sex), such that tests in the intervention group had a 73% higher chance of high-quality performance than those of the control group. We noticed that PCi professionals performed a higher number of spirometric manoeuvres than those of the PCc group. Up to 3% of intervention subjects made eight manoeuvres whereas the maximum amount of manoeuvres in the control group was six.
Percentage of high-quality tests, i.e. scores A and B (three acceptable manoeuvres and best of two with differences in forced vital capacity (FVC) and/or forced expiratory volume in 1 s (FEV1) <150 mL, and three acceptable manoeuvres and best of two with differences in FVC and/or FEV1 <200 mL, respectively), in the intervention and control groups throughout the study period. *: p<0.05; ***: p<0.001.
Automatic assessment of quality
In the subset of primary care centres using EasyOne, we compared quality scores automatically generated by the system with those provided remotely by experienced professionals.
Automatic quality assessment presented a pattern indicating statistically significant effects of the intervention similar to those indicated in figure 1, but the absolute figures of spirometric manoeuvres identified as acceptable tests were consistently lower than those seen with remote assessment by experienced professionals, as indicated below. At the beginning of the study, automatic quality assessment did not show differences in percentages of acceptable manoeuvres between intervention and controls, whereas at the end of the follow-up the amount of high-quality spirometries in the intervention group (55%) was higher than in the control group 43% (p=0.035) with an average difference of 13.5%-points. The equivalent figures using the same equipment but with the remote professional assessment were 71.5% (intervention group) and 59.5% (control group) (p<0.0001) with a similar mean difference of 12%-points between intervention and control. Accordingly, underestimation of acceptable spirometric manoeuvres generated by automatic quality assessment as compared with assessment carried out by expert professionals showed an average of -16%-points.
Acceptability of the web application
The results of the survey carried out among the GPs (n=126, 86% response rate) 1 month after the end of the study indicated an acceptable level of overall appreciation of the web functionalities (97% were satisfied with a score of 7.3±2, from 0 to 10) together with a rather low percentage (26%) of GPs indicating problems of implementation of the intervention. Overall, the GPs reported that the web application provided added value both enhancing quality of the tests and providing support for interpretation.
Finally, the usability of the web application was examined by administering the SUMI [20] questionnaire to the 34 nurses that carried out the tests with an 87% response rate. Figure 3 presents the results obtained for the five dimensions assessed in the questionnaire (efficiency, effect, helpfulness, control and learnability) as well as the score of overall satisfaction. Notice that except for the control, all the scores were >50, representing an acceptable degree of usability. As expected, the control (fig. 3) was uniformly <50, consistent with the fact that the tested software, by study design, did not allow choices to the users.
Median scores of the different dimensions of the Software Usability Measurement Inventory questionnaire [20] to assess usability of the web application (see text for further explanation). Whiskers represent 95% confidence intervals.
DISCUSSION
The principal aims of the current research were to examine efficacy, acceptability and usability of a web-based application covering three main functionalities: 1) accessibility to educational material for continuous professional development; 2) remote support for quality assurance of tests performed by nonexperts; and 3) remote assistance for lung function interpretation. We acknowledge that previous reports [11–14] have indicated the potential of telemedicine to enhance quality of both testing and diagnosis of FS carried out by nonexpert healthcare professionals, but none of the studies show potential for generalisation across the healthcare system due to technological and/or logistical factors precluding their scalability.
Our research clearly indicates a sustained beneficial impact of the intervention increasing high-quality tests (scores A and B) by ∼20% (fig. 2) and decreasing the percentage of very low-quality spirometries (score F) during the follow-up period. Also, the professionals acknowledged the usefulness of the web application as a tool for remote assistance on interpretation of the tests and to empower nonexpert professionals increasing their skills to perform high-quality FS in primary care. It is of note, however, that the impact of the application on diagnosis was beyond the scope of the current research.
Does the intervention fulfil unmet needs in primary care?
The baseline survey carried out with the participating GPs indicated that the professionals acknowledged the need for support on training and interpretation of the tests in order to achieve the full potential of FS when used in primary care. Moreover, international clinical guidelines endorse extensive use of high-quality FS in primary care. Unfortunately, despite enhanced awareness of the problem over the last few years, COPD is still associated with marked underdiagnosis, without a significant decrease during the last decade (from 78% to 73% between 1997 and 2007) [21]. Still too often, a diagnosis of COPD is made after an episode of severe exacerbation or during the first hospital admission.
Our data confirm that accessibility of appropriate support facilitating quality assurance of the tests performed at the primary care level or in the patient’s home is needed. It is classically accepted that ∼10% of patients’ data may need to be disregarded in lung function laboratories because of technical inadequacies. This percentage can be as high as 40% in epidemiological surveys without a proper quality assurance strategy [4]. We must acknowledge, however, that these figures show a marked decline when efforts to ensure quality control are adequately implemented [4, 22].
It is well accepted that training constitutes a pivotal element to achieve high-quality FS when performed by nonexperts. Recent data on a centralised quality control programme carried out as part of the PLATINO (Proyecto Latinoamericano de Investigación en Obstrucción Pulmonar) study [23] fully endorse the statement. In the primary care setting, Walters et al. [24] recently showed that the percentage of high-quality FS tests with trained nurses was ∼76% whereas that percentage dropped to 44% in untrained professionals. Different authors [6, 25–27] have elaborated on the need to transfer well-established quality assurance programmes from lung function laboratories to the primary care setting to ensure quality of the tests. There is evidence [28, 29] suggesting that external quality assurance to primary care needs to be implemented. In an extensive review of FS performed in primary care, it was found that general practitioners identified ∼90% of their own tests as acceptable; whereas the opinion of an expert decreased the acceptance rate to 64%. Moreover, a recent report [30] indicates that conventional training does not ensure sustainability of high-quality testing. Interestingly, our research found that the effects of the intervention were also seen by automatic assessment of quality. But such a modality of assessment generated a marked underestimation (-16%-points) of acceptable spirometric manoeuvres as compared with assessment by experienced professionals.
To our knowledge, the current study constitutes the first attempt to successfully implement a web-based standard training programme reinforced by telecollaboration tools allowing remote assistance of primary care professionals by specialists. In this regard, the intervention was conceived to provide long-term sustainability of the training programme through continuous empowerment of primary care professionals. The results generated by the current research endorse this vision and they suggest that the current approach meets the requirements for an extensive adoption of FS in primary care.
Limitations of the study
The quality assessment was based on visual examination of the curves which, in some cases, may limit accurate identification of the end of the test. We acknowledge that implementation of an automatic algorithm should be considered as a useful decision support tool for the node coordinator. As indicated above, the study is not addressing the impact of remote assistance on diagnosis with FS. Moreover, we did not aim to perform a detailed analysis of factors modulating extensive deployment and adoption of the intervention. The latter would have required a specific design including several types of chronic patients covering a broad spectrum of disease severity.
Conclusions
The current study shows that telecollaboration between primary care professionals and lung function specialists has a positive impact on quality assurance of FS performed by nonexperts. We would like to emphasise that the intervention assessed in the current study seems to show high potential for generalisation across the healthcare system, such that future studies aiming to examine adoption of the proposed strategy should be encouraged.
Acknowledgments
The e-Spir@p group investigators are as follows: N. Roger (Hospital General de Vic, Vic, Spain); M. Rovira (CAP, Torello, Spain); J. Oliva (CAP, Santa Eugenia, Spain); C. Esteve (CAP, Santa Eugenia, Spain); F. Comas (CAP, Sant Hipolit, Spain); M.J. Bravo (CAP, Sant Hipolit, Spain); D. Lafont (CAP, Tona, Spain); A. Baucells (Universitat de Vic, Vic, Spain); C. Disdier (Hospital Universitario de Valladolid, Vallaloid, Spain); J.M. Gutierrez† (Centro de Salud Manuel Encinas, Cáceres, Spain); C. Jimenez (Centro de Salud Manuel Encinas, Cáceres, Spain); R. Montero (Centro de Salud Manuel Encinas, Cáceres, Spain); A. Garrido (Centro de Salud San Jorge, Cáceres, Spain); M. Reviriego (Centro de Salud San Jorge, Cáceres, Spain); A. Barroso (Centro de Salud Plaza de Argel, Cáceres, Spain); J. Magariño (Centro de Salud Plaza de Argel, Cáceres, Spain); C. Rubio (Centro de Salud Plaza de Argel, Cáceres, Spain); M.L. Rivera (Hospital Tries i Pujol, Badalona, Spain); E. Monsó (Hospital Tries i Pujol, Badalona, Spain); R. Hervàs (Hospital Tries i Pujol, Badalona, Spain); L. Palacios (Gorg, Sant Adriá de Besos, Spain); M. Martínez (Gorg, Sant Adriá de Besos, Spain); G. Tintoré (Gorg, Sant Adriá de Besos, Spain); Y. Chicon (Barraquer, Sant Adriá de Besos, Spain); A.M. Andres (Barraquer, Sant Adriá de Besos, Spain); M.L. Garcia (San Roc, Sant Adriá de Besos, Spain); S. Maneus (San Roc, Sant Adriá de Besos, Spain); E. López-Santamaria (Hospital de Cruces, Bilbao, Spain); B. Galdiz (Hospital de Cruces, Bilbao, Spain); A. Telleria (Centro de Salud Las Arenas, Las Arenas, Spain); A. del Val (Centro de Salud Las Arenas, Las Arenas, Spain); I. Llanas (Centro de Salud Las Arenas, Las Arenas, Spain); M.J. Lana (Centro de Salud Sodupe, Sodupe, Spain); T. Vazquez (Centro de Salud Sodupe, Sodupe, Spain); J.A. Pardo (Centro de Salud La Paz, La Paz, Spain); B. Roldan (Centro de Salud Desierto, Barakaldo, Spain); E. Durán (PAMEM Viladomat, Barcelona, Spain); C. Costi (PAMEM Viladomat, Barcelona, Spain); M. Prados (PAMEM Viladomat, Barcelona, Spain); L. Ricart (PAMEM Viladomat, Barcelona, Spain); J. Moreno (PAMEN Barceloneta, Barcelona, Spain); J. Esteban (PAMEN Barceloneta, Barcelona, Spain); R. Gali (PAMEN Villa Olimpica, Barcelona, Spain); E. Font (PAMEN Villa Olimpica, Barcelona, Spain); M. Gamiz (PAMEN Larrad, Barcelona, Spain); S. Roig (PAMEN Larrad, Barcelona, Spain); C. Alonso (PAMEN Larrad, Barcelona, Spain); M. Serrate (PAMEN Larrad, Barcelona, Spain); F. Burgos (Hospital Clinic, Barcelona, Spain); J. Roca (Hospital Clinic, Barcelona, Spain); A. Alonso (Hospital Clinic, Barcelona, Spain); B. Vallespin (Linkcare, Barcelona, Spain); D. Fonollosa (Linkcare, Barcelona, Spain); and W. Lalinde (Linkcare, Barcelona, Spain).
The authors thank I. Serra (CREAL, IMIM Barcelona, Barcelona, Spain) for the statistical and data analysis, C. Gistau, M. Simó, C. Argaña, J.L. Valera (all Hospital Clínic, Barcelona, Spain) and all the respiratory technologists that participated in the study for their cooperation.
Footnotes
Support Statement
The study was supported by CIBERES, FIS grants 04/2728 and 04/2749, MAPFRE 2006, PII-EPOC (SEPAR), NEXES (Supporting Healthier and Independent Living for Chronic Patients and Elderly; grant CIP-ICTPSP- 2007-225025) and an unrestricted grant from Boheringer-Pfizer. J. Garcia-Aymerich has a researcher contract from the Instituto de Salud Carlos III (grant CP05/00118), Ministry of Health, Spain.
Statement of Interest
Statements of interest for F. Burgos, C. Disdier and J. Roca, and for the study itself can be found at www.erj.ersjournals.com/site/misc/statements.xhtml
- Received October 27, 2010.
- Accepted September 15, 2011.
- ©ERS 2012
REFERENCES
