Abstract
Patient-centred outcomes are significantly modified by long-term home noninvasive ventilation (NIV), but a short, self-administered, specific tool for routine clinical assessment is lacking. The aim of this study was to develop and validate the S3-NIV questionnaire, a short questionnaire to measure respiratory symptoms, sleep quality and NIV-related side effects.
Patients with stable disease who were under long-term home NIV were recruited from three outpatient NIV services. Questionnaire development consisted of a selection of core items for analysis, followed by item reduction, validation and test–retest reliability.
338 patients completed a 22-item questionnaire. 11 items were removed because of non-scalability (n=2), redundancy (n=8) and lack of fit (n=1). The final version of the S3-NIV questionnaire consisted of 11 items covering two dimensions: “respiratory symptoms” (Cronbach's α=0.84) and “sleep & NIV-related side effects” (Cronbach's α=0.77). Convergent validity was high between the “respiratory symptoms” subscale of the S3-NIV questionnaire and the St George's Respiratory Questionnaire (rho= −0.76, p<0.001), and between the “sleep & NIV-related side effects” subscale and the Quebec Sleep Questionnaire (rho=0.51, p<0.001). The S3-NIV questionnaire had good test–retest reliability after 4 weeks (intraclass correlation coefficient=0.72).
The S3-NIV questionnaire is a short, valid and repeatable self-completed tool for the routine clinical assessment of patients undergoing home NIV.
Abstract
The S3-NIV questionnaire provides clinicians and patients with a simple and reliable tool to assess important domains (symptoms, sleep quality and NIV-related side effects) as a complement to physiological monitoring http://ow.ly/rR8e30mKGCG
Introduction
Home noninvasive ventilation (NIV) is increasingly used as first-line, evidence-based treatment in chronic hypercapnic respiratory failure [1, 2]. Over the past decades, the transition from acute care to the home setting has been characterised by a paradigm shift in the medical and societal approach to chronic respiratory care. Such a transition was also made possible by the technological improvements in ventilators, including software that allows users to download data [3]. However, implementing home NIV services is time-consuming, labour intensive and expensive. Therefore, it is critical to document whether improvements in lung function, arterial blood gas levels, readmission rates and eventually survival also translate into evidence-based improvements in patient-centred outcomes, such as respiratory symptoms and sleep quality, with acceptable NIV-related side effects.
The correlation between patient-centred endpoints and physiological parameters used to monitor NIV is notoriously low [4, 5]. Indeed, some of the most important domains affected by chronic respiratory failure remain poorly explored, whereas complex monitoring tools such as polysomnography are recommended for the adjustment of NIV [3, 6]. Working with valid, reliable and specific instruments to assess patient-centred outcomes as a complement to NIV physiological monitoring is crucial for setting achievable treatment goals and improving communication between the patient and the physician. The Severe Respiratory Insufficiency (SRI) questionnaire and the Maugeri Respiratory Foundation Questionnaire-28 are most often used in NIV trials to assess patient-reported outcomes [7, 8]. However, their length and complex scoring algorithms limit their use in everyday clinical practice. In addition, treatment-related side effects of NIV are not covered by these classical questionnaires. Our aim was to develop a simple, short and reliable tool to assess respiratory symptoms, sleep and comfort (or discomfort) as a complement to physiological monitoring of home NIV efficacy [9].
Methods
A detailed Methods section with a full description of models used for item analysis and selection as well as additional figures are available in the supplementary material. A shorter version of the Methods section is provided here.
Participants
Patients with stable disease who were treated at home with NIV were recruited in three French-speaking regional and university hospitals: Geneva University Hospitals (n=153) and Hôpital du Valais (n=95) in Switzerland, and Grenoble University Hospital (n=90) in France. All participants were “non-naïve” patients established on home NIV for at least 4 months prior to inclusion. NIV was used to treat a primary diagnosis of chronic obstructive pulmonary disease (COPD) (n=72, 21%), obesity hypoventilation syndrome (OHS) (n=96, 29%), central breathing disturbances during sleep (CBD) (n=114, 34%) [10], neuromuscular disorders (n=39, 10%) and restrictive disorders (n=17, 5%). Patients were excluded if they had a history of recent exacerbation or hospitalisation (<3 months), were unable to read or write French language, were unable to give informed consent or were unable to understand the questionnaire. Questionnaires were self-administered during a routine medical visit. All patients provided written informed consent and the study was approved by the ethics committee of all participating centres. The research was conducted according to the principles of the Declaration of Helsinki.
Item selection
We selected all items pertaining to “respiratory complaints” and “attendant symptoms and sleep” from the French translation and cultural adaptation of the SRI questionnaire [11] for further psychometric validation. All NIV experts in our group agreed that an additional assessment of a “comfort” dimension was needed. To this end, issues relating to the experience of NIV treatment (mainly comfort and side effects) were investigated in depth during 15 qualitative interviews with patients (D.A.). Other items were selected from two NIV comfort scales with no formal psychometric validation [12, 13]. All items were rated on a five-point Likert scale from “strongly disagree” to “strongly agree” and concerned the patient's status during the “last 4 weeks”. Selected core items consisted of 22 questions. Of these, eight questions were related to the “respiratory complaints” domain, seven to “attendant symptoms and sleep” and seven to NIV-related side effects.
Item analysis and reduction
An exploratory factor analysis was performed to evaluate the number of underlying dimensions [14]. An item response theory (IRT) graded response model was used on each unidimensional subscale for item reduction. Items with a low information function (i.e. item precision based on difficulty and discrimination for all values of the total scale) or with information that was redundant with another item were removed [15]. Item reduction also took into consideration expert opinion from investigators and other clinical experts in the field. Mean values for each subscore were transformed accordingly with a total score ranging from 0 to 10. The lowest possible score (0) corresponded to the highest impact of disease and treatment on quality of life, while the highest possible score (10) corresponded to the lowest impact of disease and treatment on quality of life.
Reliability
The internal consistency of each subscale was calculated by Cronbach's α coefficient [16]. Test–retest reliability of the final instrument was assessed by the intraclass correlation coefficient (ICC) in a subset of patients with stable disease (n=43) who completed the questionnaire 4 weeks after initial testing. Differential item functioning (DIF) analysis was used to assess whether the selected items were perceived differently by patients in different disease categories. To assess the validity of the factor structure, the fit of a confirmatory item factor model was evaluated using the M2 model fit statistic [17], the root mean square error of approximation and the standardised root mean square residual.
Construct and discriminant validity
Construct and discriminant validity was assessed by evaluating correlations between the final instrument and the French version of the St George's Respiratory Questionnaire (SGRQ) [18] and the Quebec Sleep Questionnaire [19].
All analyses were performed with R version 3.4.4 [20] and the mirt [21] and psych packages [22].
Results
Patient demographics
A total of 338 patients were included in the item reduction phase of the S3-NIV questionnaire. Demographics and clinical data for the overall population and each participating centre are presented in table 1. The most frequent indications for home NIV across all centres were CBD, OHS and COPD, including “overlap syndrome”. Patients had been established on home NIV for a median of 45 months (interquartile range (IQR) 21–93). Median forced expiratory volume in 1 s (FEV1) % pred was 36% (IQR 25–45) in COPD, 68% (47–85) in OHS, 88% (75–102) in CBD, 51% (28–76) in neuromuscular disorders and 51% (42–62) in restrictive disorders. Daily adherence downloaded from built-in ventilator software was 7.8 h (IQR 5.4–9.1) in COPD, 7.5 h (5.5–9.2) in OHS, 7.1 h (5.9–8.1) in CBD, 9.4 h (8.1–10.7) in neuromuscular disorders and 8.2 h (6.3–10.0) in restrictive disorders.
Demographics and clinical characteristics by recruiting centres
S3-NIV questionnaire score and subscores according to disease category
Item analysis and reduction
Patients used the full range of responses for the 22 selected core items (0–4), with mean item responses on the 0–4 scale ranging from 1.32±1.46 for “I have difficulties breathing during physical exertion” to 3.38±1.01 for “My ventilator inflates my lungs with too much pressure”. The rate of missing data was very low for all items (<3%). Preliminary nonparametric IRT analysis performed on the 22 core items indicated that two items, “I go to sleep easily” and “I sleep through the night easily”, were not scalable and were rejected. Based on the 20 remaining items, exploratory factor analysis suggested two latent factors that were interpreted as “respiratory symptoms” and “sleep & NIV-related side effects” dimensions (supplementary figure S1).
Four items pertaining to “respiratory symptoms” were rejected because their information function showed redundancy with other items: “I suffer from breathing problems even without physical exertion”, “I sometimes feel dizzy”, “I am tired during the day” and “I cough a lot”. For instance, the “cough” item was redundant with a “phlegm” item and was deleted after a round of expert opinion because cough encompasses less disease severity than the “phlegm” item, which was kept in the final tool. Four items pertaining to the “sleep & NIV-related side effects” dimension were also rejected because of redundancy with other items: “I often have neck pain”, “I often wake up at night”, “My ventilator doesn't inflate my lungs enough” and “My ventilator is too fast-paced”, and one item was rejected because of lack of fit: “My ventilator is too noisy”. The three most important issues related to the experience of NIV were: “My mask is uncomfortable”, “I am disturbed by leaks” and "I suffer from nasal or oral dryness”. All three items contributed to model information and fit and were kept in the final questionnaire.
Psychometric properties of the finalised S3-NIV questionnaire
The 11 items retained were used to assess the psychometric properties of the finalised version of the S3-NIV questionnaire (figure 1). The mean±sd of the S3-NIV questionnaire score was 6.71±1.82. The internal consistency of each subscale was good, with a Cronbach's α coefficient of 0.84 for the “respiratory symptoms” dimension, and 0.77 for the “sleep & NIV-related side effects” dimension. Test–retest reliability was equally good (ICC=0.72, 95% CI 0.54–0.84) in 44 patients 4 weeks after the initial assessment.
The final version of the S3-NIV questionnaire. The total score can be computed as the average of all answered items multiplied by 2.5. The lowest possible score (0) corresponds to the highest impact of disease and treatment, while the highest possible score (10) corresponds to the lowest impact of disease and treatment.
The distribution of the S3-NIV score is depicted in figure 2a and shows that the entire scaling range was used in our validation study. Of the 338 patients, 80% used 48% of the scaling range; 10% had a score <4.32 units and 10% had a score >9.01 units. Figure 2b shows S3-NIV total scores by disease category with no floor or ceiling effect in any disease category. The impact of disease and treatment in COPD patients was as severe as in neuromuscular patients, whereas a lower impact of disease and treatment was demonstrated for OHS and CBD patients (ANOVA, p=0.004) (figure 2b). The DIF analysis of the “respiratory symptoms” dimension identified only one item (“I am often short of breath”) as significantly different between disease categories (p=0.03). All items from the “sleep & NIV-related side effects” domain were identified as invariant across disease categories. The validity of the latent structure of the 11 remaining items was assessed using a confirmatory item factor model with two latent factors for the two domains. The fit of the model was very good (M2=0.5, df=10, p=0.400; root mean square error of approximation=0.011, 95% CI 0–0.061; standardised root mean square residual=0.076).
a) Empirical cumulative distribution of the S3-NIV questionnaire total score in 338 patients treated with home noninvasive ventilation (NIV). b) Distribution of the S3-NIV total score by disease category. Dots represent individual data. COPD: chronic obstructive pulmonary disease; OHS: obesity hypoventilation syndrome; CBD: central breathing disturbances during sleep.
External validation
Construct validity was assured by the correlation analysis between S3-NIV questionnaire scales and various scales of the SGRQ and Quebec Sleep Questionnaire. A high correlation was found between the “respiratory symptoms” domain of the S3-NIV questionnaire (five items) and the symptom scale of the SGRQ (eight items) (rho= −0.76, p<0.001). The correlation between the “sleep & NIV-related side effects” domain of the S3-NIV questionnaire and the Quebec Sleep Questionnaire was rho=0.51 (p<0.001) (supplementary figure S2A, B). Correlation of the S3-NIV questionnaire total score with the Quebec Sleep Questionnaire was rho=0.67 (p<0.001) and rho= −0.60 (p<0.001) with the SGRQ symptom scale. Patients' perceptions of symptoms as assessed by the S3-NIV questionnaire did not correlate with objective pulmonary function measurements (FEV1 % pred: rho=0.12, p=0.109; forced vital capacity % pred: rho=0.08, p=0.270) or with daily adherence downloaded from the built-in ventilator software (rho=0.09, p=0.158).
Discussion
This study has validated a short, simple, patient-completed, specific tool that monitors home NIV as a complement to the monitoring of physiological variables. The S3-NIV questionnaire covers important patient-oriented dimensions related to NIV monitoring and treatment, i.e. respiratory symptoms, sleep quality and NIV-related side effects. This tool uses a limited number of items (n=11), which have been validated in a large international sample of patients corresponding to practices in different NIV services. Testing for internal consistency demonstrated acceptable to excellent reliability. Construct validity was also established using the SGRQ and the Quebec Sleep Questionnaire as references.
A stepwise approach combining arterial blood gas samples, pulse oximetry with capnography (when available), data from built-in ventilator software and polygraphy or polysomnography under NIV [6, 9, 12, 23] is the currently recommended method for adapting ventilator settings. However, there is a lack of tools to assess patient-oriented outcomes in order to inform the organisation of NIV services and allow comparisons for quality of care. Patient-centred outcomes and physiological parameters used to monitor chronic respiratory failure address different dimensions of care, as illustrated in our data by the weak association between the S3-NIV questionnaire and FEV1.
We combined factor analysis and IRT with expert clinical judgement. Items with poor measurement properties were first flagged for deletion according to predetermined rules. When designing the scale, we balanced weaknesses and strengths in overall contributions before deciding whether to include or exclude any item. No single theoretical criterion was used. Clinical judgement was shared and discussed between expert co-authors throughout the entire questionnaire design process. The final S3-NIV questionnaire consists of 11 items formatted as a five-point Likert scale, thus making the questionnaire easy for patients to understand and to complete. We consider the S3-NIV questionnaire to possibly be the most suitable tool currently available because it has been specifically developed for clinical practice in NIV services. Until the wider application of the S3-NIV questionnaire is supported by evidence, its use should be restricted to the assessment of clinically stable, non-naïve patients admitted for routine follow-up of NIV. Another application of a short, patient-oriented, self-administered tool can be anticipated with the exponential development of home NIV telemonitoring [24]. In this particular setting, it could be very beneficial to merge the data downloaded from the built-in software with a brief, self-administered questionnaire in an integrated e-health platform tailored to patients' needs and NIV services. This paradigm shift in care planning has already been tested with success in cancer patients [25], as well as in severe COPD [26] and sleep apnoea [27].
The 11 items were selected to cover a wide range of disease severity in a large cohort with the intention of demonstrating persistent discriminant power across different settings, practices and underlying diseases. This is illustrated by the distribution of the S3-NIV questionnaire score over the entire scaling range for the overall study population, including in specific disease subgroups, as demonstrated by DIF analysis. Indeed, both ambulatory patients with moderately severe disease and individuals with high NIV dependency, such as those in late stages of neuromuscular disorders, were included in the study population to reflect current trends in home NIV indications [28]. Excellent measurement properties have been demonstrated for the S3-NIV questionnaire across all patient subgroups, which indicates that it is suitable for widespread use, increases practicability for the organisation of home NIV services and facilitates the training of home care providers. Overlap syndrome (i.e. COPD with obstructive sleep apnoea syndrome) was merged with the COPD subgroup. However, a post hoc analysis (data not shown) of the S3-NIV questionnaire subscores in COPD and overlap syndrome demonstrated that even if the total scores were the same, patients diagnosed with overlap syndrome exhibited a trend for higher scores on the “respiratory symptoms” domain and lower scores on the “sleep & NIV-related side effects” domain compared to patients with only COPD. Further studies with an appropriate sample size are needed to address this specific question.
Our study has some limitations. It presents cross-sectional data, similar to the original SRI questionnaire [7]. Thus, a prospective longitudinal study will be required to assess the minimal clinically important difference, as well as the sensitivity of our tool to changes over time or changes induced by NIV settings/interface modifications. Another limitation stems directly from the questionnaire development methodology. We only included items pertaining to respiratory symptoms, sleep quality and side effects related to NIV in the preselected core of items before the reduction process. We then eliminated items that were infrequent, redundant or had poor scaling properties with the aim of developing a short, practical tool with the least number of items. This has two potential undesirable consequences. First, the S3-NIV questionnaire can only be used as a clinical tool for NIV monitoring and is not a surrogate measure of general health status or quality of life. In contrast, the original longer SRI questionnaire may be more appropriate if a systematic analysis of health status is required or for interventional randomised controlled trials. Second, we may have underestimated important items for a small group of patients that would have been retained using a different methodology based on clinical importance. For instance, noisy alarms may seriously disturb sleep and, in turn, alter daytime symptoms in a few patients using conservative alarm settings. Therefore, our instrument may not be appropriate for a small number of very dependent patients. Specific studies need to be dedicated to theses subgroups. Finally, the S3-NIV questionnaire was only tested in a French-speaking international cohort with a thorough item reduction methodology starting from the original SRI questionnaire. This was made possible because the French translation and cultural adaptation of the original questionnaire was already available [11]. However, because most items pertaining to respiratory symptoms and sleep have been already translated and formally validated in English [29], we argue that the S3-NIV questionnaire does not need further English cultural validation to be relevant and used in patients with stable disease treated with home NIV.
In summary, the S3-NIV questionnaire provides clinicians and patients with a simple and reliable tool to assess three important domains related to home NIV as a complement to physiological monitoring of home NIV. This will facilitate shared decision-making in home NIV services. Although the S3-NIV questionnaire is short with a simple scoring algorithm, its content and layout can serve as a backbone during consultations to identify key areas to be further explored in a personalised interview in order to optimise care delivery in this patient population.
Supplementary material
Supplementary Material
Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.
Supplementary methods, table, figure legends and references ERJ-01182-2018_Supplement
Supplementary figures ERJ-01182-2018_Suppl_Figures
Acknowledgements
The authors would like to thank all patients who participated in the study. The authors wish to thank Rosemary Sudan (Geneva, Switzerland) for editorial assistance.
Footnotes
This article has supplementary material available from erj.ersjournals.com
Conflict of interest: G. Gex has nothing to disclose.
Conflict of interest: P. Pasquina has nothing to disclose.
Conflict of interest: P-O. Bridevaux has nothing to disclose.
Conflict of interest: J-C. Borel reports grants and personal fees from Philips Healthcare (for the EXA-VNI study, lectures and a congress invitation), salaries from AGIR à dom (home care provider), personal fees and other support from ResMed (for lectures and a congress invitation), and a patent from Nomics SA, all outside the submitted work.
Conflict of interest: P. Gasche-Soccal has nothing to disclose.
Conflict of interest: W. Windisch reports that the Cologne study group received an open research grant from Weinmann, Vivisol, Heinen and Löwenstein, and VitalAire (all in Germany), during the conduct of the study; and reports receiving speaking fees from companies dealing with long-term oxygen treatment, outside the submitted work.
Conflict of interest: J-L. Pépin has nothing to disclose.
Conflict of interest: J-P. Janssens has nothing to disclose.
Conflict of interest: D. Adler has nothing to disclose.
Conflict of interest: E. Dupuis-Lozeron has nothing to disclose.
Support statement: This study was partly supported by the Fonds de la Loterie Romande 2014 and by the Fonds de la Ligue Pulmonaire Genevoise 2014. J-L. Pépin is supported by the French National Research Agency, in the framework of the Investissements d'avenir programme (grant no. ANR-15-IDEX-02), and the Chair of Excellence of the E-health and integrated care in chronic disease, from the Université Grenoble Alpes foundation. Funding information for this article has been deposited with the Crossref Funder Registry.
- Received June 27, 2018.
- Accepted September 12, 2018.
- Copyright ©ERS 2018
References
