Abstract
The home ventilator market has grown in size and complexity. The aim of this study was to determine if common home ventilators are user-friendly for trained intensive care unit (ICU) physicians.
Eleven ventilator models were tested by 13 ICU physicians without practical experience in home mechanical ventilation. Six tests were defined (start-up, unlocking, mode and setting recognition, mode change, pressure setting and alarm). For each test, the physicians were timed and their performance compared with a reference time established by a technician. The physicians also had to rate their global assessment of each machine on a visual analogue scale.
The start-up test was the only test for which there was no significant difference between the physicians and the technician, except for two ventilators. The physicians were slower than the technician to unlock the ventilator and change the ventilatory mode, with some complete failures during these tests and heterogeneous results between physicians and between ventilators. Mistakes occurred in close to 50% of cases during the ventilatory mode and settings recognition test. The mean time for the most rapid of the physicians for all the tests was 58±53 s, compared with 15±9 s for the technician.
In conclusion, trained intensive care unit physicians perform poorly when confronted with home mechanical ventilators without specific prior training. Therefore, it is hypothesised that the user-friendliness of home ventilators for other categories of users might be questionable.
The indications for home mechanical ventilation are numerous in both adults and children 1. Developments in design and technology since 1996 have led to considerable improvements in the mechanical ventilators available to physicians and patients for home use. In the past, the very limited number of models available only permitted controlled ventilation with very few settings possible, and only basic monitoring. There are now >30 models on the market, with each providing several ventilation modes and offering numerous options for settings. However, no common nomenclature exists (table 1⇓). This diversity introduces flexibility and also complexity, thus involving the risk that medical and paramedical personnel responsible for the care of patients using home ventilation may not be able to properly manage the technical aspects. This risk is particularly worrying when urgent or semi-urgent reaction to a situation is required, in the knowledge that the personnel involved is more unlikely than likely to have previous familiarity with home mechanical ventilators. It is probably desirable that any physician dealing with a patient on home mechanical ventilation should be able to easily recognise the ventilation mode administered, understand the source of alarms or malfunctions without alarms, and take simple rapid measures for the patient's safety. This is particularly important in patients who are ventilator dependent or nearly so, a population that is of growing importance in the home ventilation setting.
Ventilation modes supplied by the various ventilators tested, using the names devised by the manufacturers
Various factors contribute to making such expectations unrealistic. There are very few training programmes in home mechanical ventilation for physicians and caregivers 2. Manufacturers of home ventilators are familiar with technological bench-test assessments 3–5 and are, however, unable to easily evaluate the user interface of the machines they develop. When they do, they usually turn to physicians experienced in the use of home mechanical ventilators, thus biasing the findings. Paradoxically, there are no marketing regulations for ventilators. This leaves manufacturers free to offer novel control panels and choose the names given to the ventilation modes they provide (to the extent that an identical mode can have several names), and so on. An overview of the home mechanical ventilator market gives the general impression that there is no homogeneity; combinations of buttons are frequently required to start or stop a function, the labelling of buttons is not very clear, and control screens tend to be too small and difficult to read.
Despite this observation, no published data seem to exist that would convert this impression into findings, and thus prompt manufacturers to concentrate their efforts on designing sufficiently simple machine-user interfaces to guarantee safe quality care. In this context, the objective of the current study was to evaluate the user-friendliness of the 11 home mechanical ventilators most frequently used in France for trained intensive care unit (ICU) physicians.
MATERIAL AND METHODS
Ventilators
Eleven ventilator models were tested, in accordance with the following list: Eole 3 XLS® (Saime, Savigny le Temple, France), Hélia 2 (Saime), Onyx plus® (Tyco, Saint Louis, MO, USA), VPAP III® (ResMed, North Ryde, NSW, Australia), BiPAP Synchrony® (Respironics, Murrysville, PA, USA), Smartair PLUS® (Airox, Pau, France), VS Ultra® (Saime), Neftis® (Taema, Anthony, France), Knightstar® (Tyco), PV 403® (Breas Medical, Mölnlyche, Sweden) and Légendair® (Airox).
Each ventilator was connected to a 2-L test bag while tests were being performed.
Physicians
Thirteen physicians with sound experience in mechanical ventilation in the context of intensive care, but without practical experience in home mechanical ventilation, participated in the study (five specialists in respiratory medicine, five specialists in intensive care, two neurologists, one anaesthetist), all qualifying as “senior ICU physicians” although with various degrees of experience due to an age range of 32–56 yrs. Only one of the 13 ICU physicians had been in contact with the Onyx plus® ventilator before, two had previous contacts with the Helia 2® (Saime, Savigny le Temple, France), three with the VS Ultra®, and one with the Légendair®. In all of these cases, the participants did not consider themselves familiar with the ventilators. The situation was slightly different for Eole 3 XLS®, which seven of the participants already knew with some degree of familiarity.
Tests
Six tests were defined. Each test was explained to the physician; the examiner gave the starting signal and timing was either stopped as soon as the objective fixed had been achieved, or at the arbitrarily decided limit of 3 min. Each physician performed the six tests consecutively for the specified ventilator, but the order in which the ventilators were evaluated was randomised. The test list was as follows.
Test 1: Start-up
With the ventilator completely assembled and connected to the power supply, the physicians had to start the ventilator; the stop signal was given at the first insufflation produced by the ventilator.
Test 2: Unlocking
The International Organization for Standardization (ISO) standard 6, 7 stipulates that there must be a safety mechanism to prevent any accidental adjustment of controls on a mechanical ventilator installed at the home of a patient; a physician wanting to change any ventilation setting must first disable this safety mechanism. However, the standard does not provide any information on what this safety mechanism should be; home mechanical ventilator manufacturers have adopted very different solutions. Test 2 required physicians to unlock a previously started ventilator, without consulting the operating manual. The stop signal was given as soon as the physician had actual access to ventilator settings.
Test 3: Recognition
This test required physicians, with a ventilator that was turned on and supplying a given ventilation mode, to fill in a chart identifying the ventilation mode and the main preset parameters, which were tidal volume (VT) and breathing frequency (f) in volume-controlled mode, and inspiratory pressure support and positive end-expiratory pressure in pressure-controlled mode.
The ventilator modes were as follows. Onyx plus®: ventilation spontanée avec aide inspiratoire [spontaneous ventilation with pressure support] (VSAI); Légendair®, ventilation en pression contrôlée [pressure control ventilation] (VPC); Neftis®: ventilation assistée contrôlée [assist-control ventilation] (VAC); PV 403®: aide inspiratoire avec ou sans fréquence de sécurité [inspiratory pressure support (with or without minimal frequency)] (AI); BiPAP Synchrony®: spontané/temporisé (VPAP III®) [spontaneous with temporisation] (S/T); Knightstar®: 2 niveaux de pression avec fréquence minimale [two pressure levels with minimal frequency] (A/C); Smartair PLUS®: aide inspiratoire avec fréquence respiratoire de sécurité [inspiratory pressure support with security frequency] (Aifr); VPAP III®: S/T; VS Ultra®: aide inspiratoire avec volume assuré [inspiratory pressure support with minimal volume] (AIVt); Eole 3 XLS®: ventilation assistée contrôlée [assist-control ventilation] (VAC); and Helia 2®: AIVt (table 2⇓).
Scores given to ventilators by physicians after completing the tests
The stop signal was given as soon as the chart was filled in.
4) Mode change
Starting with a ventilator preset to supply pressure support ventilation and unlocked, the physicians had to change to volume-controlled mode and adjust VT and f to predefined values. The stop signal was given as soon as the first insufflation was achieved with the required settings. This test only concerned mixed type ventilators providing the possibility of both pressure- and volume-controlled ventilation (VS Ultra®, Helia 2®, Légendair® and Neftis®).
5) Pressure setting
Starting with a preset and unlocked ventilator, the physicians had to set a precise level of inspiratory pressure support. The stop signal was given as soon as the first insufflation was achieved with the required settings. This test only concerned ventilators providing pressure-controlled ventilation (Knightstar®, VPAP III®, BiPAP Synchrony®, Smartair PLUS® and Onyx plus®).
6) Alarms
Starting with a preset and unlocked ventilator, the physicians had to adjust alarms (high pressure, low pressure and apnoea) to predefined values. The stop signal was given as soon as the alarm values had been adjusted to the required levels. This test naturally only concerned ventilators equipped with alarms (Légendair®, Eole 3 XLS® and VS Ultra®)
Evaluation
For each test, the time taken by the physicians was compared with a “reference time” established by a technician from the Comité d'Assistance Respiratoire à Domicile d'Ile-de-France (CARDIF; Paris Region Committee for Home Respiratory Assistance, Paris, France) with thorough knowledge of the ventilators tested.
Moreover, once all the tests were completed for a given ventilator, the physicians had to rate their assessment on a visual analogue scale along a 10-cm line marked with (0) on the left for “very difficult to use”, and (10) on the right for “very easy to use”.
Statistical analysis
For each of the six tests performed, variance analysis was carried out using a “physician” factor (including the results of the 13 physicians and those of the technician), and a “ventilator” factor. Comparison of the physician results with those of the technician was performed using a post hoc Dunnett test. Comparison of results between physicians and comparison of ventilators was performed using a Tukey test. For all comparisons, the significance threshold was fixed at the value of p 0.05. The results were expressed in the form of mean±sd.
RESULTS
Overall results
Figure 1⇓ shows the mean results obtained by the physicians (for all the tests on all the ventilators) compared with the technician. The mean time (mean±sd (range)) for the most rapid of the physicians for was 58±53 (5–180) s, compared with 15±9 (6–27) s for the technician.
Mean time for performing all the tests on all the ventilators by physicians (□) and the technician (▓). There were no significant differences between physicians, but all physician times were significantly longer than the technician time (≤p = 0.001).
Table 2⇑ shows the scores concerning ventilator user-friendliness given by the physicians after completing the tests. None of the differences reached the statistical significance threshold.
Results by test
1) Start up
The ventilators were started in 17±10 s for the most rapid of the physicians, versus 13±6 s [6–27] for the technician. There were no significant differences between physicians and the technician or between the physicians. The ventilators were distributed in two groups within which there were no differences, but between which there was a significant difference. In fact, two ventilators, the Neftis® (61±22 s) [29–135] and the Knightstar® (70±61 s)[12–65], required significantly more time to start than the other nine (p<0.0001). Results are shown in figure 2⇓.
Timing results for test 1 (start-up). The graph shows the time required to successfully perform the test for each of the 11 ventilators tested. For each ventilator, the box corresponds to the 75th percentile of the data distribution with indication of the median, whereas the whiskers indicate the 90th percentile. ▴: the reference time established by the technician.
2) Unlocking
On average, two physicians out of 13 did not take significantly longer than the technician to unlock the 11 ventilators, despite differences that could have a clinical impact (12 s on average for the technician, against 49 and 59 s on average for the other two physicians). The 11 other physicians were significantly slower than the technician in the procedure for unlocking ventilator settings. Concerning the ventilators, the Eole 3 XLS® proved to be significantly quicker to unlock than the other models (31±17 (12–66) s). The BiPAP Synchrony® and Knightstar® models proved to be significantly longer to unlock that the other machines (173±42 (32–180) s and 170±37 (73–180) s, respectively, p<0.0001). No physician succeeded in unlocking the VPAP III® ventilator in the allotted time limit of 3 min. Results are shown in figure 3⇓.
Timing results for test 2 (unlocking). The graph shows the time required to successfully perform the test for each of the ventilators tested. For each ventilator, the box corresponds to the 75th percentile of the data distribution with indication of the median, whereas the whiskers indicate the 90th percentile. ▴: the reference time established by the technician.
Settings for the Knightstar® ventilator took statistically longer to analyse than the others (p = 0.01), which were evenly distributed in two homogeneous groups (51–70 and 80–110 s).
3) Recognition
Eight physicians out of 13 proved to be significantly slower than the technician in this test. For the remaining five, the difference was not significant, but the physician times were 2–3 times that of the technician (24 s on average for the latter, 47–71 s for the physicians). Moreover, the answers given by the physicians proved to be erroneous on at least one point in 49% of the cases (fig. 4⇓; wrong mode: 13%; wrong frequency: 1%; confusion between inspiratory pressure support and intermittent positive airway pressure: 21%; confusion between the set value of a given variable and its measured value: 12%; no recognition at all: 2%). Results are shown in figure 5⇓.
Results of test 3 recognition of modes and settings. □: adequate recognition; ▪: no recognition; ░: confusion between set value and measured value; ▒: confusion between inspiratory pressure support and intermittent positive airway pressure; ▓: wrong frequency; ░: wrong mode.
Timing results for test 3 (recognition). The graph represents the time required to successfully perform the test for each of the 11 ventilators tested. For each ventilator, the box corresponds to the 75th percentile of the data distribution with indication of the median, whereas the whiskers indicate the 90th percentile. ▴: the reference time established by the technician.
4) Mode change
Seven physicians were significantly slower than the technician for this test, but only one physician was significantly slower than the others (124±70 s (70–180) versus 95±61 s (12–480), p = 0.04). The Helia 2® ventilator was markedly different from the others, as only one of the physicians succeeded in changing to volume-controlled mode (concealed function). Results are shown in figure 6⇓.
Timing results for test 4 (mode change). The graph shows the time required to successfully perform the test for each of the ventilators tested. For each ventilator, the box corresponds to the 75th percentile of the data distribution with indication of the median, whereas the whiskers indicate the 90th percentile. ▴: the reference time established by the technician.
5) Pressure setting
Six out of the 13 physicians were significantly slower than the technician in this test. They were distributed in two homogeneous groups within which there were no differences (a group of three physicians timed at 128–143 s, and 43–124 s for the other group). The test was carried out significantly faster on the Smartair PLUS® ventilator (mean 56 s) than the other ventilators that underwent this test (84–129 s). Results are shown in figure 7⇓.
Timing results for test 5 (pressure setting). The graph shows the time required to successfully perform the test for each of the ventilators tested. For each ventilator, the box corresponds to the 75th percentile of the data distribution with indication of the median, whereas the whiskers indicate the 90th percentile. ▴: the reference time established by the technician.
6) Alarms
Again, in this test, six out of the 13 physicians proved to be significantly slower than the technician. There were no differences between the ventilators. Results are shown in figure 8⇓.
Timing results for test 6 (alarms). The graph shows the time required to successfully perform the test for each of the ventilators tested. For each ventilator, the box corresponds to the 75th percentile of the data distribution with indication of the median, whereas the whiskers indicate the 90th percentile. ▴: the reference time established by the technician.
DISCUSSION
The present study, which is apparently one of the first of this type, brought out both positive and negative elements. On the positive side, it was noted that a variable proportion of physicians participating in the study were able, without previous training, to equal the performance of a technician experienced in the use of home mechanical ventilators. On the negative side, the reverse was true, and physicians were often slower that the trained technician. It should be emphasised that, even though differences in timing did not reach the statistically significant threshold, the physicians were sometimes very slow in comparison with the reference test. The results of the unlocking test, recognition test (49% errors, fig. 4⇑) and settings test (mean value of four times the time taken by the technician) are cause for concern. It was also noted that there were some more specific problems, such as the impossibility for all the physicians but one to access the change to “volume” mode on one of the machines tested. All these points result in a quite mediocre overall score (table 2⇑), although it is granted that the lack of significant differences between ventilators could be due to an insufficient statistical power.
Possible limitations to the study
The objective of the current study was not to describe the full extent of the difficulties that patients, and their families and caregivers, receiving home mechanical ventilation can be faced with. Rather, from the present author’s experience, it was felt that attention must be brought to the blatant lack of user-friendliness of home ventilators. For this reason, the present study was restricted to ICU physicians unaware of the specifics of home ventilation, but well accustomed to the use of various types of mechanical ventilators and also accustomed to managing some ventilators despite having little background about their particular type. The present study reveals nothing about the ease of use of the ventilators for more “ordinary consumers”. Nevertheless, the difficulties encountered by ICU physicians (who should represent the professional category with both the highest and the most homogeneous skills in mechanical ventilation) make the chances slight that other unprepared physicians or caregivers called in to provide care for home-ventilated patients will be at ease with the home ventilators. It is noteworthy that the survey was deliberately not conducted with physicians experienced in home mechanical ventilation. In all likelihood, they would have obtained results closer to those of the technician who established the reference times, but it would have been difficult to objectify what their “experience” actually was, and, thus, to constitute a homogeneous group. Physicians were also avoided who had no experience of artificial ventilation, as this would have created the opposite bias. Similar studies involving other professional categories would be interesting.
Some of the participants in the study had some prior knowledge of some of the ventilators tested (see Methods). This did not influence the results, except perhaps for the Eole 3 XLS® ventilator, which was the best known of the 13 models tested: this may explain why it was the fastest ventilator to unlock during test 2.
The technician who established the reference times was highly trained, and perhaps these reference times were unreasonably short. Nevertheless, while it is not surprising that unfamiliarised physicians would take longer than a trained technician to perform the tests, some of the recorded differences are huge, and the important variability among physicians must be noted. In addition, the performance of the ICU physicians was poor, not only in relation to the reference time, with time-independent recognition errors and many complete failures to perform some of the tests. Not having given any training to the physicians before the tests could also be criticised, but this appeared to be the best possible standardisation, and does in fact correspond to many real-life situations.
Finally, on methods, the range of ventilators tested in this study does not represent all the available machines. However, it does correspond to the machines most often used in France, and is varied in terms of brands and models. From this point of view, the present study is representative of possible clinical situations.
Start-up and unlock procedures
Even though all the physicians had taken longer than the reference time to start the ventilators, the results of test 1 can be considered to be satisfactory. The only two ventilators that proved to be more difficult to start were unlike the others. In one case, it was the position of the “on/off” button (Knightstar®: on the side of the machine instead of the front panel); in the other case, it was the type of operation required to activate the button (Neftis®: brief instead of prolonged pressure). Even though it can appear to be a trivial point, this suggests that the ventilator on/off button should be systematically placed on the front panel of the machine, and should be operated by pressure sufficiently long enough to meet the ISO standard safety requirements (i.e. “means shall be provided to prevent accidental operation of the on/off switch” 6, 7), but without imposing a time limit.
The ISO standard in force 6, 7 recommends the presence of “a means of protection against inadvertent adjustment of controls that can create a hazardous output (involving risks)”. The standard does not specify whether the aim of these means is to avoid accidental adjustments, or to avoid access to adjustments by patients and their families. However, it is obvious that the recommendation is directed at the first and not the second case, as the unlocking procedures are in the ventilator operating manual and can be found by trial and error. Patients who want to modify the settings of their ventilators for some reason or other will always find a way. In this context, the results of test 2 provide a good indication of a real problem: the mean unlocking times greatly exceeded the reference times, and a certain number of failures were recorded (systematically for one of the machines). Two physicians achieved unlocking times that were not statistically different from the reference time, yet the difference (average 49 and 59 s, versus 12 s for the technician) might be clinically significant in a crisis situation. The ISO 10654-6:2004 standard 6 states that “mechanical control techniques such as locks, shielding, friction-loading and detents are considered suitable”. The present authors consider these solutions to be preferable to the current ones, especially when the present solutions impose multiple button combinations that are particularly “anti-intuitive”. Indeed, it is believed to be important to be able to unlock a ventilator relatively easily, as this is a prerequisite to any intervention if the need for a change in ventilatory mode or ventilatory settings arises. In a caricatural manner, if the concerned patient is ventilator-dependent or nearly so, failure to unlock the ventilator makes a machine switch or manual ventilation the only solutions.
Recognition of settings
Though the setting recognition charts were completed quite rapidly by the physicians, 49% of the charts were incorrect. The two main sources of error were, on the one hand, the sequential display of the measured values and the set values on the same screen and, on the other hand, the heterogeneous terminology (table 2⇑). Theoretically, correcting the first factor would be simple; for manufacturers this would involve allowing for the separate display of measured values and set values. This obviously has a cost, but is unlikely to be weighed against the safety flaw revealed by the present results. Concerning heterogeneous terminology, it is probably up to the medical profession to take action to establish an international nomenclature for modes of assisted ventilation. In France, Chopin and Chambrin 8 have published a proposition of this type in the journal “Réanimation-Urgences”. Recommendations have also been issued by the French learned society for intensive care 9. To the present authors’ knowledge, these initiatives have not had much following. Several not mutually exclusive explanations can be put forward. First, the journal “Réanimation-Urgences” (now “Réanimation”) is not indexed in the Medline database. Secondly, the nomenclature proposed by Chopin and Chambrin 8 is very “physiological” in nature, but a certain degree of pragmatism is probably required, particularly concerning terms that are already accepted through use. Thirdly, awareness of the risks created by the absence of common terminology for modes of assisted ventilation (whether home ventilation or in intensive care) is necessary at a medical community level, including the learned societies. It is hoped that this study will contribute towards this.
Mode changes, settings and alarms
The results of these three tests appear to be, in a way, less worrying than the preceding tests. However, it remains that some ventilators posed problems for certain physicians, including some regarding the particularly important issue of alarms. It should be emphasised that many physicians had inadvertently changed ventilator settings while trying to analyse the preset parameters; this possibility had not been foreseen in the study design but would have warranted specific analysis.
Conclusions
Home mechanical ventilators have benefited from considerable advances in design and technology. They are sophisticated machines whose reliability and performance are validated by detailed technical evaluations 3–5. It is regrettable that this technical excellence is tarnished by inadequate ergonomics; at the most, this is an unjustifiable source of risk for patients and, at the least, a cause of suboptimal use by physicians and caregivers. To some extent, this issue also pertains to ICU mechanical ventilators about which some research has already been performed regarding technological specificities 10 and user interfaces 11.
The results of the evaluation carried out here should encourage corrective actions by both the manufacturers and the medical community. These actions require institutional management, through learned society working groups (for example, with a view to drawing up an international nomenclature) or statutory measures. It is indeed surprising that sensitive devices, such as ventilators, are not subject to evaluation regulations similar to those in force for medications. Thus it would not be outrageous to envisage ventilator manufacturers being obliged to conform to a few simple regulations (standardised starting and locking systems, homogeneous nomenclature). In any event, improving home ventilator user-friendliness is important (and would be relatively easy); ventilators will become more numerous with the diversification of indications for this treatment method and increases in the populations concerned.
Acknowledgments
The authors would like to thank F. Bolgert, C. Cracco, S. Demeret, N. Deye, F. Lellouche, M. Prella, H. Prodanovic, C. Raynaud, M. Raux, C. Straus and M. Wysocki for the time they devoted to performing the tests. They would also like to thank W. Brossard for his technical assistance, C. Melot for his advice on statistical analysis, and M. Amouyal-Jones for assistance with the English manuscript.
- Received July 6, 2005.
- Accepted February 2, 2006.
- © ERS Journals Ltd
