|
|
||||||||
Depts of 1 Physiology and 2 Medicine, Queen's University, Kingston, Ontario, Canada
CORRESPONDENCE: P.K. McFawn, Dept of Physiology, 4th Floor Botterell Hall, Queen's University, Kingston, Ontario, K7L 3N6, Canada. Fax: 1 6135336880
Keywords: airway mechanics, bronchoscopy, endoscopy, image analysis
Received: September 6, 2000
Accepted April 18, 2001
This work was supported by the Canadian Institutes of Health Research and the Ontario Thoracic Society. P.K. McFawn was supported by the John Alexander Stewart Fellowship and a Canadian Lung Association/CIHR/GlaxoWelcome Fellowship.
| Abstract |
|---|
|
|
|---|
Test objects of known size were imaged with four different bronchoscopes. Commercial image analysis software was used to measure the size of features in the images before and after applying distortion correction algorithms. The technique was then applied by measuring airway narrowing in anaesthetized pigs during vagal stimulation.
Without correction, object size near the edge of the field of view is underestimated by
40%. The error in measured diameter of concentric circles was dependent on the radius of the circle, increasing to 25% for circles occupying 90% of the field. Third order polynomial functions were required to correct these errors. After correction, errors were independent of object size or location in the image. Correction for lens distortion was independent of the distance between bronchoscope and object.
The authors conclude that modern image processing software can correct for the distortion produced by wide-angle bronchoscope lenses.
Bronchoscopy is a commonly used and relatively inexpensive clinical tool. Frequent diagnostic uses of bronchoscopy include dimensional evaluation of airways, masses, tracheal stenosis or foreign bodies, as well as assessment of the effectiveness of therapeutic interventions, such as radiotherapy of endobronchial cancer, etc. 14. Until now, these evaluations have been qualitative, with quantitative assessment limited by the optical distortion produced by the bronchoscope lens. The ability to measure endobronchial objects or airway dimensions accurately would move bronchoscopy from a qualitative to a quantitative research and clinical tool.
Quantification of bronchoscopic images has the potential to be a valuable tool for both research and clinical investigation. Traditional methods of measuring airway responsiveness using lung or airway resistance provide information about the tracheobronchial tree as a whole. However, these measurements are a composite of many different airways in the bronchial tree and do not allow for the observation of the response of an individual bronchi 59. Previously, quantitative studies of airway narrowing in vivo have relied on tantalum bronchography or high-resolution computed tomography (HRCT), both of which are prohibitively expensive or unavailable for most laboratories 612. In contrast, bronchoscopy is a relatively inexpensive and routine procedure that allows direct, real-time imaging of the bronchial lumen. Quantification of bronchoscopic images would provide a useful alternative to tantalum bronchography or HRCT for measuring the responsiveness of individual airways in vivo 13.
To provide an adequate field of view (FOV), bronchoscopes are fitted with a wide-angle "fish-eye" lens, which distorts the image, making objects at the periphery of the image appear smaller than those at the centre. This form of distortion has been termed "barrel distortion" 14, 15. Bronchoscopic distortion is radial, increasing with displacement from the centre of the FOV in a nonlinear manner 13, 1518. Bronchoscopic distortion also alters the apparent shape of objects, making straight lines appear curved and compressing objects radially so that a circle will appear elliptical near the periphery of the field. The extent of distortion depends on how close the object is located to or extends to the edge of the field 13. Without correction, only small objects near the centre can be measured accurately 13, 15, 19, 20.
Most previous studies that measured dimensions using bronchoscopy created correction factors that could be applied to a measured area or diameter based on the location and size of the measured object in the FOV 13, 21, 22. Such attempts are relatively crude approximations, and require several different correction factors to deal separately with the varying effects of location and size. Because the distortion of bronchoscopic images is regular and uniform, numerical methods should be able to correct for the distortion by reversing the distortion mathematically to allow accurate measurements. While little work on correcting bronchoscopic images has been published, several studies have investigated other applications of endoscopic measurement 1618, 20, 23. They used gastric endoscopes, which are much larger than bronchoscopes, with correspondingly larger lenses and potentially different optics and extent of distortion. These endoscopic studies used polar coordinates because they simplify the correction methods. However, polar coordinates also complicate image handling, as common file formats, computer displays, image capture cards, etc. use Cartesian coordinates. In a recent study, numerical methods were applied to bronchoscopic image distortion, but the algorithms used for the correction were not described 24. Previous applications of numerical methods to correct either endoscopic or bronchoscopic images for lens distortion have used software designed by the authors that is unavailable to other laboratories 1618, 21, 22. These factors have restricted the use of quantitative bronchoscopy and endoscopy.
Recent developments in imaging hardware and software have made powerful distortion correction techniques available in inexpensive commercial image analysis systems. The present study investigated the ability of distortion correction algorithms to compensate for distortion and make quantitative measurements from bronchoscopic images. Firstly, the order of polynomial function required to correct for distortion was determined. The accuracy of the correction procedures for small peripheral versus small central objects and for central objects of different sizes were then investigated. Finally, the authors applied the technique to measure airway narrowing during vagal stimulation.
| Methods |
|---|
|
|
|---|
For each bronchoscope, correction functions were established by acquiring images of graph paper with 1 mm line spacing placed 5, 10 and 15 mm from the bronchoscope tip. A micromanipulator and swivel were used to ensure that the graph paper was imaged square-on and aligned with a vertex in the centre of the FOV. For all measurements, the centre of the FOV was found and used as the reference origin (i.e. the point (0, 0)). A grid of points was entered into the computer by marking the location of vertices in the image. Each of these points was assigned coordinates (x, y) to form a Cartesian plane. The location of each of these points in the image, relative to the centre of the FOV, was then measured to generate a set of distorted image coordinates (x', y'). Correction functions were generated by curve fitting between the known (x', y') and distorted (x', y') coordinate pairs using Equation 2 in the Appendix. First to fourth order polynomial correction functions were generated for each bronchoscope to determine which solution was appropriate for bronchoscope lenses. The generation of the correction functions is a feature of the Optimas software accessed through the "calibrate spatial" command.
The previous process, from taking images to generating correction functions, was repeated three times for each bronchoscope and the best set of correction functions (i.e. those that produced the least errors) was used for that bronchoscope. The output of the correction algorithm is in pixels that are corrected for distortion but not calibrated into real units (e.g. mm). To calibrate pixels into real units, such as millimetres, a scale mark, such as an object of known size, is required. A more complete description of the correction algorithms is presented in the Appendix.
Testing the correction using patterns and objects of known size
The accuracy of the distortion correction system was tested using images of graph paper and objects of known size. Images of graph paper were taken with the bronchoscope 5, 10 and 15 mm from the test object. A 2x2 square at the centre and periphery of the FOV was measured and the area of the peripheral square was calculated as a percentage of the central squares (fig. 1a
). For a small peripheral object, the whole object is subject to the same level of distortion, but for a large object, the extent of distortion will change across its length. To investigate the effectiveness of the correction functions for large objects, concentric circles with known diameters, which were generated on images of graph paper using the Optimas circle tool, were compared (fig. 2a
). The error in measured diameter was then regressed against the proportion of the FOV occupied by the circle. To simulate an intraluminal object, a copper disc (3 mm in diameter) was imaged on a sheet of graph paper at the centre and periphery of the FOV using the Pentax bronchoscope (fig. 3
). The discs used were electron microscopy grids with a diameter of 3 mm and the graph paper was used to provide a 1-;mm scale marker.
|
|
|
35 kg) were anaesthetized with ketamine (25 mg·kg1/xylazine (2.2 mg·kg1) i.m. and maintained with i.v. pentobarbital through a femoral vein catheter. After intubation, the animals were ventilated (40% oxygen (O2)) and the chest was opened. Tidal volume and respiratory rate were set to maintain a carbon dioxide tension in arterial blood (Pa,CO2) of 35 mmHg, arterial blood was sampled through a femoral artery catheter and blood gases measured by a blood gas analyser (ABL-30, Radiometer, Brønshøj, Denmark). The vagus was cut and stimulated electrically using platinum electrodes placed on the nerve (18 V, 2 ms). Bronchoscopic images of the right stem bronchus (
45 mm diameter) were obtained using the Pentax bronchoscope system described previously. Steel ball bearings (1.58 mm diameter) were introduced into the bronchial lumen through the bronchoscope channel and used as calibration markers (fig. 4a
|
Statistics
Results are reported as mean±sem. The error produced by the bronchoscope lens was calculated either by comparing identical objects at the periphery and centre or by comparison of the software-calculated size of an object to the known size of an object. When identical squares at the centre and periphery were compared, no scale calibration was made and the measured area of the peripheral square was calculated as a percentage of the central square. For objects of known size (the concentric circles or copper discs), the measured area or diameter was expressed as a percentage of the true value. Positive errors represented underestimation and negative errors represented overestimation of object size. Comparisons between uncorrected and corrected measurements were made using the paired t-;test. Regression analysis of the measurement error versus the diameter as a fraction of the FOV was performed. For animal studies, n is the number of animals, and for other experiments, n is the number of images measured. Frequency response curves were analysed by analysis of variance (ANOVA) with linear contrast.
| Results |
|---|
|
|
|---|
Correction of small peripheral objects
The measured area of a peripheral square before correction was less than that of an identical central square, by 44±0.85% for the Pentax, 36±2.2% for the paediatric Olympus and 42±4.2% for the Olympus BF 2T10 bronchoscopes (n=5 in all cases). After correction with a third order polynomial function, the errors were significantly reduced to 4.08±2.5%, 3.0±1.5% and 2.7±0.97% for the Pentax, paediatric Olympus and Olympus BF 2T10, respectively (p<0.001 for each bronchoscope, n=5 paired t-;test). The extent of distortion was similar in all four quadrants.
The errors in the measured area and perimeter of the peripheral disc (relative to its known size) in figure 3a
were 43% and 22%, respectively, prior to correction. The bronchoscope also distorts the apparent shape of the disc, producing a radial axis 25% shorter than the circumferential axis. After correction for distortion, the errors in the measured area and diameter of the peripheral disc were reduced (to <10% and <5%, respectively) and the circumferential and radial axes of the disc were within 5% of each other. As expected, when imaged at the centre of the FOV, neither the size nor shape of the disc was distorted.
Correction for large central objects
Figure 2
shows the error in measured diameter of concentric circles versus the radius as a percentage of the FOV for the four bronchoscopes (as expected, errors in measured area were approximately twice those in diameter (data not shown)). Without correction for distortion, the percentage error in measurement was dependent on the proportion of the FOV occupied by the circle (linear regression analysis, r=0.97, 0.96, 0.97 and 0.86, and n=7, 6, 7 and 8 for the Pentax, paediatric Olympus, Olympus BF 2T10 and video bronchoscopes, respectively, p<0.01 in all cases, fig. 2b
). After correction, the error was independent of the proportion of FOV occupied (r=0.20, 0.62, 0.54 and 0.10, n as above, p>0. 15 in all cases, fig. 2b
).
Effect of distance to target
Correction functions generated using images taken at 5, 10 and 15 mm were used to correct measurements made on images taken at different distances from the bronchoscope (fig. 5
). Using a correction function generated at any of the three distances reduced the error in measurement, regardless of the distance at which the test image was taken (fig. 5
). Therefore, a correction function generated on a grid at 15 mm could correct images taken at 5, 10 or 15 mm, and would perform similarly for correction functions generated at 5 or 10 mm. Thus, image distortion is independent of distance from the bronchoscope tip.
|
For these experiments, a steel ball bearing was introduced into the airway lumen near the site of interest to provide a calibration scale. As this type of scale marker is not applicable to chronic animal experiments or human use, the viability of other scale markers made of agar and the plug from biopsy brushes was also tested. Figure 6
shows an Evans blue stained agar plug next to a steel ball bearing in an airway. With care, plugs could be positioned end on to the bronchoscope and were clearly visible in the lumen. Agar plugs lasted for
0.5 h with no appreciable change, while plugs from biopsy brush catheters dissolved rapidly and were only useful markers for 2 min at the most.
|
| Discussion |
|---|
|
|
|---|
Given the customized nature of previous numerical distortion correction methods, it is not surprising that they have had little impact on clinical bronchoscopy or on the use of bronchoscopy as a research tool. Most of the previous attempts to quantitate bronchoscopic images have used a series of correction values that are applied to measurements depending on the location and relative size of the measured object in the FOV 13, 21, 22. Such corrections are approximations and are only valid for objects of the same shape, which they were designed to correct. Indeed, the authors' earlier efforts relied on tedious hand measurements and nomograms that were difficult to use 13. In contrast, numerical methods, such as those used in this study, are easy to use once suitable correction functions are generated. These methods correct the location of individual pixels in the image, and because the correction acts on individual pixels, it is not affected by the shape of the object measured. Furthermore, since each pixel in the image has its location uniquely corrected, these functions provide a continuity of correction over the whole FOV.
Some previous studies of GI endoscopy 1618, 23 and one recent bronchoscopic study 24 have used numerical methods and correction algorithms to correct images. However, these studies used unique, customized software, which is not readily available. The requirement for each group to develop their own software has, until now, no doubt limited the use of numerical correction algorithms. The barrel distortion correction algorithms of the "off the shelf" commercial image processing that the authors used, compare well to the custom software developed by others 20. This suggests that such correction could be routinely used for research or possibly clinical applications.
Previous groups that used numerical methods to correct for lens distortion, used polar coordinates in their correction algorithms. While polar coordinates simplify the correction function mathematics, most current computer technology operates in Cartesian coordinates, necessitating conversion from Cartesian to polar coordinates for the correction and then back again. To avoid this, the present authors used Cartesian coordinates for the correction algorithms.
One approach to image correction is to display a corrected image and take measurements of this undistorted image. While displaying a corrected image is qualitatively useful, it has several attendant technical difficulties (see 17 and Appendix) and is not necessary for quantitative measurement. Rather than correcting and displaying the whole image, the present approach was to correct only the location of the pixels of interest for measurement calculations. This allows accurate measurement without altering the image display, and, to the authors' knowledge, this approach has not been used previously for endoscopy or bronchoscopy.
The primary feature of bronchoscopic distortion is that length, perimeter and area measurements are dependent on location in the FOV. As the distortion is a function of the wide-angle lens, it is independent of the distance to the target. The same correction functions can be used regardless of the distance to the target 15, 20. Previous bronchoscopic studies have generated separate correction functions for each distance studied 24, which is unnecessary and increases the work needed to implement the procedure. The absolute area or perimeter of an object is dependent on distance, but this is a problem of calibration, not of distortion correction. Once the distortion is corrected, other procedures, such as markers placed in the lumen, can be used to calibrate the measurements 13, 19, 20, 2426. Unlike distortion correction, calibration is distance sensitive and it is important that any calibration marker used is the same distance from the bronchoscope as the measured object.
In the animal experiments, steel ball bearings were used as calibration markers. These are convenient for acute animal experiments as they are inexpensive, readily available, of uniform size and spherical, making their orientation in the bronchus unimportant. However, an alternative calibration system is required for human studies or chronic animal experiments. The authors have found that the plugs from clinically used protective brush catheters make useful calibration markers that last for 12 min in the bronchus before dissolving. The protected brush technique is used to take brushings from the bronchial lumen for microbiology and employs a plug at the end of the catheter to ensure sterility of the brush. In human subjects, these plugs are routinely left inside the bronchus when a sample is taken. Plugs were also manufactured from agar dissolved in saline. Unfortunately, both types of plug are cylindrical and not spherical, so care must be taken in orientating them correctly. Biopsy forceps can also be used as a calibration marker, although they can obscure the view of the airway.
Bronchoscopic distortion also affects the shape of objects. The radial nature of the distortion compresses objects along their radial axis, making circles appear elliptical and straight lines appear curved (see figs. 13![]()
![]()
). This affects quantitative estimates of shape, such as the ratio of an object's longest to shortest axis. Although the correction approach employed in the present study will not improve the qualitative, on-screen appearance of an object, it will correct shape-based quantitative measurements, such as long to short axis ratios. Furthermore, calibration is not required as the measurement is a ratio.
Distortion correction relies on generating a series of polynomial correction functions, as described in the Appendix. These correction functions are then applied to each pixel to calculate the location of that pixel in an undistorted image. The order of polynomial functions (linear, quadratic, cubic, etc.) appropriate for the correction was experimentally determined. Third order polynomials were required for all bronchoscopes and were sufficient to correct distortion. Asari and coworkers 16, 17 also found that third order polynomials are required to correct images from GI endoscopes using polar coordinates, whereas Kouwenhoven et al. 18 found that a second order function was sufficient (also in polar coordinates). A second order fit was insufficient for the examined bronchoscopes. Although each bronchoscope must have a separate correction function, once obtained, the correction for a given bronchoscope remains constant.
Any errors present in the correction functions will be reflected in the measurements obtained using them. The data used to generate the correction functions are the measured location and assigned coordinates of a set of reference points. If the bronchoscope views the reference grid at an angle, the spacing between reference points will not be uniform and the correction functions will be skewed 17. Thus, great care must be taken in imaging the reference grid, marking reference points and generating the correction functions.
Quantifying bronchoscopic images provides a useful research tool, enabling the assessment of individual airway dynamics in vivo 13. Current methods for measuring airway narrowing in vivo require expensive radiological techniques, such as tantalum bronchograms or HRCT 612. Methods assessing narrowing using airways resistance cannot easily be used to study individual bronchi, as they depend on the change in resistance throughout the bronchial tree, the site of which is not generally known. The ability to quantitate the narrowing of porcine bronchi during vagal stimulation using the bronchoscopic technique, thus, provides proof of the quantitative use of this technique.
Quantification of bronchoscopic images may also have significance for clinical bronchoscopy. Currently, it is difficult to accurately estimate the size of lesions, tumours, etc. or the degree of stenosis 15, 1921, 25. Accurate measurements of tumour size or other lesions may prove useful for treatment decisions and prognostication. Currently, the presented technique cannot be used in humans because of the lack of a suitable calibration. Biopsy forceps or the plug from a bronchoscopy biopsy brush are alternative scale markers that could be used in humans.
In conclusion, polynomial correction functions can be used to correct for bronchoscope lens distortion and the steps required to use commercially available software to generate and use these corrections have been described. Third order polynomial correction functions were required to give adequate correction for the bronchoscopes tested. Although different bronchoscopes require their own set of correction functions, once generated, the correction functions are always valid for a given bronchoscope. The correction is independent of distance in that measurements are independent of location in the field of view. Suitable correction functions and software can overcome the problem of distortion from the wide-angle lens used on bronchoscopes, providing a new method for investigating narrowing of individual bronchi.
|
Several numerical methods have been devised to correct for distortion in endoscopic images of GI features or heart valves 1618, 20, 23. These methods employ a correction function to calculate the real location of a pixel from the apparent location within an image. Typically, these correction functions are found by imaging a grid of regularly spaced points and measuring the deviation of the image points from their known location.
For a simplified example of the correction procedure, consider a hypothetical distortion, which only affects the x-;axis, a hypothetical number line (L), with evenly spaced points, and the distorted image (L'). L', contains unevenly spaced points and the distance between points decreases in a nonlinear manner from the centre to the edge of the FOV (fig. 7a
).
If x is the true distance (in the arbitrary units of a Cartesian plane) of a point from the origin on L (i.e. the x-;axis coordinate of the point) then x' is the apparent distance (once again in arbitrary units) of the same point from the origin in the image L'. The plot of x versus x' provides a graphical solution to find the true location, x, of any point on L, from its measured location, x', in the image L'. As the distortion produced by a bronchoscope is nonlinear, the relationship between x and x' is nonlinear, but an equation for the correction can be found by fitting a polynomial equation to the data 16, 23. Repeating the previous procedure along a line perpendicular to L will generate an equation to correct distortion along the y/y'-axis. For a two-dimensional Cartesian plane, an x, y grid of points with known locations can be used to derive correction functions for both the x/x' and y/y' axes of the image. Thus:
|
| (001) |
Equations of this form can correct images where the distortion of the x-; and y-;axis do not interact, i.e. the distortion of the x coordinate is unaffected by the y coordinate and vice versa. This is not the case for the radial distortion of bronchoscopes. Complex distortions, where the distortion of one coordinate depends on the other, can be corrected by replacing the constants a0, a1, a2, a3 ... an and b0, b1, b2, b3 ... bn in Equation 1
with functions of the other coordinate. So, the correction applied to the x-;axis (and similarly to the y-;axis) location of a pixel is dependent on both the x' and y' locations in the image. In this case, multifactorial regression is used to fit x to x' and y' and y to x', y'. Thus:
|
| (002) |
|
| (005) |
|
| (006) |
Each correction function will then have 2(n+1)2 coefficients that are found by curve fitting. Correction functions in the form of Equation 2 can compensate for many different forms of distortion, including situations where there are nonsymmetrical or local distortions, for example, if the distortion of the top left quadrant were different to the bottom left due to a defect in the bronchoscope lens.
Endoscopes and bronchoscopes normally produce a uniform, symmetrical and purely radial distortion 16, 23. Polar coordinates could be used to take advantage of this symmetry about the centre of the FOV 1618, 24. Bronchoscopic distortion is purely radial so only the distance from the centre, not the angle, is altered in the image 18. Using polar coordinates reduces the complexity of the correction functions used to compensate for bronchoscope image distortion by limiting the number of coefficients to n+1. However, all common image file formats, frame capture cards and computer displays function in Cartesian coordinates, necessitating conversion of the image from Cartesian to polar coordinates and back again after the distortion has been corrected. All previous studies that used numerical distortion correction, used algorithms based on polar coordinates and developed their own custom software to do so 1618, 23, 24.
Once a set of correction functions is generated, they can be applied to the location of any pixel in the image. In principle, the location of every pixel in the image could be corrected and a new undistorted image displayed 1618, 23. However, displaying the corrected image has several technical difficulties. First, to display the corrected image, either the periphery of the field of view must be expanded or the centre compressed. Secondly, computer displays operate using integer values for pixel locations. After correction, the calculated locations of many pixels will not be integers, hence a display cannot be generated from the corrected pixel locations without further image processing. Measuring an object within an image only requires that those pixels defining the object be corrected, reducing the number of calculations needed, and avoiding all of the difficulties of generating a new display from the corrected pixel locations. Once a feature is identified, the image analysis system applies the correction functions to the pixels defining that feature and then provides measurements based on the corrected pixel locations.
| Acknowledgements |
|---|
|
|
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
N. Jowett, R. A. Weersink, K. Zhang, P. Campisi, and V. Forte Airway Luminal Diameter and Shape Measurement by Means of an Intraluminal Fiberoptic Probe: A Bench Model Arch Otolaryngol Head Neck Surg, June 1, 2008; 134(6): 637 - 642. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. B. Noble, A. Sharma, P. K. McFawn, and H. W. Mitchell Airway narrowing in porcine bronchi with and without lung parenchyma Eur. Respir. J., November 1, 2005; 26(5): 804 - 811. [Abstract] [Full Text] [PDF] |
||||
![]() |
I B Masters, M M Eastburn, R Wootton, R S Ware, P W Francis, P V Zimmerman, and A B Chang A new method for objective identification and measurement of airway lumen in paediatric flexible videobronchoscopy Thorax, August 1, 2005; 60(8): 652 - 658. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |