Abstract
Bronchoscopy is a commonly used clinical tool that provides a direct image of the bronchial lumen. However, bronchoscopy has seen little use as a quantitative measurement tool, mainly because of the wide-angle lens which distorts the image. The present authors have tested the ability of numerical algorithms and commercial software to correct for this distortion.
Test objects of known size were imaged with four different bronchoscopes. Commercial image analysis software was used to measure the size of features in the images before and after applying distortion correction algorithms. The technique was then applied by measuring airway narrowing in anaesthetized pigs during vagal stimulation.
Without correction, object size near the edge of the field of view is underestimated by ∼40%. The error in measured diameter of concentric circles was dependent on the radius of the circle, increasing to 25% for circles occupying 90% of the field. Third order polynomial functions were required to correct these errors. After correction, errors were independent of object size or location in the image. Correction for lens distortion was independent of the distance between bronchoscope and object.
The authors conclude that modern image processing software can correct for the distortion produced by wide-angle bronchoscope lenses.
This work was supported by the Canadian Institutes of Health Research and the Ontario Thoracic Society. P.K. McFawn was supported by the John Alexander Stewart Fellowship and a Canadian Lung Association/CIHR/GlaxoWelcome Fellowship.
Bronchoscopy is a commonly used and relatively inexpensive clinical tool. Frequent diagnostic uses of bronchoscopy include dimensional evaluation of airways, masses, tracheal stenosis or foreign bodies, as well as assessment of the effectiveness of therapeutic interventions, such as radiotherapy of endobronchial cancer, etc. 1–4. Until now, these evaluations have been qualitative, with quantitative assessment limited by the optical distortion produced by the bronchoscope lens. The ability to measure endobronchial objects or airway dimensions accurately would move bronchoscopy from a qualitative to a quantitative research and clinical tool.
Quantification of bronchoscopic images has the potential to be a valuable tool for both research and clinical investigation. Traditional methods of measuring airway responsiveness using lung or airway resistance provide information about the tracheobronchial tree as a whole. However, these measurements are a composite of many different airways in the bronchial tree and do not allow for the observation of the response of an individual bronchi 5–9. Previously, quantitative studies of airway narrowing in vivo have relied on tantalum bronchography or high-resolution computed tomography (HRCT), both of which are prohibitively expensive or unavailable for most laboratories 6–12. In contrast, bronchoscopy is a relatively inexpensive and routine procedure that allows direct, real-time imaging of the bronchial lumen. Quantification of bronchoscopic images would provide a useful alternative to tantalum bronchography or HRCT for measuring the responsiveness of individual airways in vivo 13.
To provide an adequate field of view (FOV), bronchoscopes are fitted with a wide-angle “fish-eye” lens, which distorts the image, making objects at the periphery of the image appear smaller than those at the centre. This form of distortion has been termed “barrel distortion” 14, 15. Bronchoscopic distortion is radial, increasing with displacement from the centre of the FOV in a nonlinear manner 13, 15–18. Bronchoscopic distortion also alters the apparent shape of objects, making straight lines appear curved and compressing objects radially so that a circle will appear elliptical near the periphery of the field. The extent of distortion depends on how close the object is located to or extends to the edge of the field 13. Without correction, only small objects near the centre can be measured accurately 13, 15, 19, 20.
Most previous studies that measured dimensions using bronchoscopy created correction factors that could be applied to a measured area or diameter based on the location and size of the measured object in the FOV 13, 21, 22. Such attempts are relatively crude approximations, and require several different correction factors to deal separately with the varying effects of location and size. Because the distortion of bronchoscopic images is regular and uniform, numerical methods should be able to correct for the distortion by reversing the distortion mathematically to allow accurate measurements. While little work on correcting bronchoscopic images has been published, several studies have investigated other applications of endoscopic measurement 16–18, 20, 23. They used gastric endoscopes, which are much larger than bronchoscopes, with correspondingly larger lenses and potentially different optics and extent of distortion. These endoscopic studies used polar coordinates because they simplify the correction methods. However, polar coordinates also complicate image handling, as common file formats, computer displays, image capture cards, etc. use Cartesian coordinates. In a recent study, numerical methods were applied to bronchoscopic image distortion, but the algorithms used for the correction were not described 24. Previous applications of numerical methods to correct either endoscopic or bronchoscopic images for lens distortion have used software designed by the authors that is unavailable to other laboratories 16–18, 21, 22. These factors have restricted the use of quantitative bronchoscopy and endoscopy.
Recent developments in imaging hardware and software have made powerful distortion correction techniques available in inexpensive commercial image analysis systems. The present study investigated the ability of distortion correction algorithms to compensate for distortion and make quantitative measurements from bronchoscopic images. Firstly, the order of polynomial function required to correct for distortion was determined. The accuracy of the correction procedures for small peripheral versus small central objects and for central objects of different sizes were then investigated. Finally, the authors applied the technique to measure airway narrowing during vagal stimulation.
Methods
Generation of correction functions
Three flexible fibre-optic bronchoscopes (Pentax FH-15H (Pentax, Tokyo, Japan), Olympus BF 2T10 (Olympus, Tokyo, Japan) and Olympus BF 3C20 paediatric bronchoscope) connected to a KR202 CCD (Panasonic, Secaucus, NJ, USA) camera, and a digital bronchoscope (Olympus BF P200) connected to an Olympus WM-30 workstation were tested. Images were digitized with a Flash Point FPG 2.70 frame capture card (Integral Technologies, Silver Spring, MD, USA), and Optimas 6.1 (Media Cybernetics, Indianapolis, IN, USA) was used for image processing and the “spatial distortion” command of this program was used for distortion correction.
For each bronchoscope, correction functions were established by acquiring images of graph paper with 1 mm line spacing placed 5, 10 and 15 mm from the bronchoscope tip. A micromanipulator and swivel were used to ensure that the graph paper was imaged square-on and aligned with a vertex in the centre of the FOV. For all measurements, the centre of the FOV was found and used as the reference origin (i.e. the point (0, 0)). A grid of points was entered into the computer by marking the location of vertices in the image. Each of these points was assigned coordinates (x, y) to form a Cartesian plane. The location of each of these points in the image, relative to the centre of the FOV, was then measured to generate a set of distorted image coordinates (x′, y′). Correction functions were generated by curve fitting between the known (x′, y′) and distorted (x′, y′) coordinate pairs using Equation 2 in the Appendix. First to fourth order polynomial correction functions were generated for each bronchoscope to determine which solution was appropriate for bronchoscope lenses. The generation of the correction functions is a feature of the Optimas software accessed through the “calibrate spatial” command.
The previous process, from taking images to generating correction functions, was repeated three times for each bronchoscope and the best set of correction functions (i.e. those that produced the least errors) was used for that bronchoscope. The output of the correction algorithm is in pixels that are corrected for distortion but not calibrated into real units (e.g. mm). To calibrate pixels into real units, such as millimetres, a scale mark, such as an object of known size, is required. A more complete description of the correction algorithms is presented in the Appendix.
Testing the correction using patterns and objects of known size
The accuracy of the distortion correction system was tested using images of graph paper and objects of known size. Images of graph paper were taken with the bronchoscope 5, 10 and 15 mm from the test object. A 2×2 square at the centre and periphery of the FOV was measured and the area of the peripheral square was calculated as a percentage of the central squares (fig. 1a⇓). For a small peripheral object, the whole object is subject to the same level of distortion, but for a large object, the extent of distortion will change across its length. To investigate the effectiveness of the correction functions for large objects, concentric circles with known diameters, which were generated on images of graph paper using the Optimas circle tool, were compared (fig. 2a⇓). The error in measured diameter was then regressed against the proportion of the FOV occupied by the circle. To simulate an intraluminal object, a copper disc (3 mm in diameter) was imaged on a sheet of graph paper at the centre and periphery of the FOV using the Pentax bronchoscope (fig. 3⇓). The discs used were electron microscopy grids with a diameter of 3 mm and the graph paper was used to provide a 1-;mm scale marker.
Application of image correction to the measurement of airway narrowing
All animal experimental procedures conformed to the guidelines of the Canadian Council on Animal Care and were approved by the Queen's University Animal Care Committee. Pigs (n=8, ∼35 kg) were anaesthetized with ketamine (25 mg·kg−1/xylazine (2.2 mg·kg−1) i.m. and maintained with i.v. pentobarbital through a femoral vein catheter. After intubation, the animals were ventilated (40% oxygen (O2)) and the chest was opened. Tidal volume and respiratory rate were set to maintain a carbon dioxide tension in arterial blood (Pa,CO2) of 35 mmHg, arterial blood was sampled through a femoral artery catheter and blood gases measured by a blood gas analyser (ABL-30, Radiometer, Brønshøj, Denmark). The vagus was cut and stimulated electrically using platinum electrodes placed on the nerve (18 V, 2 ms). Bronchoscopic images of the right stem bronchus (∼4–5 mm diameter) were obtained using the Pentax bronchoscope system described previously. Steel ball bearings (1.58 mm diameter) were introduced into the bronchial lumen through the bronchoscope channel and used as calibration markers (fig. 4a⇓). Vagal stimulation was performed while lung volume was held at a static transpulmonary pressure of 5 cmH2O (1 min maximum duration). Frequency response curves were generated by stimulating at 5, 10 and 20 Hz in a random order.
In some of the studies, the protective plug from a biopsy brush (type 1650 microbiology specimen brush, Microvasive, Natick, MA, USA) was used as a calibration marker. The present authors also manufactured plugs from agar (2% agar in normal saline), dyed with Evans blue to improve contrast. The agar solution was autoclaved and poured into petri dishes. Plugs were punched out by pressing a catheter (from a used Microvasive kit) into the agar. The diameter and length of these plugs were measured using calipers.
Statistics
Results are reported as mean±sem. The error produced by the bronchoscope lens was calculated either by comparing identical objects at the periphery and centre or by comparison of the software-calculated size of an object to the known size of an object. When identical squares at the centre and periphery were compared, no scale calibration was made and the measured area of the peripheral square was calculated as a percentage of the central square. For objects of known size (the concentric circles or copper discs), the measured area or diameter was expressed as a percentage of the true value. Positive errors represented underestimation and negative errors represented overestimation of object size. Comparisons between uncorrected and corrected measurements were made using the paired t-;test. Regression analysis of the measurement error versus the diameter as a fraction of the FOV was performed. For animal studies, n is the number of animals, and for other experiments, n is the number of images measured. Frequency response curves were analysed by analysis of variance (ANOVA) with linear contrast.
Results
Polynomial correction function order
For each bronchoscope, the area of a peripheral test square was compared to a central square. Without correction, the measured area of the peripheral square was considerably less (35–43%) than the central square for all bronchoscopes (fig. 1⇓). First and second order correction functions produced little improvement, while a third order fit significantly reduced the error in the measured area of the peripheral square (fig. 1⇓). A fourth order fit produced no further improvement and third order correction functions were used for all subsequent analysis.
Correction of small peripheral objects
The measured area of a peripheral square before correction was less than that of an identical central square, by 44±0.85% for the Pentax, 36±2.2% for the paediatric Olympus and 42±4.2% for the Olympus BF 2T10 bronchoscopes (n=5 in all cases). After correction with a third order polynomial function, the errors were significantly reduced to 4.08±2.5%, 3.0±1.5% and 2.7±0.97% for the Pentax, paediatric Olympus and Olympus BF 2T10, respectively (p<0.001 for each bronchoscope, n=5 paired t-;test). The extent of distortion was similar in all four quadrants.
The errors in the measured area and perimeter of the peripheral disc (relative to its known size) in figure 3a⇓ were 43% and 22%, respectively, prior to correction. The bronchoscope also distorts the apparent shape of the disc, producing a radial axis 25% shorter than the circumferential axis. After correction for distortion, the errors in the measured area and diameter of the peripheral disc were reduced (to <10% and <5%, respectively) and the circumferential and radial axes of the disc were within 5% of each other. As expected, when imaged at the centre of the FOV, neither the size nor shape of the disc was distorted.
Correction for large central objects
Figure 2⇓ shows the error in measured diameter of concentric circles versus the radius as a percentage of the FOV for the four bronchoscopes (as expected, errors in measured area were approximately twice those in diameter (data not shown)). Without correction for distortion, the percentage error in measurement was dependent on the proportion of the FOV occupied by the circle (linear regression analysis, r=0.97, 0.96, 0.97 and 0.86, and n=7, 6, 7 and 8 for the Pentax, paediatric Olympus, Olympus BF 2T10 and video bronchoscopes, respectively, p<0.01 in all cases, fig. 2b⇓). After correction, the error was independent of the proportion of FOV occupied (r=0.20, 0.62, 0.54 and 0.10, n as above, p>0. 15 in all cases, fig. 2b⇓).
Effect of distance to target
Correction functions generated using images taken at 5, 10 and 15 mm were used to correct measurements made on images taken at different distances from the bronchoscope (fig. 5⇓). Using a correction function generated at any of the three distances reduced the error in measurement, regardless of the distance at which the test image was taken (fig. 5⇓). Therefore, a correction function generated on a grid at 15 mm could correct images taken at 5, 10 or 15 mm, and would perform similarly for correction functions generated at 5 or 10 mm. Thus, image distortion is independent of distance from the bronchoscope tip.
Application of the technique to the measurement of airway narrowing
A limited number of experiments were performed with pigs to demonstrate quantitative measurement using bronchoscopic images (fig. 4⇓). Vagal stimulation evoked narrowing of the bronchi and the production of mucus. During stimulation at 20 Hz, the bronchi narrowed by 34±3.1%. Narrowing was frequency dependent (p<0.01 ANOVA, p<0.01 linear contrast, n=8, fig. 4b⇓).
For these experiments, a steel ball bearing was introduced into the airway lumen near the site of interest to provide a calibration scale. As this type of scale marker is not applicable to chronic animal experiments or human use, the viability of other scale markers made of agar and the plug from biopsy brushes was also tested. Figure 6⇓ shows an Evans blue stained agar plug next to a steel ball bearing in an airway. With care, plugs could be positioned end on to the bronchoscope and were clearly visible in the lumen. Agar plugs lasted for ≥0.5 h with no appreciable change, while plugs from biopsy brush catheters dissolved rapidly and were only useful markers for 2 min at the most.
Discussion
The present study clearly demonstrates that the distortion correction algorithms of commercially available software can be used to obtain quantitative measurements from bronchoscopic images. The present study is the first to test and validate the capabilities of commercial image analysis software on bronchoscopy, and, therefore, to introduce the concept of a widely available distortion correction system for experimental or clinical bronchoscopy. This is a significant advance from previous studies (mostly of larger gastrointestinal (GI) endoscopes with different optics) that employed proprietary methodology, or the authors' own previous attempts using cumbersome nomograms 13.
Given the customized nature of previous numerical distortion correction methods, it is not surprising that they have had little impact on clinical bronchoscopy or on the use of bronchoscopy as a research tool. Most of the previous attempts to quantitate bronchoscopic images have used a series of correction values that are applied to measurements depending on the location and relative size of the measured object in the FOV 13, 21, 22. Such corrections are approximations and are only valid for objects of the same shape, which they were designed to correct. Indeed, the authors' earlier efforts relied on tedious hand measurements and nomograms that were difficult to use 13. In contrast, numerical methods, such as those used in this study, are easy to use once suitable correction functions are generated. These methods correct the location of individual pixels in the image, and because the correction acts on individual pixels, it is not affected by the shape of the object measured. Furthermore, since each pixel in the image has its location uniquely corrected, these functions provide a continuity of correction over the whole FOV.
Some previous studies of GI endoscopy 16–18, 23 and one recent bronchoscopic study 24 have used numerical methods and correction algorithms to correct images. However, these studies used unique, customized software, which is not readily available. The requirement for each group to develop their own software has, until now, no doubt limited the use of numerical correction algorithms. The barrel distortion correction algorithms of the “off the shelf” commercial image processing that the authors used, compare well to the custom software developed by others 20. This suggests that such correction could be routinely used for research or possibly clinical applications.
Previous groups that used numerical methods to correct for lens distortion, used polar coordinates in their correction algorithms. While polar coordinates simplify the correction function mathematics, most current computer technology operates in Cartesian coordinates, necessitating conversion from Cartesian to polar coordinates for the correction and then back again. To avoid this, the present authors used Cartesian coordinates for the correction algorithms.
One approach to image correction is to display a corrected image and take measurements of this undistorted image. While displaying a corrected image is qualitatively useful, it has several attendant technical difficulties (see 17 and Appendix) and is not necessary for quantitative measurement. Rather than correcting and displaying the whole image, the present approach was to correct only the location of the pixels of interest for measurement calculations. This allows accurate measurement without altering the image display, and, to the authors' knowledge, this approach has not been used previously for endoscopy or bronchoscopy.
The primary feature of bronchoscopic distortion is that length, perimeter and area measurements are dependent on location in the FOV. As the distortion is a function of the wide-angle lens, it is independent of the distance to the target. The same correction functions can be used regardless of the distance to the target 15, 20. Previous bronchoscopic studies have generated separate correction functions for each distance studied 24, which is unnecessary and increases the work needed to implement the procedure. The absolute area or perimeter of an object is dependent on distance, but this is a problem of calibration, not of distortion correction. Once the distortion is corrected, other procedures, such as markers placed in the lumen, can be used to calibrate the measurements 13, 19, 20, 24–26. Unlike distortion correction, calibration is distance sensitive and it is important that any calibration marker used is the same distance from the bronchoscope as the measured object.
In the animal experiments, steel ball bearings were used as calibration markers. These are convenient for acute animal experiments as they are inexpensive, readily available, of uniform size and spherical, making their orientation in the bronchus unimportant. However, an alternative calibration system is required for human studies or chronic animal experiments. The authors have found that the plugs from clinically used protective brush catheters make useful calibration markers that last for 1–2 min in the bronchus before dissolving. The protected brush technique is used to take brushings from the bronchial lumen for microbiology and employs a plug at the end of the catheter to ensure sterility of the brush. In human subjects, these plugs are routinely left inside the bronchus when a sample is taken. Plugs were also manufactured from agar dissolved in saline. Unfortunately, both types of plug are cylindrical and not spherical, so care must be taken in orientating them correctly. Biopsy forceps can also be used as a calibration marker, although they can obscure the view of the airway.
Bronchoscopic distortion also affects the shape of objects. The radial nature of the distortion compresses objects along their radial axis, making circles appear elliptical and straight lines appear curved (see figs. 1–3⇓⇓⇓). This affects quantitative estimates of shape, such as the ratio of an object's longest to shortest axis. Although the correction approach employed in the present study will not improve the qualitative, on-screen appearance of an object, it will correct shape-based quantitative measurements, such as long to short axis ratios. Furthermore, calibration is not required as the measurement is a ratio.
Distortion correction relies on generating a series of polynomial correction functions, as described in the Appendix. These correction functions are then applied to each pixel to calculate the location of that pixel in an undistorted image. The order of polynomial functions (linear, quadratic, cubic, etc.) appropriate for the correction was experimentally determined. Third order polynomials were required for all bronchoscopes and were sufficient to correct distortion. Asari and coworkers 16, 17 also found that third order polynomials are required to correct images from GI endoscopes using polar coordinates, whereas Kouwenhoven et al. 18 found that a second order function was sufficient (also in polar coordinates). A second order fit was insufficient for the examined bronchoscopes. Although each bronchoscope must have a separate correction function, once obtained, the correction for a given bronchoscope remains constant.
Any errors present in the correction functions will be reflected in the measurements obtained using them. The data used to generate the correction functions are the measured location and assigned coordinates of a set of reference points. If the bronchoscope views the reference grid at an angle, the spacing between reference points will not be uniform and the correction functions will be skewed 17. Thus, great care must be taken in imaging the reference grid, marking reference points and generating the correction functions.
Quantifying bronchoscopic images provides a useful research tool, enabling the assessment of individual airway dynamics in vivo 13. Current methods for measuring airway narrowing in vivo require expensive radiological techniques, such as tantalum bronchograms or HRCT 6–12. Methods assessing narrowing using airways resistance cannot easily be used to study individual bronchi, as they depend on the change in resistance throughout the bronchial tree, the site of which is not generally known. The ability to quantitate the narrowing of porcine bronchi during vagal stimulation using the bronchoscopic technique, thus, provides proof of the quantitative use of this technique.
Quantification of bronchoscopic images may also have significance for clinical bronchoscopy. Currently, it is difficult to accurately estimate the size of lesions, tumours, etc. or the degree of stenosis 15, 19–21, 25. Accurate measurements of tumour size or other lesions may prove useful for treatment decisions and prognostication. Currently, the presented technique cannot be used in humans because of the lack of a suitable calibration. Biopsy forceps or the plug from a bronchoscopy biopsy brush are alternative scale markers that could be used in humans.
In conclusion, polynomial correction functions can be used to correct for bronchoscope lens distortion and the steps required to use commercially available software to generate and use these corrections have been described. Third order polynomial correction functions were required to give adequate correction for the bronchoscopes tested. Although different bronchoscopes require their own set of correction functions, once generated, the correction functions are always valid for a given bronchoscope. The correction is independent of distance in that measurements are independent of location in the field of view. Suitable correction functions and software can overcome the problem of distortion from the wide-angle lens used on bronchoscopes, providing a new method for investigating narrowing of individual bronchi.
Impact of polynomial order on error correction. a) Typical image of a graph paper grid (1 mm line spacing) with a central square and a peripheral square in each quadrant marked on the image (from a Pentax bronchoscope 5 mm from the paper). b) To determine the order of the correction function required, the percentage error in measured area of a peripheral square (((area central-area peripheral)/area central)×100) was determined before and after correction with different order polynomial functions. For the three bronchoscopes, first to fourth order polynomial correction functions were generated and applied to the measurement of a peripheral and central square. Without correction, the measured area of the peripheral square was less than the central square. Little improvement was produced when first or second order polynomials were used to generate the correction functions. Third order polynomials reduced the difference in area to <10% and in perimeter to <5% (not shown). Fourth order equations produced no further improvement. 
: Pentax; □: Olympus Paediatric; ┘: Olympus; └: Digital.
Accuracy of the correction functions for large versus small objects. a) Concentric circles were superimposed on grid (1 mm line spacing) images using the Optimas circle tool to provide circles of known radii. In this typical example, circles with radii of 1, 2, 3 and 4 grid units (corresponding to 28.7, 55.1, 77.2 and 95.6% of the field of view (FOV)) are shown. b) Percentage error in diameter for concentric circles plotted versus the radius of the circle as a percentage of the FOV. Errors with and without correction are shown for the three bronchoscopes and with images taken at 5 or 10 mm distance (negative errors indicate an over estimation of diameter). Each line connects measurements made under the same conditions (bronchoscope, distance, correction) on circles of increasing size. Closed and open symbols represent images taken 5 and 10 mm from the bronchoscope, respectively. Percentage errors increase with the size of the circle relative to the full FOV and the errors are similar for all bronchoscopes and both distances. After correction, the error in diameter is independent of radius and generally <5%. ——: uncorrected values; - - - -: corrected values measured on the same images; ⋄, ♦: Pentax; ▵, ▴: Olympus Paediatric; ○, •: Olympus; □, ▪: Olympus Video.
Effect of correction on central versus peripheral objects. a) Copper discs (3 mm diameter) were placed on graph paper (1 mm line spacing) at either the centre or periphery of the field of view to simulate an object observed during bronchoscopy. This example image was taken 10 mm from the bronchoscope tip. Note the distorted shape of the peripheral disc where the more central half of the disc appears much larger than the more peripheral region and the disc appears compressed radially. The central disc also appears much larger although both discs are identical. b) Errors in measured area and diameter for the discs shown in a) (using the graph paper as a calibration marker). The correction functions markedly reduced measurement error for a peripherally placed disc. □: centre; ┘: periphery.
Bronchoscopic measurement of airway narrowing. a) Bronchoscopic images of pig bronchi immediately before vagal stimulation. The metallic ball in the image is a calibration marker (1.58 mm) and the ball is sitting in a small side branch b) The same bronchi after stimulation of the cut vagus (20 Hz, at 5 cmH2O Ptp). The time between this image and the image in a) is 30 s. The calibration marker is still sitting in the small side branch and has not moved relative to the airway wall. The calibration marker is nearly the same size as in a), indicating that the bronchoscope is approximately the same distance from the marker as in a). Narrowing of the airway has moved the marker closer to the centre of the field of view. c) Narrowing of airways during vagal stimulation measured using bronchoscopy. The apparent size of the calibration marker was used to calibrate the images, correcting for changes in distance from the bronchoscope. Airway diameter decreased with stimulus frequencies (p<0.01 analysis of variance, p<0.01 linear contrast, n=8).
Effect of distance between the bronchoscope and target object on correction. Images of graph paper were taken at 5, 10 and 15 mm and correction functions generated for each of these distances. The relative area of a peripheral to central square was then measured in the three images, without correction and using each of the correction functions (generated at different distances) in turn. Without correction, peripheral squares were underestimated at all distances and correction with any of the correction functions improved the error. That is, even if the correction function was generated from a grid image taken at a different distance, it was still effective. Correction functions generated using grid images taken at different distances could be used interchangeably to correct for lens distortion. Also, the correction functions that gave the lowest error for an image were not necessarily generated at the same distance to the target. Distance does affect the scaling of the image after correction and the measurement in real units, but these issues are separate from distortion. □: no correction; 
: 5 mm correction; └: 10 mm correction; ┘: 15 mm correction.
Alternative calibration markers. a) Image showing the steel ball (1.5 mm) and agar plug calibration markers on a graph paper background (1 mm line spacing). The agar plug was a short cylinder (1.5 mm long, 1.4 mm in diameter), viewed end on that has been dyed with Evans blue. b) A large central bronchus with both a steel ball (1.5 mm) and an agar plug (1.6 mm diameter) placed in the lumen as calibration markers. The plug was a cylinder with its long axis running parallel to the airway and viewed end on by the bronchoscope.
The distortion of a number line by hypothetical bronchoscope-;like distortion. a) The hypothetical number line L and its distorted image L′. b) Hypothetical plot of the location of points on the distorted number line L′ versus their location on the undistorted line L. An equation for this plot could be found by polynomial curve fitting and used as a correction function to calculate the true location, x, of a point on L from the measured location, x′.
Appendix: background theory for the correction method
Several numerical methods have been devised to correct for distortion in endoscopic images of GI features or heart valves 16–18, 20, 23. These methods employ a correction function to calculate the real location of a pixel from the apparent location within an image. Typically, these correction functions are found by imaging a grid of regularly spaced points and measuring the deviation of the image points from their known location.
For a simplified example of the correction procedure, consider a hypothetical distortion, which only affects the x-;axis, a hypothetical number line (L), with evenly spaced points, and the distorted image (L′). L′, contains unevenly spaced points and the distance between points decreases in a nonlinear manner from the centre to the edge of the FOV (fig. 7a⇑).
If x is the true distance (in the arbitrary units of a Cartesian plane) of a point from the origin on L (i.e. the x-;axis coordinate of the point) then x′ is the apparent distance (once again in arbitrary units) of the same point from the origin in the image L′. The plot of x versus x′ provides a graphical solution to find the true location, x, of any point on L, from its measured location, x′, in the image L′. As the distortion produced by a bronchoscope is nonlinear, the relationship between x and x′ is nonlinear, but an equation for the correction can be found by fitting a polynomial equation to the data 16, 23. Repeating the previous procedure along a line perpendicular to L will generate an equation to correct distortion along the y/y′-axis. For a two-dimensional Cartesian plane, an x, y grid of points with known locations can be used to derive correction functions for both the x/x′ and y/y′ axes of the image. Thus: 
where a0, a1, a2, a3 … an are constants found by fitting x to x′, b0, b1, b2, b3 … bn are constants found by fitting y to y′, and n is the order of the polynomial equations.
Equations of this form can correct images where the distortion of the x-; and y-;axis do not interact, i.e. the distortion of the x coordinate is unaffected by the y coordinate and vice versa. This is not the case for the radial distortion of bronchoscopes. Complex distortions, where the distortion of one coordinate depends on the other, can be corrected by replacing the constants a0, a1, a2, a3 … an and b0, b1, b2, b3 … bn in Equation 1 with functions of the other coordinate. So, the correction applied to the x-;axis (and similarly to the y-;axis) location of a pixel is dependent on both the x′ and y′ locations in the image. In this case, multifactorial regression is used to fit x to x′ and y′ and y to x′, y′. Thus: 
where 
and 
where a00, a01, a02, … a0n, … ann are constants found by fitting x to x′, y′ and b00, b01, b02, … b0n, … bnn are constants found by fitting y to x′, y′.
Each correction function will then have 2(n+1)2 coefficients that are found by curve fitting. Correction functions in the form of Equation 2 can compensate for many different forms of distortion, including situations where there are nonsymmetrical or local distortions, for example, if the distortion of the top left quadrant were different to the bottom left due to a defect in the bronchoscope lens.
Endoscopes and bronchoscopes normally produce a uniform, symmetrical and purely radial distortion 16, 23. Polar coordinates could be used to take advantage of this symmetry about the centre of the FOV 16–18, 24. Bronchoscopic distortion is purely radial so only the distance from the centre, not the angle, is altered in the image 18. Using polar coordinates reduces the complexity of the correction functions used to compensate for bronchoscope image distortion by limiting the number of coefficients to n+1. However, all common image file formats, frame capture cards and computer displays function in Cartesian coordinates, necessitating conversion of the image from Cartesian to polar coordinates and back again after the distortion has been corrected. All previous studies that used numerical distortion correction, used algorithms based on polar coordinates and developed their own custom software to do so 16–18, 23, 24.
Once a set of correction functions is generated, they can be applied to the location of any pixel in the image. In principle, the location of every pixel in the image could be corrected and a new undistorted image displayed 16–18, 23. However, displaying the corrected image has several technical difficulties. First, to display the corrected image, either the periphery of the field of view must be expanded or the centre compressed. Secondly, computer displays operate using integer values for pixel locations. After correction, the calculated locations of many pixels will not be integers, hence a display cannot be generated from the corrected pixel locations without further image processing. Measuring an object within an image only requires that those pixels defining the object be corrected, reducing the number of calculations needed, and avoiding all of the difficulties of generating a new display from the corrected pixel locations. Once a feature is identified, the image analysis system applies the correction functions to the pixels defining that feature and then provides measurements based on the corrected pixel locations.
Acknowledgments
The authors would like to acknowledge the technical support of S. Vincent.
- Received September 6, 2000.
- Accepted April 18, 2001.
- © ERS Journals Ltd
References
