Abstract
Background
As we move toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. In the era of minimally invasive surgery, there is a need for a reliable and valid tool dedicated to measure competence in video-assisted thoracoscopic surgery. The purpose of this study is to create such an assessment tool, and to explore its reliability and validity.
Methods
An expert group of physicians created an assessment tool consisting of 10 items rated on a five-point rating scale. The following factors were included: economy and confidence of movement, respect for tissue, precision of operative technique, creation and placement of ports, localization of pathologic tissue, use of staplers, retrieval of tissue in bag and placement of chest tube. Fifty consecutive thoracoscopic wedge resections were recorded and assessed blindly and independently by two experts using the tool.
Results
Four residents, four fellows and five consultants performed 1–10 (median 4) operations each. The fellows performed significantly better than the residents (P = 0.03; effect size, ES = 0.72). The consultants scored 11% higher than the fellows, but this difference was not significant (P = 0.10, ES = 0.64). The inter-rater reliability was acceptable (Cronbach’s alpha 0.71).
Conclusions
This tool for assessing performance in thoracoscopy is reliable and valid. It can provide unbiased feedback to trainees, and can be used to evaluate new teaching curricula, i.e. simulation-based training. Furthermore, it has potential to aid in certification of new thoracic surgeons.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Lee P, Mathur PN, Colt HG (2010) Advances in thoracoscopy: 100 years since Jacobaeus. Respiration 79:177–186
Hansen HJ, Petersen RH, Christensen M (2011) Video-assisted thoracoscopic surgery (VATS) lobectomy using a standardized anterior approach. Surg Endosc 25:1263–1269
Whitson BA, Groth SS, Duval SJ, Swanson SJ, Maddaus MA (2008) Surgery for early-stage non-small cell lung cancer: a systematic review of the video-assisted thoracoscopic surgery versus thoracotomy approaches to lobectomy. Ann Thorac Surg 86:2008–2016
Yan TD, Black D, Bannon PG, McCaughan BC (2009) Systematic review and meta-analysis of randomized and nonrandomized trials on safety and efficacy of video-assisted thoracic surgery lobectomy for early-stage non-small-cell lung cancer. J Clin Oncol 27:2553–2562
Moorthy K, Munz Y, Sarker SK, Darzi A (2003) Objective assessment of technical skills in surgery. BMJ 327:1032–1037
Reznick RK, MacRae H (2006) Teaching surgical skills–changes in the wind. N Engl J Med 355:2664–2669
van Hove PD, Tuijthof GJ, Verdaasdonk EG, Stassen LP, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987
Colt HG, Davoudi M, Quadrelli S, Zamanian RN (2010) Use of competency-based metrics to determine effectiveness of a postgraduate thoracoscopy course. Respiration 80:553–559
Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278
Ringsted C, Ostergaard D, Ravn L, Pedersen JA, Berlac PA, Van dV (2003) A feasibility study comparing checklists and global rating forms to assess resident performance in clinical skills. Med Teach 25:654–658
Yudkowsky R (2010) Performance Tests. In: Downing SM, Yudkowsky R (eds) Assessment in health professions education, 1st edn. Routledge, New York, pp 119–148
Reznick R, Regehr G, MacRae H, Martin J, McCulloch W (1997) Testing technical skill via an innovative “bench station” examination. Am J Surg 173:226–230
Streiner DL, Norman GR (2008) Reliability. In: Streiner DL, Norman GR (eds) Health measurement scales—a practical guide to their development and use, 4th edn. Oxford University Press, Oxford, pp 167–210
Cicchetti DV, Fleiss JL (1977) Comparison of the null distributions of weighted kappa and the C ordinal statistic. Appl Psychol Measure 1(2):195–201
Zakzanis KK (2001) Statistics to tell the truth, the whole truth, and nothing but the truth: formulae, illustrative numerical examples, and heuristic interpretation of effect size analyses for neuropsychological researchers. Arch Clin Neuropsychol 16:653–667
Downing SM (2004) Reliability: on the reproducibility of assessment data. Med Educ 38:1006–1012
Konge L, Larsen KR, Clementsen P, Arendrup H, Buchwald CV, Ringsted C (2011) Reliable and valid assessment of clinical bronchoscopy performance. Respiration, in press
McGaghie WC, Butter J, Kaye M (2009) Observational assessment. In: Downing SM, Yudkowsky R (eds) Assessment in health professions education, 1st edn. Routledge, New York, pp 185–215
Iwasaki A, Moriyama S, Shirakusa T (2008) New trainer for video-assisted thoracic surgery lobectomy. Thorac Cardiovasc Surg 56:32–36
Meyerson SL, LoCascio F, Balderson SS, D’Amico TA (2010) An inexpensive, reproducible tissue simulator for teaching thoracoscopic lobectomy. Ann Thorac Surg 89:594–597
Solomon B, Bizekis C, Dellis SL, Donington JS, Oliker A, Balsam LB, Zervos M, Galloway AC, Pass H, Grossi EA (2011) Simulating video-assisted thoracoscopic lobectomy: a virtual reality cognitive task simulation. J Thorac Cardiovasc Surg 141:249–255
Disclosures
Authors Konge, Lehnert, Hansen, Petersen and Ringsted have no conflicts of interest or financial ties to disclose.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Konge, L., Lehnert, P., Hansen, H.J. et al. Reliable and valid assessment of performance in thoracoscopy. Surg Endosc 26, 1624–1628 (2012). https://doi.org/10.1007/s00464-011-2081-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-011-2081-7