Forskning ved Københavns Universitet - Københavns Universitet

Forside

Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningfagfællebedømt

Standard

Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study. / Dagnaes-Hansen, Julia; Konge, Lars; Andreassen, Kim Hovgaard; Hansen, Rikke Bølling.

I: Scandinavian Journal of Urology, 2021.

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningfagfællebedømt

Harvard

Dagnaes-Hansen, J, Konge, L, Andreassen, KH & Hansen, RB 2021, 'Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study', Scandinavian Journal of Urology. https://doi.org/10.1080/21681805.2021.1960599

APA

Dagnaes-Hansen, J., Konge, L., Andreassen, K. H., & Hansen, R. B. (Accepteret/In press). Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study. Scandinavian Journal of Urology. https://doi.org/10.1080/21681805.2021.1960599

Vancouver

Dagnaes-Hansen J, Konge L, Andreassen KH, Hansen RB. Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study. Scandinavian Journal of Urology. 2021. https://doi.org/10.1080/21681805.2021.1960599

Author

Dagnaes-Hansen, Julia ; Konge, Lars ; Andreassen, Kim Hovgaard ; Hansen, Rikke Bølling. / Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study. I: Scandinavian Journal of Urology. 2021.

Bibtex

@article{3e1e222abef04a5ab10bacba610bb64d,
title = "Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study",
abstract = "Objective: The growing use of simulation-based training makes it necessary to develop efficient training programs in order to ensure optimal use of time and resources. Our aim was to develop and gather validity evidence for a simulation-based test in ureteronephroscopy and set a pass/fail standard for the test that will allow future mastery learning. Design: This study is a validation study. A test in ureteronephroscopy and stone removal on the URO Mentor{\texttrademark} virtual reality simulator (3D Systems, USA) was developed by two experienced urologists in order to ensure content. Participants with different experience completed three standardized tasks on the simulator and simulator-generated metrics were used as outcome parameters to minimize bias and ensure a fair response process. Results: Twenty novices, 15 intermediates, and 8 experienced urologists were included in the study. Validity evidence for internal structure and relationship to other variables was questionable with weak and mostly insignificant correlations across all four metrics (Cronbach{\textquoteright}s alpha = 0.14, p = 0.15) and across the three modules (Cronbach{\textquoteright}s alpha = 0.41 (p = 0.02), 0.35 (p = 0.06), 0.10 (p = 0.35), and 0.30 (p = 0.09) for each metric, respectively). It was not possible to establish a pass/fail score for the simulation test with meaningful consequences. Conclusion: Our study showed that automatically generated simulator metrics cannot be used as a valid way of assessing competence in ureteronephroscopy. Virtual-reality simulator training could still be a valuable and patient-safe way to practice these skills, but an experienced supervisor is needed to determine when the trainee is ready to continue to supervised practice on patients.",
keywords = "medical education, Ureteroscopy, urology urolithiasis, virtual reality",
author = "Julia Dagnaes-Hansen and Lars Konge and Andreassen, {Kim Hovgaard} and Hansen, {Rikke B{\o}lling}",
note = "Publisher Copyright: {\textcopyright} 2021 Acta Chirurgica Scandinavica Society.",
year = "2021",
doi = "10.1080/21681805.2021.1960599",
language = "English",
journal = "Scandinavian Journal of Urology and Nephrology",
issn = "0036-5599",
publisher = "Taylor & Francis",

}

RIS

TY - JOUR

T1 - Virtual reality simulator metrics cannot be used to assess competence in ureteronephroscopy and stone removal–a validation study

AU - Dagnaes-Hansen, Julia

AU - Konge, Lars

AU - Andreassen, Kim Hovgaard

AU - Hansen, Rikke Bølling

N1 - Publisher Copyright: © 2021 Acta Chirurgica Scandinavica Society.

PY - 2021

Y1 - 2021

N2 - Objective: The growing use of simulation-based training makes it necessary to develop efficient training programs in order to ensure optimal use of time and resources. Our aim was to develop and gather validity evidence for a simulation-based test in ureteronephroscopy and set a pass/fail standard for the test that will allow future mastery learning. Design: This study is a validation study. A test in ureteronephroscopy and stone removal on the URO Mentor™ virtual reality simulator (3D Systems, USA) was developed by two experienced urologists in order to ensure content. Participants with different experience completed three standardized tasks on the simulator and simulator-generated metrics were used as outcome parameters to minimize bias and ensure a fair response process. Results: Twenty novices, 15 intermediates, and 8 experienced urologists were included in the study. Validity evidence for internal structure and relationship to other variables was questionable with weak and mostly insignificant correlations across all four metrics (Cronbach’s alpha = 0.14, p = 0.15) and across the three modules (Cronbach’s alpha = 0.41 (p = 0.02), 0.35 (p = 0.06), 0.10 (p = 0.35), and 0.30 (p = 0.09) for each metric, respectively). It was not possible to establish a pass/fail score for the simulation test with meaningful consequences. Conclusion: Our study showed that automatically generated simulator metrics cannot be used as a valid way of assessing competence in ureteronephroscopy. Virtual-reality simulator training could still be a valuable and patient-safe way to practice these skills, but an experienced supervisor is needed to determine when the trainee is ready to continue to supervised practice on patients.

AB - Objective: The growing use of simulation-based training makes it necessary to develop efficient training programs in order to ensure optimal use of time and resources. Our aim was to develop and gather validity evidence for a simulation-based test in ureteronephroscopy and set a pass/fail standard for the test that will allow future mastery learning. Design: This study is a validation study. A test in ureteronephroscopy and stone removal on the URO Mentor™ virtual reality simulator (3D Systems, USA) was developed by two experienced urologists in order to ensure content. Participants with different experience completed three standardized tasks on the simulator and simulator-generated metrics were used as outcome parameters to minimize bias and ensure a fair response process. Results: Twenty novices, 15 intermediates, and 8 experienced urologists were included in the study. Validity evidence for internal structure and relationship to other variables was questionable with weak and mostly insignificant correlations across all four metrics (Cronbach’s alpha = 0.14, p = 0.15) and across the three modules (Cronbach’s alpha = 0.41 (p = 0.02), 0.35 (p = 0.06), 0.10 (p = 0.35), and 0.30 (p = 0.09) for each metric, respectively). It was not possible to establish a pass/fail score for the simulation test with meaningful consequences. Conclusion: Our study showed that automatically generated simulator metrics cannot be used as a valid way of assessing competence in ureteronephroscopy. Virtual-reality simulator training could still be a valuable and patient-safe way to practice these skills, but an experienced supervisor is needed to determine when the trainee is ready to continue to supervised practice on patients.

KW - medical education

KW - Ureteroscopy

KW - urology urolithiasis

KW - virtual reality

U2 - 10.1080/21681805.2021.1960599

DO - 10.1080/21681805.2021.1960599

M3 - Journal article

C2 - 34338581

AN - SCOPUS:85111916169

JO - Scandinavian Journal of Urology and Nephrology

JF - Scandinavian Journal of Urology and Nephrology

SN - 0036-5599

ER -

ID: 276279065