Fiber-optic Bronchoscopy Simulator

•

Real-time Fiber-optic Bronchoscopy Simulator

The human airway extends from the nasal cavity and mouth, through the vocal cord and epiglottis region, and into the trachea and bronchial bifurcations. Bronchoscopes are used by pulmonologists and emergency physicians to access the airway for a variety of procedures, including emergency intubation, diagnosis, and biopsy. The device consists of fiber-optic tubing with an endoscopic camera tip, and a joystick-like handle for rotation of the camera.

This simulator gives the user a virtual experience of using a bronchoscope to navigate a human airway and experience force-feedback when the scope makes contact with flesh. Objectives include developing psychomotor and cognitive skills by learning to navigate through the airway with minimal discomfort and injury to the virtual patient, and being able to identify anatomy within the throat, upper airway and bronchi. This will ensure that the user is trained and assessed sufficiently before encountering a real patient for the first time. The simulation runs on Microsoft’s XBOX XNA platform, and can be operated with both the XBOX standard controller and a custom-made hardware controller created by our team.

Screen capture of gameplay (Best viewed in HD fullscreen view)

Process:

The project team consisted of one artist, two programmers, one mechanical engineer, and physician subject matter experts. We first looked at existing bronchoscopy simulation trainers and found two areas that we could improve upon:

1) No simulator was able to create an accurate feeling of force-feedback when anatomy was approached/grazed.
2) No simulator had high resolution graphics for accurate identification of various sub-anatomical regions.

To address the desired output of force-feedback from the software perspective, we decided to create a collision mesh which consisted of a low-fidelity replica of the high-resolution anatomical 3D mesh. For parts of the anatomy which caused the scope to be pushed and blocked significantly, such as the vocal cords, and tracheal muscle contractions, we created collision cylinders which contained a skeletal rig that would mimic the bones of the original anatomy rig.

In order to create anatomically accurate high resolution graphics, we started by finding average adult airway measurements in Gray’s Anatomy, the bible for anatomical proportions. To gather a sense of three dimensional proportions, we looked at human and sheep cadavers as reference. To visualize what living and breathing airways look like, we observed pulmonary procedures in-person, and simultaneously captured video footage from the bronchoscope camera. The 3D mesh, animation, texture and wetness (specularity) were generated from an averaging of all these references.

Looking at the epiglottis and vocal cords in a wireframe mesh view in 3D Studio Max

Looking down the trachea in a wireframe mesh view, with exposed skeletal bones in 3D Studio Max

The skeletal structure of the animation rig, inside the vocal cord region. More than 120 bones were rigged and animated throughout the airway to create a realistic feeling of flesh and muscle motion.

Gamification

In order to translate this procedure into a user experience on the XBOX, we worked with pulmonogists to understand the visual elements needed to capture the essence of performing a bronchoscopy. Many pulmonogists debated over the value of audio vs. visual depiction of the patient's vital signs, so we included both in the simulation's HUD (head-up display). Additionally, for the process of learning how to orient oneself in the airway for the first time, we created an additional viewport capturing an orthographic view of the scope's motion inside the patient, and a corresponding scope model that tilted and shifted as the player tilted controller toggles.

The player had the option of operating a custom-designed XBOX-controlled hardware scope that mimicked all motions of a real bronchoscope (see publication below)