Virtual Neurosurgery- Training for the future
Michael Vloeberghs1, Tony Glover2, Steve Benford2, Arthur Jones3, Peji Wang3,
Adib Becker3.
1 Academic Division of Child Health, School of Human Development
2 School of Computer Science and Information Technology
3 School of Mechanical, Materials, Manufacturing Engineering and Management,
Correspondence address:
Mr Michael Vloeberghs, MD, PhD
Clinical Associate Professor
Consultant Paediatric Neurosurgeon
Paediatric Neurosurgery
Nottingham University Hospital
Clifton Blvd
Nottingham
NG72UH
United Kingdom
e-mail: [email protected]
Key words: Virtual reality, Neurosurgical training, Haptics, Boundary elements,
Working times directives.
1
Abstract:
Virtual Reality (VR) simulators have been created for various surgical specialties. The
common theme is extensive use of graphics, confined spaces, limited functionality
and limited tactile feedback. A development team at the University of Nottingham,
UK, consisting of: computer scientists, mechanical engineers, graphic designers and a
Neurosurgeon, set out to develop a haptic e.g. tactile simulator, for Neurosurgery
making use of Boundary Element (BE) techniques. The relative homogeneity of the
brain, allows boundary elements i.e. “surface only” rendering, to simulate the brain
structure. The surface-only modelling feature of the BE formulation reduces the
number of algebraic equations and saves computing time, by assuming the properties
of the surface equal the properties of the body. A limited audit was done by
Neurosurgical users confirming the potential of the simulator as a training tool.
This paper focuses on the application of the computational method and refers to the
underlying mathematical structure. Full references are included regarding the
mathematical methodology.
Introduction
The need for surgical simulation is driven by the limitation in training hours set by
working time directives in Western countries and increasing litigation in surgical
incidents. The net result is limited training opportunities and less chance for junior
surgeons to acquire the necessary experience during regulated training.
2
Hands-on simulation is well established in aerospace training, and audit of simulation
in medicine has shown a decrease in the number of adverse events in actual surgery
(30).
Method
In order for surgery simulation to become established as training tool, VR simulators
need to provide a diverse range of capabilities, close to reality (Table 1). This
simulator was focused on the haptic capabilities and to a lesser extent on graphics.
The development team decided that graphics are a secondary issue and easily
implemented in comparison to the real-time BE computations.
Table 1: Surgical acts to be simulated in VR.
•
Simulate the process of pushing and pulling
•
Simulate the cutting or separation of tissue, including multiple-cuts and
incisions.
•
Simulate gravitational deformation of the cut tissue
•
Allow for self-contact between the cut tissues
•
Simulate post-cutting manipulation
•
Allow for two-handed interaction
•
Allow for continuous cutting and separation of tissue, e.g. to reach a tumour
•
Simulate the complete removal of tissue, e.g. a separation of a tumour
•
Provide a realistic position and posture for the surgeon
•
Use 3D stereo vision
•
Incorporate surgical tools and implements physically connected to forcefeedback devices
•
Show accurate visual 3D models with light projection and shadows
•
Utilise patient specific virtual models, e.g. from MRI data
3
A Boundary Element virtual surgery environment
The BE-based simulator was developed at the University of Nottingham, as a
collaborative research project between the Schools of Mechanical Materials and
Manufacturing Engineering, School of Computer Science and Information
Technology and School of Human Development, Division of Child Health (34, 35).
Neurosurgery simulation is particularly challenging because it involves interaction
with the gelatinous structure of the brain and lesions of varying consistency. This
simulator allows the user to operate on an area of the brain surface, use diathermy
(cutting), retract the brain substance and remove a mass from inside the brain
substance.
The simulator is based on in-house created real-time BE software combined with
advanced computer graphics and commercially available force-feedback haptic
devices. (Table 2)
Table 2. Hardware set-up
•
A PC (high specification, typically 3 GHz) fitted with a graphics card capable of
rendering 3D images
•
A monitor compatible with the stereo vision system, angled such that the image is
reflected in a semi-silvered mirror and a visual refresh rate of at least 25 Hz
•
3D Stereo vision goggles and interface
•
Two haptic devices for position sensing and force feedback and a haptic rendering
rate around 1000 Hz (currently the Sensable Technologies PHANToM Omni
system is used) (27)
4
Replace with these ?
Figure 1: The surgery simulation hardware. A custom built rig accommodates the
computer, the haptic devices, the monitor and the reflective semitransparent mirror
needed for stereoscopic vision.
Real-time simulation of Neurosurgery
In practice, neurosurgical operations are limited to a small part of the exposed brain,
which is identified from pre-operative imaging e.g. magnetic Resonance Imaging
(MRI). MRI images can be imported directly into the original visualisation software.
Only this part of the brain is rendered with a fine 3D mesh, while the other parts can
remain relatively coarse meshed to save computing time.
BE algorithms were developed by the authors for simulating the pushing, pulling,
cutting e.g. using a bipolar forceps, retracting and two handed operating generally
used in Neurosurgery. When the haptic device is moved by the user, a contact search
algorithm detects which points are in contact. If the haptic device penetrates the
virtual surface, the penetration displacement is used as a displacement in the BE
software. The feedback forces are simultaneously computed as reaction forces and fed
5
back to the user via the haptic device. The graphics are updated accordingly in real
time.
Replace with this?
An especially challenging aspect of simulating surgery is the real-time simulation of
the surgical cutting i.e. incising the cortex with a bipolar forceps. The initial cut is
created as a gap between existing surface elements. New surface elements are created
in the direction of the cutting plane, without altering the original surface mesh.
(Figures 2-6)
(a) Pulling action
(b) Pushing action
Figure 2: Examples showing pulling and pushing actions
6
Figure 3: Example showing the application of two haptic devices
(a) Single retractor
(b) Two retractors
Figure 4: Example showing the application of retractors to separate the cut surfaces
7
(a) Initiating a cut
(b) Extending a cut
Figure 5: Examples showing an initial cut being extended
Figure 6: Example showing a post-cutting manipulation
8
Simulation of removing a tumour
The simulator allows the surgeon to cut deeper until a tumour is reached. The cut can
then continue around the tumour surface until the tumour is totally separated and
subsequently removed. Although the tumour is located within the brain, it can be
simulated as a BE surface in full contact with the surrounding brain, i.e. the surfaceonly feature of BE modelling is preserved. Figure 7 shows a wire frame model
containing a tumour underneath the surface.
Figure 7: 3D wire frame of a tumor beneath the surface of the organ
Feedback from surgeons
A preliminary evaluation of the simulator was performed. Two initial sessions were
undertaken within the Nottingham University Hospital (NUH) where the system was
demonstrated
to
more
than
24
neurosurgical-related
staff
(ranging from
9
neurosurgeons to theatre nurses). A more refined version of the simulator was then
evaluated at a subsequent session in October 2005 with 13 participants who were
either consultants or trainee neurosurgeons. Participants tried the simulator for a short
period of time (a few minutes each) before giving feedback through a short
questionnaire which gathered their opinions about its realism and potential
improvements (Table 3). The evaluation only addressed prodding and pinching and
making a few cuts. The working prototype was used with basic graphics and only one
haptic device.
Preliminary feedback suggests that the current BE-based simulator and the hardware
can achieve a sufficient level of realism to have a useful role in surgical training.
Participants’ responses suggest that the simulation of pushing felt realistic, but pulling
was less realistic since the current system allows an unlimited amount of tissue
stretching. The simulation of cutting, while functional, requires further improvements
in terms of feel and extra features such as simulating bleeding e.g. augmented reality
would be a useful addition.
10
Table 3. Results of the VR questionnaire put to 13 Neurosurgeons, October 2005
(Scoring system: 1-5, 1=Very Good, 5=Very Bad)
Question
Mean
Standard
deviation
In general, how easy was the simulator to use?
1.31
0.48
How realistic did the brain look whilst pushing?
2.15
0.55
How realistic did pushing the brain feel?
2.46
0.5
How easy was pushing the brain?
2.08
0.76
How realistic did the brain look whilst pulling?
2.62
0.65
How realistic did pinching the brain tissue feel?
2.77
0.6
How easy was pulling the brain?
2.0
0.74
How realistic did the brain look whilst cutting?
3.0
0.82
How realistic did cutting the brain feel?
3.38
0.51
How easy was cutting the brain?
2.38
0.87
How realistic was the stereo viewing?
1.77
0.73
How comfortable was the physical setup?
1.54
0.66
Could the simulator help you understand basic surgical acts?
13
12
11
10
9
8
7
6
5
4
3
2
1
0
Yes
No
Do you think the simulator has a role in surgical training?
13
12
11
10
9
8
7
6
5
4
3
2
1
0
Yes
No
11
Discussion
VR in surgery simulation:
The application of VR to surgery simulation was first proposed in the early 1990’s (1,
9, 11, 24, 29) and focussed on task simulation. With the increasing computer
processing power and the availability of sophisticated input/output devices such as
force-feedback devices (26), surgery simulation gained in sophistication and realism.
The lack of sufficient opportunities for trainee surgeons to practice surgery and
clinical governance issues gives surgical simulators a role in training. Guidelines
regarding surgical competence from the Royal college of Surgeons of England
emphasise the parallel between civil aviation training and surgical training and
highlight the role of simulation (30). Trainees also voice their concern about training
and the time spent exposed to surgery (10, 19, 23). In the previous UK training
system, trainees would spend and excess of 30000 hours training in their specialty. In
the Modernised Medical Career (MMC) system, the hours are reduced to 15000 i.e.
50%. In comparison a NASA astronaut trains 10000 hours and a long haul airline
pilot trains 5000 hrs (10). Building on previous VR work of the first author, this
simulator uses “Patient specific” data from MRI (32, 33). Simulation of an actual
patient is possible and extends the use of the device to the senior level where
simulators have a role in Continuous Medical Training and pre-operative simulation
of complex cases.
VR surgery is used in many specialties, such as endoscopy (31), microsurgery (12),
neurosurgery (28), urology (14), orthopaedics (7) and ophthalmology (17), and is
gradually gaining acceptance in the medical profession (13, 20). Previous simulators
12
have concentrated on endoscopic surgery e.g. operating in a confined space with
limited freedom, limiting the simulator to confined spaces and “drag and drop”
surgical acts. This simulator approaches Neurosurgery from an “outside in“
perspective and uniquely allows the user to operate on the surface of the brain.
Real-time Boundary Element computations
The Boundary Element (BE) method is well established as an accurate stress analysis
technique in which only the surface (boundary) needs to be represented (see, e.g. 2, 3,
8, 25); this is contrast with the finite element (FE) method, which requires
representation of the entire model (see, e.g. 4, 5, 6). The interior of a BE domain is
not rendered, resulting in a better resolution of the surface. A recent review of
deformable models for surgery simulation by Meier et al (21) has identified the BE
technique as being one of the most promising routes to surgical simulation..
Two early attempts, published by James, and Pai (15, 16) and Monserrat et al (22), to
use BE to simulate deformable objects in VR support the method, demonstrated the
basic feasibility of this approach, but did not cover the simulation of cutting, postcutting deformation or two-handed operations. The BE work presented in this paper
has built on this baseline, further extending the use of BE in surgical simulation (18,
34, 35).
Conclusion
An overview of a unique BE-based VR Neurosurgery simulator is presented which
features real-time visual and haptic feedback allowing the user to perform the basic
13
Neurosurgical acts on the brain. The research team has created the BE algorithms for
this complex simulation and have proven that the surface-only modelling capability of
the BE techniques are highly suitable for VR surgery.
Initial trials of the system by Neurosurgeons have indicated that a sufficient degree of
realism can be achieved and that such simulators can play a useful role in surgical
training. There are many challenges to address, which include more realism by use of
augmented reality to simulate bleeding, tearing of tissue etc.
The authors believe simulation techniques, especially VR with haptic feedback as
described in this paper, will in part address the concerns raised by training and
governance bodies regarding training hours and litigation.
Acknowledgements
The authors wish to acknowledge the financial support of the UK, Engineering and
Physical Sciences Research Council (EPSRC) research grant GR/R84030.
References:
1) Bathe K, “Finite element procedures in engineering analysis”, Prentice-Hall, New
Jersey, 1982.
2) Becker A, “The boundary element method in engineering”, McGraw-Hill, London,
14
1992.
3) Becker A and Mihsein M, “Boundary element analysis of gravitational loading on
structures”, Developments in Computational Techniques for Structural
Engineering, edited by B.H.V. Topping, 197-204, Civil-Comp press, Edinburgh,
1995.
4) Belytschko T, Liu W, Moran B, “Nonlinear Finite Elements for Continua and
Structures”, J. Wiley & Sons, New York , 2000.
5) Berkley J, Turkiyyah G, Berg D, Ganter M, Weghorst S, “Real-time finite element
modelling for surgery simulation: an application to virtual suturing”, IEEE
Trans. Visualization and Computer Graphics 2004; 10: 314-325
6) Bielser D, Glardon P, Teschner M, and Gross M, “A state machine for real-time
cutting of tetrahedral meshes”, Graphical Models 2004; 66: 398-417
7) Blyth P, Anderson I, Stott N, “Virtual reality simulators in orthopedic surgery:
What do surgeons think?” J. Surgical research2006; 131, 133-139
8) Brebbia C, Telles J, Wrobel L, “Boundary element techniques”, Springer Verlag,
Berlin, 1983.
9) Cover S, Ezquerra N, O’Brian J, Rowe R, Gadacz T, Palm E, “Interactively
deformable models for surgery simulation”, IEE Comput. Graph. Appl. 1993; 68-75
15
10) Devey L, “Will Modernised Medical Careers Produce a Better Surgeon?”
British Medical Journal 2005; 331: 1346
11) Delingette H, Cotin S, Ayache N, “Efficient linear elastic models of soft tissues
for real-time surgery simulation”, Studies in Health Technology and
Informatics1999; 62: 100-101
12) Erel E, Aiyenibe B, Butler P, “Microsurgery simulators in virtual reality:
Review”, J. Microsurgery 2003; 23: 147-152
13) Gallagher A, Cates C, “Virtual reality training for the operating room and cardiac
catheterisation laboratory”, The Lancet 2004; 364: 1338-1540
14) Jacomides L, Ogan K, Cadeddu J, “Use of a virtual reality simulator for
ureteroscopy training”, J. Urol. 2004; 171: 320-323
15) James D, Pai D “ArtDefo: Accurate real time deformable objects”, Proc
SIGGRAPH-99 1999; 65-72
16) James D, Pai D, “A Unified Treatment of Elastostatic and Rigid Contact
Simulation for Real Time Haptics”, Haptics-e, the Electronic Journal of Haptics
Research 2001; 2
17) Khalifa Y, Bogorad D, Gibson V, Peifer, Nussbaum J, “Virtual reality in
16
opthalmology training”, Survey of Opthalmology 2006; 51: 259-273
18) Leahy J, Becker A, “A quadratic boundary element formulation for threedimensional contact problems with friction”, J. Strain Analysis 1999; 34: 235251
19) Lieske B, “Dilemma of a Surgical Trainee” British Medical Journal 2005; 331:
1347
20) Meier A, Rawn C, Krummel T, “Virtual reality: surgical application- Challenges
for the new millennium”, J. American Coll. Surgeons 2001; 192: 372-384
21) Meier U, Lopez C, Monserrat C, Juan, Alcaniz M, “Real-time deformable models
for surgery simulation: a survey”, Computer Methods and Programs in
Biomedicine2005; 77: 183-197
22) Monserrat C, Meier U, Alcaniz M, Chinesta F, Juan M, “A new approach for the
real-time simulation of tissue deformations in surgery simulation”, Computer
Methods and Programs in Biomedicine 2001; 64: 77-85
23) Moorty K, Vincent C, Darzi A, “Simulation Based Training” British Medical
Journal 2005; 330: 493-494
24) Picinbono G, Delingette H, Ayache N, “Non-linear anisotropic elasticity for realtime surgery simulation” Graphical Models 2003; 65: 305-321
17
25) Portela A, Aliabadi M, “The Dual boundary element method: effective
implementation for crack problems”, Int. J. Numerical Methods in Eng. 1992;
33: 1269-1287
26) Schijven M, Jakimowicz J, “Virtual reality surgical laparoscopic simulators”, J.
Surgical Endoscopy 2003; 17: 1943-1950
27) SensAble Technologies Limited (2005), http://www.sensable.com.
28) Spicer M, Apuzzo M, “Virtual reality surgery: Neurosurgery and the
contemporary landscape”, Neurosurgery 2003; 52: 489-497
29) Stava R, “Virtual reality surgical simulator- The first steps”, Surgical
Endoscopy1993; 7, 203-205
30) Surgical Competence, Challenges of Assessment in Training and Practice, Royal
College of Surgeons of England, The Smith & Nephew Foundation, 11/1999,
ISBN 0902166301
31) Tanoue K, Yasunaga T, Konishi K, Okazaki K, Ieiri S, Kawabe Y, Matsumoto K,
Kakeji Y, Hashizume M, “Effectiveness of training for endoscopic surgery
using a simulator with virtual reality: Randomised study”, Int. Congress Series
2005; 1281: 515-520
18
32) Vloeberghs M, Hatfield F, Daemi F “A Virtual Reality Model of the Human
Ventricular System” Computer Integrated Surgery 1997; Suppl ISSN 1092-9088
33) Vloeberghs M, Daemi F, Demeshki J, Hatfield F “A Virtual Reality Model of the
Human Ventricular System” Minimally Invasive Neurosurgery 1998; 41: 126
34) Wang P, Becker A, Glover A, Benford S, Greenhalgh C, Vloeberghs M, Jones I,
“Application of the Boundary Element Method to the simulation of surgery
including haptic feedback” Proc. Seventh Int. Conf. on Computational
Structures Technology, Lisbon, 7-9 September 2004, edited by B.H.V. Topping
and C.A. Mota Soares, Civil-Comp Press, Stirling, paper 100, 2004.
35) Wang, P., Becker, A. A., Jones, I. A. , Glover, A.T., Benford, S. D., Greenhalgh,
C. M. and Vloeberghs, M., “A virtual reality surgery simulation of cutting and
retraction in neurosurgery with force-feedback”,
J. Computer Methods &
Programs in Biomedicine, accepted July 2006 (in press)..
19