Autonomous robotic systems have been used in surgery for tool positioning and for machining, and in particular for orthopedic or neurosurgical procedures. Traditionally the robot is calibrated with respect to the patient space beforehand so that the relative positions have to be kept steady along the procedure [2].
In a hybrid navigated and robotic surgery scenario, an integrated use of multimodal tracking systems would improve the safety by providing redundant information both for the robot control loop and for the surgeons. Consistence of information monitors safe operation, while inconsistence signals faulty conditions.
In such a framework, the goal of this work has been to design, implement and test a software application to simultaneously manage robotic actuators and multimodal sensors (optical tracking system, electromagnetic tracking system, US imaging device) based on the Kitware open-source library the Image-Guided Surgery Toolkit (IGSTK) [3].

Figure 1: Experimental setup. Active optical markers allow the
localization of the robot and of the EM field generator.
Materials
The experimental surgical framework we propose is composed by an active marker optical tracking system, NDI Optotrak Certus (Northern Digital Inc.); an electromagnetic (EM) tracking system, NDI Aurora (Northern Digital Inc.); an ultrasound imaging device, Prosound Alpha 7 (ALOKA Co., Ltd.); and a six Degrees of Freedom (DoF) serial industrial robot, Kawasaki FS03N (Kawasaki Heavy Industries Ltd.). The experimental setup is illustrated in Figure 1. On the software side we decided to take advantage of the functionalities provided by IGSTK.
Methods
IGSTK provides a robust framework for the development of custom computer-aided surgery applications based upon a state machine approach. This and other software features used to obtain a deterministic behavior, make IGSTK a suitable platform for our purpose [4].
IGTSK does not provide any support for NDI Optotrak Certus tracking system and no classes have been implemented for robotic actuators management, thus the first step in our work was to extend the toolkit functionalities.
IGSTK and NDI Optotrak Certus integration
On one hand NDI offers an extensive set of API function in C language to allow developers to write custom applications; the NDI API defines a client/server based communication protocol between the Optotrak Certus System Control Unit (SCU) and the host PC. On the other hand IGSTK uses a Tracker component to manage communication between the host computer and the tracking device. The core idea of this component is to continuously gather tracking data in a separate thread and periodically read the data in the main thread from a common buffer. Device specific classes are derived from the Tracker; a TrackerTool class and its device specific derived classes represent tracked objects. Additionally the IGSTK community proposed a tracker controller utility capable of managing the IGSTK supported tracking system in a device-independent manner. The user would only need to write a simple XML configuration file and the controller would recognize the device and instantiate the correct class.
Thus, to incorporate NDI Optotrak Certus in IGSTK [5] we:
(a) derived an NDICertusTracker class from the Tracker component
(b) derived an NDICertusTrackerTool class from the TrackerTool one
(c) extended the TrackerController utility to manage the new NDICertusTracker class
The Tracker component has several pure virtual methods that have to be overloaded in the device-specific class. Concerning points (a) and (b), we had to choose suitable NDI API functions to fill the overloaded methods in order to have the desired behavior for that object.
As an example, the following lines of code, demonstrate how the InternalOpen method has been overloaded to achieve the initialization of the Optotrak Certus system:
NDICertusTracker::ResultType
NDICertusTracker::InternalOpen( void )
{
igstkLogMacro( DEBUG,
"igstk::NDICertusTracker::
InternalOpen called..\n");
// Determine system configuration
igstkLogMacro( DEBUG,
"Determine system configuration...\n");
if( TransputerDetermineSystemCfg( NULL ) )
{
igstkLogMacro( CRITICAL,
"Error in determining the system parameters.\n");
return FAILURE;
}
// Load the system of processors.
if( TransputerLoadSystem("system")
!= OPTO_NO_ERROR_CODE )
{
igstkLogMacro( CRITICAL, "Error loading the system
of processors");
return FAILURE;
}
// Communication Initialization
if(TransputerInitializeSystem()!=OPTO_NO_ERROR_CODE)
{
igstkLogMacro( CRITICAL, "Error
initializing CERTUS system of processors");
return FAILURE;
}
}//InternalOpen (end)
Concerning point (c), we first defined the XML tags to be used in the Optotrak Certus configuration file:
As shown above, it was necessary to indicate the path of a setup file that NDI API functions exploit to set up the acquisition parameters. Different tracker tools in IGSTK are distinguished by means of a name. The rigid_body_name tag, instead, indicates the name of the RIG file containing markers of spatial relationships that the Certus system uses to identify a specific object.
Afterward, the code to interpret these tags was added to the tracker controller class along with a suitable interface to the new NDICertusTracker class.
IGSTK and Robot integration
Even though IGSTK is not designed to manage robotic systems, we implemented an igstk::Actuator class in order to use the safe and robust programming mechanism of the IGSTK state machine for the whole application. To do that, we defined some simple sets of states, input and transitions to model the basic tasks of a robotic system.
IGSTK provides developers with a set of macros to easily set up a state-machine within a class. The following lines show how we set (within the class constructor) the transition between two states of the Actuator class:
igstkAddInputMacro( SetTargetPose);
igstkAddTransitionMacro(
CommunicationEstablished, //Starting state
SetTargetPose, //Input
AttemptingToSetTargetPose, //Next state
SetTargetPose //Associated action
);
}
The complete state machine diagram of the igstk::Actuator class is presented in [6].
The idea was to adopt the same approach used for other IGSTK components: the igstk::Actuator class is an abstract class containing the basic state machine definitions (for example, the igstk::Tracker component), from which it is possible to derive device-specific classes that implement particular communication and control protocols.
As shown in [6], the following are the main states defined in the Actuator class:
As a starting point we implemented a KawaActuator class to manage the Kawasaki FS03N, an industrial serial 6-link robot from Kawasaki Heavy Industries Ltd. Similar to the implementation of a device-specific class based on the Tracker component, deriving the Kawasaki interface from the Actuator class consists of overloading the set of virtual methods that specify the behavior of the base class state machine.
System calibration and testing
Once the framework is set up (Figure 2), temporal and spatial alignment between sensory systems is required to gather consistent and usable data for the surgical procedure.

Figure 2: Block diagram of software organization and physical connections with system’s submodules
The management of synchronism in IGSTK is an integral part of the architectural design. To ensure temporal alignment, we exploited timing collaborations between classes in IGSTK. Each component has its own pulse generator and communicates with other objects through an event-based architecture. In addition, position data is stored in a transform object, which also provides a timestamp and a validity time. This way it is easy to verify if data acquired with different tracking systems was acquired at the same instant in time (within a certain user-defined threshold, depending on the velocity of the tracked objects in the specific application).
For targeting tasks, a common global reference frame was defined, so that we can always determine the coordinate transformation between coordinate frames in our system. We choose the NDI Certus optical tracking system as the master for our architecture because of its wide measurement volume (up to 7.0 m), high accuracy (up to 0.15 mm for a single marker), and high refresh rate (over 100 Hz, depending on the number of markers).
In such an architecture, the calibration between each system and the NDI Certus is needed [7]. We integrate the calibration procedures in the workflow of our application, which takes care of synchronous data acquisition and calibration matrix calculation (based on an iterative minimization algorithm [8]). The system performances were tested during targeting procedures with a calibrated surgical needle.
Results
The software application correctly initialized all the sensory systems and the robot and was able to provide synchronized data within the threshold of 8 ms latency.
This allowed a consistent calibration procedure that showed calibration residual errors in the range of 1 mm for translation and 0.2 degree for rotation.
As the Kawasaki FS03N is not supposed to be a surgical robot, real time control of the robot was tricky. Demonstrative targeting tasks have been successfully performed with a median position error of 2.5 mm (25th percentile = 1.56 mm, 75th percentile = 2.73 mm) and a median orientation error of 0.6 degree (25th percentile = 0.51 degree, 75th percentile = 0.82 degree).
Conclusion
Regardless of the specific system used in our set-up, the proposed architecture could lead to a less invasive, safer, and more robust platform for robotic neurosurgery. In particular, the redundancy of the sensors will provide multimodal intra-operative information to help the surgeons in critical decision during the procedure.
The results showed temporal tracker synchronization sufficient for most of the surgical procedures, in particular for slow robot movements.
Calibration residuals are probably too high for the accuracy required in neurosurgery applications and should be improved with the study of more efficient calibration algorithms, in order to have a position error less than 0.5 mm.
The software application we developed can be a starting point for an integrated and modular management of different localization and imaging systems and robotic platforms. Additionally, our application showed the capability of the IGSTK framework and its ability to be extended as needed.
The proposed architecture for the Actuator component does not provide the possibility to read encoders information about the current robot pose. A possible implementation of this feature could be based on the Tracker component structure. A separate thread is spawned in order to continuously query the encoders information while in the main thread data are read from a common buffer. This would allow real time 3D visualization in an IGSTK scene in case a 3D model of the robot is available. Further study on other robotic platforms will be carried out soon in order to assess the feasibility of this approach.
Acknowledgements
We'd like to thank Patrick Cheng and Andinet Enquobahrie for their support and helpfulness on the IGSTK mailing list. We would also like to acknowledge the support of the EU FP7 program for the ROBOCAST project (ICT FP7 215190).
References
[1] Grunert, P., Darabi, K., Espinosa, J. & Filippi, R. 2003, "Computer-aided navigation in neurosurgery", Neurosurgical review, vol. 26, no. 2, pp. 73-99.
[2] Cleary, K. & Nguyen, C. "State of the art in surgical robotics: clinical applications and technology challenges", Journal of image guided surgery, vol. 6, no. 6, pp. 312-328.
[3] Cleary, K., Cheng, P., Enquobahrie, A., Yaniv, Z. 2009, "IGSTK: The book", Signature Book Printing, Gaithersburg, Maryland.
[4] Enquobahrie, A., Cheng, P., Gary, K., Ibanez, L., Gobbi, D., Lindseth, F., Yaniv, Z., Aylward, S., Jomier, J. & Cleary, K. 2007, "The image-guided surgery toolkit IGSTK: an open source C software toolkit", Journal of Digital Imaging, vol. 20, pp. 21-33.
[5] http://public.kitware.com/IGSTKWIKI/index.php/How_to_extend_tracker_support
[6] http://www.biomed.polimi.it/nearlab/images/projects/igstk/actuator_state_machine.jpg
[7] Chung, A.J., Edwards, P.J., Deligianni, F. & Yang, G.Z. 2004, "Freehand cocalibration of optical and electromagnetic trackers for navigated bronchoscopy", LECTURE NOTES IN COMPUTER SCIENCE, pp. 320-328.
[8] De Momi, E., Cerveri, P., Gambaretto, E., Marchente, M., Effretti, O., Barbariga, S., Gini, G. & Ferrigno, G. 2008, "Robotic alignment of femoral cutting mask during total knee arthroplasty", International Journal of Computer Assisted Radiology and Surgery, vol. 3, no. 5, pp. 413-419.
Alberto Vaccarella received his M.Sc. in Biomedical Engineering from Politecnico di Milano, Milano, Italy, in 2009. He is currently a Ph.D. student at the Politecnico di Milano, involved in Computer Assisted Surgery research area and he is actively participating in the EU project ROBOCAST.
Elena De Momi received her M.Sc. in Biomedical Engineering and her Ph.D. in Bioengineering from Politecnico di Milano, Italy, for work addressing computer aided orthopedic surgery. She is currently involved in the EU project ROBOCAST for robotic neurosurgery.
Pietro Cerveri has a Master degree in Electronic Engineering from the Politecnico di Milano (1994) and has obtained his Ph.D. in Bioengineering in 2001 with a thesis on development of methods for semantic queries of biomedical images in cooperation with National Institute of Health, Bethesda, MD, USA. Since 2008, he has been an Assistant Professor at the Department of Bioengineering of Politecnico di Milano.