Advancing Breast Surgical Planning with 3D Slicer

Precise breast tumor removal depends on a thorough understanding of patient anatomy and tumor location before the first incision is made. As part of the PSI/SURPASS project, funded by ARPA-H and led by Johns Hopkins University, Kitware is developing an intuitive workflow for robotic breast surgery planning. This workflow is built using 3D Slicer, an open-source platform for medical image visualization and analysis. 
 
This workflow illustrates how open science and collaborative software development can accelerate innovation in image-guided surgery, from research prototypes to clinical translation.  By leveraging the 3D Slicer imaging platform, we were able to quickly develop and use the application by focusing on adding novel features on top of an existing foundation. 

Building the breast surgical planning workflow 

The workflow is designed to integrate seamlessly with image acquisition systems and robotic platforms used for breast tumor localization and removal. The process begins once the imaging device (surgical endoscope) and robotic system (6 DOF robot arm) are connected and live, enabling real-time interaction with the patient’s imaging data from the surgical endoscope. The workflow, developed in 3D Slicer, includes the following major steps: 

System Setup and Connection  

The clinician initiates the session, and the interface verifies that the imaging device and robotic hardware are connected and communicating. Once both systems are confirmed active, the clinician can proceed to the next step. 

Field Scan 

The Field Scan tab enables users to capture a high-resolution surface scan of the surgical cavity.  In preparation for the field scan, the user places the robot into a cooperative control mode which allows them to manually position the robot arm at its starting point. During scanning, the live feed from the imaging device helps ensure full coverage of the surgical cavity, forming the basis for subsequent registration and navigation tasks.  The robot executes a pre-defined set of movements during the scan to capture a series of images that will later be stitched together. 

Live Demo Mode 

The Live Demo feature provides a guided example for testing and demonstration. This mode showcases how real-time data flows between the imaging hardware, 3D Slicer interface, and downstream analysis tools — useful both for development validation and stakeholder demonstrations. 

Workflow Settings and Interface Testing 

The Workflow Settings tab gives users control over parameters such as scan resolution, data storage location, and robot communication protocols. The Interface Testing Dashboard is a diagnostic view used to verify data exchange between hardware and software modules, helping ensure reliable performance in both simulation and live trials. 

Why 3D Slicer? 

3D Slicer provides an ideal foundation for this type of workflow. Its modular architecture supports: 

• Custom UI panels for guided procedures. 

• Integration with external devices via OpenIGTLink. 

• Rapid prototyping of medical image computing tools using Python and C++. 

• Extensibility — enabling other PSI/SURPASS partners, as well as projects working on other surgical procedures, to build upon Kitware’s work. 

By leveraging these capabilities, the breast surgical planning workflow enables researchers and clinicians to visualize imaging data, interact with robotic systems, and manage procedural steps within a single, unified environment. 

Looking Ahead 

This initial version of the workflow is currently being used for interface testing and live demonstrations as part of ongoing PSI/SURPASS milestones. Future updates will focus on integrating tumor localization, surgical planning, and intraoperative guidance — all powered by open-source technologies. 
 
As the project progresses, Kitware continues to collaborate with our partners to ensure the workflow supports clinical translation while maintaining reproducibility and openness. The lessons learned here will inform similar capabilities for other tumor types in the future. 

ACKNOWLEDGMENTS  

Research reported in this publication was funded by the Advanced Research Projects Agency for Health (ARPA-H) under Award Number D24AC00359-00. The ARPA-H award provided 100% of total costs and total up to $20.9M. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Advanced Research Projects Agency for Health.