MultiverSeg segment editor effect

Kitware is pleased to announce the release of the Slicer MultiverSeg extension for 3D Slicer, which integrates the MultiverSeg model into 3D Slicer as a new Segment Editor effect. This segment editor effect allows the use of the MultiverSeg model for segmentation and enables researchers and clinicians to automatically segment novel structures given a small number of samples manually segmented.

MultiverSeg [1] is a 2D segmentation AI model designed for medical imaging developed by Hallee E. Wong et al. at MIT CSAIL in the Clinical and Applied Machine Learning (CAML) group. It combines interactive segmentation with in-context learning to enable flexible and effective performance across a wide range of tasks.

The interactive segmentation component allows users to iteratively refine results by drawing positive and negative scribbles. Scribbles are fed back into the model to adjust the model for the next prediction. Additionally, the in-context learning mechanism enables the model to generalize to previously unseen tasks using only a few example inputs.

Context and tasks

The true strength of MultiverSeg lies in its in-context segmentation capability. A variety of options are available to manage tasks and their associated contexts. Through those management options, it is possible to automatically sample the current view and segmentation to add it to the current task.

For good results, we recommend manually segmenting a few key slices to populate the context before applying the MultiverSeg effect. The number of samples needed for the model to learn a task depends on its complexity and diversity.

Using the extension

First, select the view you want to segment and click Initialize. This will automatically download the necessary dependencies, set up the model, and create the positive and negative segments where you can draw the corresponding scribbles.

Next, select the task you want the model to perform. If no task is currently available or you want to use a new one, create it and add a few samples using the task management options.

After Selecting the task, you can either perform a 2D prediction on the current slice, or define upper and lower bounds to run the 2D segmentation for each slice across the selected range—resulting in a full 3D segmentation.

More information available about usage on github [2].

Acknowledgments

This work was funded by the Cure Overgrowth Syndromes (COSY) RHU Project (ANR-18-RHUS-005) and achieved in collaboration with MultiverSeg original author Hallee Wong who developed the MultiverSeg model and provided valuable assistance during the development of this extension.

Kitware’s Role: Helping You Go Further, Faster

While 3D Slicer offers impressive capabilities out of the box, many research teams and product developers turn to Kitware for advanced customization and support. Our experts can help:

• Develop tailored extensions for your specific research or product needs.
• Integrate new algorithms and machine learning models.
• Support regulatory pathways with validated pipelines.
• Optimize performance for large datasets or real-time applications.

Explore our success stories

• ✨ Customer Highlights: See how companies and industry partners are building commercial products with 3D Slicer.
• 🔬 Research and Development: Explore case studies that demonstrate how Slicer facilitates cutting-edge medical imaging research and product development.

We also maintain strong ties to the Slicer community and help guide the platform’s evolution through contributions, governance, and support services.

Contact our team to explore how we can help you achieve your goals.

References

[1] Wong, Hallee E., et al. “MultiverSeg: Scalable Interactive Segmentation of Biomedical Imaging Datasets with In-Context Guidance.” arXiv preprint arXiv:2412.15058 (2024).

[2] https://github.com/dalcalab/MultiverSegSlicer