Markerless 3D skeletal and implant tracking with SlicerAutoscoperM
3D Slicer is a widely used, open-source software platform for visualization and analysis of medical images. Known for its flexibility and extensibility, Slicer supports over 200 community-developed extensions that enable advanced workflows for quantitative biomedical research while narrowing its focus to specific scientific domains.
The SlicerAutoscoperM (SAM) extension provides specific functionality to track the three-dimensional motion of bones and other rigid structures during dynamic movement using single-plane, bi-plane, and multi-plane videoradiography data. In addition to the core tracking module, SAM also includes tools for CT-based pre-processing, tracking evaluation, and hierarchical 3D registration for both 4D CT and sequential 3D CT data.
Technical overview
Autoscoper performs markerless 3D tracking of rigid bodies by registering CT-derived bone models to time-synchronized videoradiography images. Tracking is most commonly performed using biplane videoradiography (BVR), though single- and multi-plane configurations are supported.
For each rigid body, a partial volume is generated from segmented CT data. Autoscoper first projects this volumetric data into the image plane of each calibrated X-ray video collecting BVR to generate Digitally Reconstructed Radiographs (DRRs). These DRRs are compared directly to the BVR data on a frame-by-frame basis.
Registration is formulated as an optimization problem in the 3D space, understanding the transform as 6 degrees of freedom with three translations and three rotations. Autoscoper supports multiple optimization strategies to obtain the best transform that fits the 3D partial volume to the 2D BVR, including Particle Swarm Optimization (PSO) for global search and Downhill Simplex for fast local refinement. Similarity between DRRs and radiographs is quantified using cost functions such as Normalized Cross-Correlation (NCC) or Sum of Absolute Differences (SAD).
Resolving these complex registration processes is time consuming, that is why SAM offers both a CPU mode through OpenCL and GPU-accelerated using CUDA. GPU acceleration enables efficient processing of high-frame-rate datasets.
Once rigid body motion has been estimated from the biplane videoradiography data, the resulting transforms can be applied directly to the initially segmented 3D bone models. This allows one to see the 3D bone motion throughout the recorded activity in BVR and allows researchers to compute clinically and biomechanically relevant measures such as joint angles, ranges of motion, contact mechanics, or inter-bone distances. These metrics can be exported for downstream analysis, integrated with musculoskeletal models, or combined with other imaging modalities to support hypothesis-driven investigations of joint function and pathology.
Since its first release in 2022, SAM has been applied to track motion across a wide range of joints such as the shoulder, wrist, knee, and ankle, showing its versatility for biomechanics and orthopedic research. An assessment of SAM tracking accuracy across multiple joint types has recently been published.
Autoscoper is released under a BSD-style open-source license and is actively developed and supported by an academic research community. Extensive documentation including tutorials, sample datasets, and community support are available to help new users get started quickly.
The team that made SlicerAutoscoperM possible includes several biomechanics laboratories at the forefront of biomechanical and orthopedic research innovation, as well as by Kitware that has lent its expertise on software engineering and algorithm optimization to the project.
Meet the team at ORS 2026
The SlicerAutoscoperM will be at the Orthopedic Research Society Annual Meeting in Charlotte NC this year and would love to connect. If you are interested in markerless skeletal and implant tracking, videoradiography workflows, or collaborating with us, please reach out. We would love to connect to teach you how SAM might fit into your research.
How Kitware can help
Our driven, experienced engineering teams are always excited about enabling new biomedical research. Our team is proud to participate in projects like SlicerAutoscoperM, that is enabling researchers to maintain research grade software while generating groundbreaking findings in biomechanics.
Kitware is at the front end of research that includes high-volume, high-quality data, human oversight and seamless integration with real clinical workflows. Advances in GPU-accelerated computing combined with extensive knowledge of medical image computing and scientific computing provide teams like ours with the unparalleled power of making a difference for researchers and clinicians in the orthopedic field.