The (VTK) Innovation Goes On….

Recently, we celebrated VTK’s 30th anniversary [1]. As mentioned in that birthday blog post, the original developers felt that a 10-year run would be a smashing success. Astonishingly, not only are we early in the fourth decade of development, but the pace of development seems to be increasing, with users and developers from Kitware and around the world making highly impactful contributions. In the blog series to follow, we will highlight a few ongoing activities that are sure to have significant, long-term effects on VTK, its associated community, and the many applications that use VTK.

The activities that we will focus on over the forthcoming posts are related to VTK’s use of the following technologies:

• WASM or WebAssembly, which is used to transform VTK C++ code into high-performance applications that can be run in a web browser.
• WebGPU [2], which is an API for JavaScript, Rust, C++, and C supporting cross-platform efficient graphics processing unit (GPU) access. WebGPU provides access to underlying Vulkan, Metal, or Direct3D 12 graphics APIs for rendering as well as computations on the machine hardware..
• AI tools and workflows. Inexorably, AI tools are making significant inroads into and providing advanced capabilities for computing applications. Much recent work involves making VTK easier to use in AI-centric systems (e.g., produce code, navigate documentation, and prototype applications), providing interfaces to AI data and systems, and adding visualization capabilities (e.g. saliency graphs for AI explainability) to review and validate output information.

The goal of this work is to ensure that VTK remains a valuable computing tool into the future. For example, by providing automated workflows for WASM, the VTK community will be able to develop high-performance capabilities in C++, and then readily embed them into applications that can be run on the web, and readily interface with systems such as trame [3], VTK.js [4], ParaView [5], and Slicer [6]. By providing WebGPU support, VTK’s current support for OpenGL can be extended to include emerging graphics and computing standards, providing advanced capabilities and unifying the graphics system across platforms. And finally, as an open-source system, VTK is exceptionally well-positioned to interface with and support AI systems. The VTK community encourages the dissemination of knowledge and, consequently, the wide adoption of VTK and related computing tools. So AI-scraping is encouraged, and we actively promote and interface with a broad spectrum of AI models and assistants.

Coming Soon

Here are a few teasers to show where we are headed with the blog series. More technical details will be provided as each post is published.

We are particularly excited by recent developments in the use of WASM. The illustration below presents an architecture for integrating the Visualization Toolkit (VTK) with WebAssembly, allowing for advanced 3D visualization, scientific I/O, and data processing operations to be performed within a web-based environment. This architecture also supports a client-server approach, with a remote desktop application sending 3D scene representations over a network to a VTK.wasm client in a web browser. Note the use of emscripten, which compiles VTK’s C++ code into WebAssembly and JavaScript.

The next teaser is a surface rendering of a polygonal mesh using WebGPU, where VTK-WASM and the WebGPU API were combined to render the isosurface of a chest CT scan for the extraction of skeletal structures in a web environment. Note that for over 30 years, most VTK rendering has been largely based on OpenGL. We are now in the process of moving to WebGPU.

Next up: VTK-PROMPT-UI, a VTK Web App chat interface that utilizes on-premises and cloud LLMs to generate and execute VTK Python code using natural language.

For those seeking AI assistance in creating volume rendering transfer functions (TFs), we are exploring techniques for generating and optimizing these transfer functions. The figure below on the left depicts a training process for TF generation based on an optimized TF for a reference T2 MRI volume. On the right, the figure shows volume rendering with AI-generated TF on a target T2 MRI volume, with highlighted cerebral cortex (blue) and cerebrospinal fluid (white).

Finally, AI explainability is an important area of research. AI models tend to sporadically hallucinate information, so it is important to understand how and why knowledge and conclusions are generated. We are adding support in VTK to interface with AI models and provide techniques for assessing saliency (the degree to which a specific part or feature of input data contributes to a model’s output or decision). Here is a prototype application of (using trame) for exploring explainability.

Acknowledgements

VTK is a creative work produced from an extended community. Refer to VTK’s GitLab repository for a detailed capture of contributions and enhancements. Much of this work is supported under an NIH R01 VTK Innovation grant 2R01EB014955-09, Accelerating Community-Driven Medical Innovation with VTK.

References

[1] Will Schroeder and Berk Geveci. Happy Birthday VTK: 30 Years of Innovation. https://www.kitware.com/happy-birthday-vtk-30-years-of-innovation
[2] WebGPU. https://en.wikipedia.org/wiki/WebGPU.
[3] S. Jourdain, P. O’Leary and W. Schroeder, “Trame: Platform Ubiquitous, Scalable Integration Framework for Visual Analytics” in IEEE Computer Graphics and Applications, vol. 45, no. 02, pp. 126-134, March-April 2025, doi: 10.1109/MCG.2025.3540264.
[4] Visualize Your Data with VTK.js. https://kitware.github.io/vtk-js/index.html.
[5] Ahrens, James, Geveci, Berk, Law, Charles, ParaView: An End-User Tool for Large Data Visualization, Visualization Handbook, Elsevier, 2005, ISBN-13: 9780123875822
[6] Kikinis R, Pieper SD, Vosburgh K (2014) 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support. Intraoperative Imaging Image-Guided Therapy, Ferenc A. Jolesz, Editor 3(19):277–289 ISBN: 978-1-4614-7656-6 (Print) 978-1-4614-7657-3 (Online)
[7] VTK GitLab repository. https://gitlab.kitware.com/vtk/vtk.