Using Virtual Reality Devices with VTK
Updated September 2016 to include Oculus Rift support
Virtual reality (VR) is a technological game changer. It allows us to look at and interact with data sets, models, and point clouds to give us a glimpse of what we can accomplish with our information. As hardware and rendering performance improve, the use of VR will only become more compelling.
With this in mind, we added two new modules into the Visualization Toolkit (VTK) to make it possible to use Oculus and OpenVR-compatible devices. The new modules vtkRenderingOculus and vtkRenderingOpenVR have been tested on the Oculus Rift and HTC Vive devices respectively.
To use these new modules, download the master branch of VTK from the VTK respository on GitHub. Before configuring VTK you should make sure you have downloaded and installed the Simple DirectMedia Layer 2 (SDL2) as the Oculus Rift and HTV Vive modules require it. If you are targeting the HTC Vive you will also need the OpenVR SDK, for the Oculus Rift you will need the Oculus Windows SDK . At this point you can configure VTK and turn on the modules you desire and start the build process. To maximize performance, make sure you build an optimized version of VTK.
Our goal is to allow VTK programs to use the Oculus and OpenVR modules with few changes, if any. If you link your executable to the vtkRenderingOpenVR module, the object factory mechanism should replace the core rendering classes (e.g., vtkRenderWindow) with the OpenVR-specialized versions in VTK. vtkRenderingOpenVR contains the following classes as drop-in replacements.
This class handles the bulk of interfacing to OpenVR. It currently supports one renderer that covers the entire window.
vtkOpenVRCamera gets the matrices from OpenVR to use for rendering. It contains a scale and translation that are designed to map world coordinates into the head-mounted display (HMD) space. Accordingly, you can keep world coordinates in the units that are best suited to your problem domain, and the camera will shift and scale the coordinates into the units that make sense for the HMD.
The vtkOpenVRRenderer class computes a reasonable scale and translation. It sets these computations on OpenVRCamera. It also sets an appropriate default clipping range expansion.
While VTK is designed to pick and interact based on X,Y mouse/window coordinates, OpenVR provides XYZ world coordinates and WXYZ orientations. vtkOpenVRRenderWindowInteractor catches controller events and converts them to mouse/window events. The class also stores the world coordinate positions and orientations for the styles or pickers that can use them. vtkOpenVRRenderWindowInteractor supports multiple controllers through the standard PointerIndex approach that VTK uses for MultiTouch.
These new Virtual Reality modules are currently still improving. Moving forward, we look to add support for the OpenVR overlay, which is great for displaying a user interface. We also aim to make the module faster and include more event interactions. Stay tuned to our blog for updates.
In the meantime, download VTK and test the modules for yourself. Let us know what you think by adding your experiences on the VTK mailing lists!