Please note, AR/VR is currently only supported on Windows Desktop when using the C++ API and the DirectX 11 driver.
Beta Release: The approach detailed below is required for adding VR to your application with Visualize HPS 2018 SP1. In future releases we will provide a generic API to make the process much easier. For a reference implementation of VR in a Visualize application, you can refer to the OpenVR Sandbox, bundled with your copy of Visualize.
12.2 Virtual Reality
For detailed instructions on setting up AR/VR with Visualize, see the Getting Started section.
12.2.1 OpenVR
Visualize supports any VR headset compatible with OpenVR. In order to add VR to your application, the following items are required:
- An OpenVR compliant headset
- The OpenVR SDK, which can be downloaded from Valve Software at https://github.com/ValveSoftware/openvr
- SteamVR, which can be downloaded from Steam
1. Initializing OpenVR in Visualize
The following steps are required to initialize OpenVR:
2. Create an Update Handler.
Once OpenVR is initialized, and Visualize has started rendering in Stereo mode, we will need to set up an update handler.
Visualize needs to pass the images it renders in stereo mode to the VR headset at the end of each update. In order to do this, we will define a FinishPictureHandler. A FinishPictureHandler is a class which contains a callback called Handle. Handle is invoked automatically every time Visualize finishes an Update.
Here is a sample implementation of the FinishPictureHandler:
3. Set FinishPictureHandler on Window
The FinishPictureHandler will then need to be set as the handler on the window we created for VR:
4. Import Your Model into Visualize
At this point you are ready to import your model into Visualize. If you wish to you can create a Canvas / View / Model structure starting from the Offscreen window key you created for VR. This is also a good point to render the models of the VR devices to the scene graph, if you wish to do so.
You can find an example of how this can be done in the OpenVR Sandbox, specifically with the function SetupRenderModels().
5. Start a Render Loop
Now you are ready to start a render loop. The update at the end of the render loop should be the only one you have in the VR portion of your application. This render loop should continue until when you decide your application should quit, or otherwise terminate its VR mode.
Considerations
- Performance is the main concern when using VR. For a smooth experience, 60FPS should be the minimum requirement. Lower or jumpy frame rates can result in the user feeling sick. Because of this, the main render loop should be as bare-bones as possible. If you need to perform expensive calculations, it would be best to perform them on a separate thread.
-
When in VR mode you should expect that the camera is in constant movement, since it is tied to the position of the VR headset. As a consequence of this, some of the approaches used in highlighting and the default operators will not work in VR mode.
For example:
- Our default orbit operator works by modifying the camera position. Since the position of the camera depends on the VR headset, it will be necessary to change the model modelling matrix instead.
- Because of the constant camera changes, Visualize will not cache the status of rendering buffers between updates. In turn this means that the Overlay highlighting setting of Drawing::Overlay::WithZValues, which relies on these cached buffers, cannot be used effectively. Highlights with an overlay setting of Drawing::Overlay::WithZValues in an AR application will be performed sub-optimally.
- Currently the Hide operation, and in general Highlight operation with the InPlace Overlay settings are very expensive to use, performance-wise, and should be avoided in VR mode. This is something we are currently addressing for Visualize 2018 SP2.
- Some view-dependent geometry, like non-transformable text, patterned lines, text backgrounds, simple reflections/shadows, shadow maps, custom marker symbols, and leader lines will not transform correctly in stereo mode. This will create artifacts where it will look like two instances of said geometry are visible, unless the user closes one of her eyes. These limitations will be addressed in the next release.
- You can download the HOOPS Demonstration Viewer for a demo more fully fledged VR experience created with Visualize. You can find the VR mode button in the HDV Visual Effects tab.