RenderManager and other developer updates

Here is a brief update on new repositories and features and major enhancements for the OSVR software platform.

Highlights

  • OSVR RenderManager is now open-sourced
  • New documentation repository as well as global search from osvr.github.io
  • Dozens of new devices
  • Support for latest Unity and Unreal versions including new samples

Details below

OSVR-Docs

A new documentation repository saves as the gateway for developer documentation, tips and tricks, configuration instructions and a whole lot more. Please visit http://github.com/OSVR/OSVR-Docs

Because this is an open-source repository, we encourage you to contribute to it and share your experience.

sensics/OSVR-RenderManager

RenderManager was previously distributed as closed-source but now it has been split into an open-source repository that performs most of the functions and two closed-source repositories (one for NVIDIA, one for AMD) that require individual NDAs with NVIDIA or AMD

What RenderManager Provides?

RenderManager provides a number of functions beyond the OSVR-Core library in support of VR rendering. It wraps the Core functions in an easy-to-use interface that implements many VR-specific needs.

  • DirectMode: On platforms that support it, RenderManager implements direct rendering to the display, bypassing operating-system delays and enabling front- buffer rendering. On Windows, this is implemented using nVidia’s VR Direct Mode rendering and AMD’s Direct-to-Display rendering. These share a common interface in RenderManager and plans are underway to extend these to new operating systems as they become available. DirectMode supports both D3D11 and OpenGL (core and legacy) on Windows.

The following capabilities are provided on all supported platforms:

  • Distortion correction: This enables undistortion of HMD lenses. Configuration files can be used to specify the type of distortion correction used, from several choices: rgb polynomial away from a center, monochromatic unstructured mesh, and rgb unstructured mesh. RenderManager includes distortion-mesh-construction for all of its rendering libraries based on all of the above input types. See RenderManager.h for more information.
  • Time Warp: Synchronous time warp is provided on all platforms. This is done at the same time as the distortion-correction render pass by reading the latest tracking information and adjusting the viewing transformation using the texture matrix to fix up changes due to motion between the start of rendering and its completion. This warping is geometrically correct for strict rotations around the center of projection and is approximated by a 2-meter distance for translations.
  • Rendering state: RenderManager produces graphics-language-specific conversion functions to describe the number and size of required textures, the viewports, projection and ModelView matrices needed to configure rendering for scenes. Configuration files specify the number of eyes, whether they are in a single screen or multiple screens, and their relative orientations. RenderManager takes all viewports and textures in their canonical (up is up) orientation and internally maps to the correct orientation, enabling the use of bitmap fonts and other rendering effects that require canonical orientation. An optional, callback-based rendering path provides these transformations for arbitrary spaces within the OSVR configuration space (head space, hand space, room space, etc.).
  • Window creation: RenderManager uses SDL on Windows, Linux, and Mac systems to construct windows of the appropriate size for a given HMD or on-screen display. Configuration file entries describe window size, placement, and orientation. For non-DirectMode operation, these show up within the operating virtual screen and can be either full-screen or windowed. For DirectMode operation, they provide full- screen operation on one or more displays.
  • OverFill & Oversampling: To enable time warp to work, the rendered view must be larger than the image to be presented on a given frame. This provides a border around the image that can be pulled in as the user’s viewport rotates. Also, the distortion caused by lenses in VR systems can cause a magnification of the screen that requires the application to render pixels at a higher density than the physical display. RenderManager handles both of these capabilities internally, hiding them from the application. Configuration file entries can adjust these; trading rendering speed for performance at run time without changes to the code.

Coming Soon

  • Asynchronous Time Warp is under development as of 2/15/2016. There is a single D3D11 example program that runs on DirectMode displays under Windows. This capability is not yet fully operational (the example program does not work when run without ATW enabled, and there are several open Github issues). When complete, this mode will be enabled by a configuration-file setting. It produces a separate rendering thread that re-warps and re-renders images at full rate even when the application renders too slowly to present a new image each frame.
  • Android support is under development. As of 2/15/2016, the OpenGL internal code is all compatible with OpenGL ES 2.0. Work is underway to port RenderManager to Android on top of the existing OSVR-Core port.
  • DirectMode/Linux is planned as graphics-card vendors finish drivers to enable it on this platform. It is being designed to use the same RenderManager interface and configuration files as the current Windows implementations.

Two RenderManager Interfaces

RenderManager provides two different interfaces, a Get/Present interface and a Callback interface. Example applications are provided that use each. The Callback interface provides the ability to easily render objects in multiple spaces (head space, hand space, etc.). The Get/Present interface lets the application have complete control over render-buffer construction.

Example programs

There are a number of example programs that highlight the different RenderManager interfaces and features.

Key features added to RenderManager

  • Added DirectMode (“Direct to Display” or D2D) support on AMD GPUs for both Direct3D11 and OpenGL on Windows through AMD LiquidVR for headset manufacturers SDK. (NDA component distributed as binary only)
  • Asynchronous Time Warp (currently only in a single example program)
  • Code compiles and links on OS X (still needs fixing for OpenGL/Core)
  • DirectMode works on nVidia cards co-installed with an Intel card
  • DirectModeDebug tool added for nVidia DirectMode
  • Code compiles and runs on Linux with all non-DirectMode features

OSVR-Core

Primarily User-Facing Changes

  • Windows builds now have the osvr_server_config.HDK13DirectMode.sample.json file copied to be their default osvr_server_config.json config file, enabling video-based tracking with IMU fusion and direct mode by default.
  • Server mainloop will now reduce CPU usage when no clients are connected.

Primarily Developer and Non-Windows Changes

  • Added cross-platform shared export header for libraries.
  • New analysis plugin: generic orientation predictive tracking, can be configured with any orientation device that reports angular velocity as well as input.
  • Fixed behavior of state interfaces when timestamps are non-monotonic: for instance, when replaying a loop of motion-capture data.
  • Dozens of new devices added. See http://osvr.github.io/compatibility/ for the full list
  • ​Build-system compatibility with OpenCV 3.1. (Windows binaries are still shipped with OpenCV 2.4.10 by default.)
  • Updates to support additional versions of Boost (1.60) for compilation.
  • Fix automated CI build and pull-request build testing on Travis-CI (building Linux and Mac).
  • New utility shipped: osvr_list_usbserial lists VID, PID, and platform-specific path for each connected USB-Serial device, primarily useful in development. (Uses the osvrUSBSerial library, intended for use by OSVR plugins: so you can embed this capability in your plugin.)
  • Experimental single-filter IMU and video-based tracker plugin merged. Not ready for general usage, but if you’re interested in tracking technology or algorithms, contributions appreciated.

OSVR-HDK-Windows-Drivers

Release 1.2.6  – Recommended upgrade for all HDK users

  • A new metadata package has been added for the HDK 1.3 beltbox audio.

versioned-beltboxes-in-devices-and-printers

  • This fixes issue #7 for Windows 7 users.
  • Since we can distinguish the beltbox shipped with the 1.3 from that shipped with the 1.2 (and earlier), the names for the devices have been updated to include an HDK version number, which may be convenient for all users. Below is a screen capture from “Devices and Printers” on a machine with both a HDK 1.2 and HDK 1.3 plugged in.
  • Similarly, a new device metadata package has been added to warn users if they have an outdated firmware version on their infrared camera, since this can seriously impact tracking performance. Right-clicking on such a device will reveal a menu entry to bring you to a page where you can download a firmware upgrade tool. If you want to manually upgrade the firmware on your IR camera, use the firmware update tool from http://osvr.github.io/using/

camera-needs-updatecamera-context-menu

  • The major version numbers on some of the “cosmetic” INF file drivers (currently that’s display, HID, and camera) have been bumped to 10 to pre-empt the basic in-box Windows 10 drivers. This isn’t critical, since they are cosmetic only (they just rename devices in the Device Manager), but it is nice to have.

 

OSVR-Unreal

  • The new OSVRInput module now implements the standard Unreal controller and motion controller interfaces. This module implements the traditional controller button interface as well as the split motion controller interface for both buttons and hand tracking.
  • VR Preview play mode in the Unreal editor now renders the scene into your HMD in direct mode.
  • OSVR DLLs are loaded dynamically at runtime from the plugin binary directory. The plugin will also search in multiple locations for the dlls, including the engine’s third party binaries directory. Previously, these DLLs were copied to the executable directory.
  • Updates for Unreal 4.11. The plugin still works with 4.10.
  • Replaced absolute paths in the build files with engine-relative paths.
  • Breaking changes:
    • All of the original OSVR blueprint API has been deprecated. Due to design issues with the original API, we’ve decided to take a second look and create a more general purpose API. In the meantime, if you are relying on the existing blueprint API, there are instructions in DOCUMENTATION.md for re-enabling the API. We do not recommend using the API for new projects.
    • Note: if you were using the original blueprint API for hand-tracking or controller buttons, please take a look at the new native controller and motion controller support. You can now access this functionality from your blueprints with the built-in blueprint events and functions from Unreal.

SteamVR-OSVR

  • Updated for SteamVR-OpenVR 0.9.15
  • Special thanks to community contributor @d235j on github for this update.

OSVR-Unity

OSVR-user-settings

A simple QT utility to help users set up their configuration parameters

Chat Rooms, Videos and other Resources

We’ve set up developer chat rooms at Gitter, and they are tied to the GitHub projects.

The list of all rooms is here: https://gitter.im/orgs/OSVR/rooms

Some existing rooms:

Ever-growing list of supported devices, operating systems and engines is here

New presentation from Unity Vision Summit 2016 on using OSVR to connect (practically) any device to AR/VR is here

Want to Participate?

Interested in contributing? Start here: http://osvr.github.io/contributing/