Frame timing in virtual reality (VR) mode works in the same way as in VSync-enabled non-VR mode. For more information, refer to Execution order of event functions. However, in VR mode, Unity doesn’t depend on the underlying 3D SDK’s VSync. In VR mode, Unity depends on whichever VR SDK it currently renders with. There is no benefit to rendering faster than the display can refresh. Instead, Unity focuses on using the time most efficiently.
In multithreaded rendering, Unity synchronizes the simulation thread and the rendering thread on the CPU. This reduces latency. The simulation thread primarily executes the scripts, sounds, Artificial Intelligence, and other tasks needed for the simulation to run. The render thread focuses on submitting draw calls to the graphics driver, which in turn validates them and sends them to the GPU. To achieve the greatest parallelism between threads, Unity executes graphics commands as they come in, and simulates the next frame at the same time.
Unity waits for the GPU to completely render and display the last frame, then submits rendering commands for the next frame.
Before Unity can submit the first rendering command that depends on the view transformation matrix, it must first get the view matrix from the VR SDK. To keep latency as low as possible, the VR SDK predicts the head transform twice per frame:
One prediction to render the current frame. This prediction corresponds with where your head actually is, in real space, when the frame arrives on the screen.
One prediction to simulate the following frame.
Unity applies the rendering prediction for the current frame to cameras, controllers, and anything that needs information for scene rendering. Unity uses the simulation prediction for the following frame if it’s unable to render the following frame.
For more information on how the hardware handles rendering, refer to the manufacturer documentation for your head-mounted display.
If Unity doesn’t submit the fully rendered frame in time for it to be ready for the next display refresh, a few things might happen, depending on which VR SDK is active. The VR SDK might:
Display the previously submitted frame. This looks like judder, and reduces the quality of the experience.
Rotationally reproject the previously submitted frame based on the current head pose. This is a crude approximation that works well as a fallback for static content, but any content with animation or positional movement doesn’t look correct.
Apply some form of positional reprojection. This might be temporal to fill in missing details.
Note: Reprojection is sometimes called time warping.
This all happens automatically. At this point, it’s usually too late for Unity to successfully finish the next frame that it should already be rendering. Instead, the VR SDK takes the last frame it received and reprojects it to the latest predicted pose, and Unity waits until the next opportunity to render a full frame to get back on track. If your app continually drops frames, Unity renders only every other frame.
For more information on how the hardware handles rendering, refer to the manufacturer documentation for your head-mounted display.
Each display has a specific refresh rate, and Unity is bound to it. You can get the refresh rate at runtime with XRDisplaySubsystem.displayRefreshRate. Divide 1 by displayRefreshRate to calculate the time allocated to a single frame.
For example, this might be 11.1 milliseconds for a refresh rate of 90 Hz.
If your app is GPU-bound, the Unity Profiler displays XR.WaitForGPU at more than one frame’s time in milliseconds.
If your app is CPU-bound, a frame takes longer than the designated frame time, but the Unity Profiler displays XR.WaitForGPU shorter than the one frame.