Unity の Google VR ビデオ非同期投影
ビデオ非同期投影 (Video Async Reprojection) とは何ですか？
Async Reprojection Video is a layer (referred to as an “external surface”) that an application can use to feed video frames directly into the async reprojection system. The main advantages to using the API are:
API を使用しない場合、ビデオは 1 度サンプリングされてアプリケーションの色のバッファにレンダリングされます。その後、色のバッファが再度サンプリングされて、歪み補正が行われます。これにより、ダブルサンプリングアーティファクトが発生します。外部のサーフェスはビデオを直接 EDS コンポジッターに渡します。サンプリングが 1 度だけのため、ビデオの視覚的な品質が向上します。
外部サーフェス API を使用する場合、ビデオフレームレートはアプリケーションのフレームレートから切り離されます。新しいフレームをレンダリングするのに 1 秒かかり、頭を動かすときに黒いバーが表示されるだけで、ビデオは正常に再生されます。これにより、ビデオフレームが欠けることが大幅に減り、AV 同期を維持します。
アプリケーションは、DRM ビデオの再生をマークすることができ、API は保護されたビデオを表示し、非同期投影のフレームレートを維持する保護されたパスを作成します。
既知の問題
ビデオ非同期投影を使用する場合、カメラは原点 (0、0、0) から始める必要があります。カメラの位置が 0,0,0 に設定されていない場合は、エラーが発生することがあります。
非同期投影のための、public にアクセス可能な C# インターフェースはありません。public の API は Java のみです。
ビデオ非同期投影を有効にする
ビデオ非同期投影は Daydream VR デバイス設定の一部です。
Click the grey arrow next to Daydream and then check the “Enable Video Surface” box to enable use of the Async Video Reprojection feature
Select the “Use Protected Memory”option ONLY if you require memory protection for all of your content as enabling this means that it is enabled for the lifetime of the application.
API ドキュメント
To take advantage of the Google VR API you will need to extend the UnityPlayerActivity. For more information, see documentation on Extending the UnityPlayerActivity
Because Java plug-ins cannot directly access Objects in your scene, you will need to provide a simple API to your C# code that will allow you to pass a transform to the Java side as well as to tell your Java code when to start rendering.
Note: This code is not complete. It contains no implementation of a video player as that is a client specific implementation detail. It also doesn’t have any playback controls, which would have to be implemented as objects in the scene and actions on those objects would need to call into Java.
Unity で Java を使用し UnityPlayerActivity を拡張する方法については、Unity の Android 開発 を参照してください。
For information about the Google Video Async Reprojection system, refer to the Android Developer Network documentation on Video Viewports.
Java sample code:
package com.unity3d.samplevideoplayer;
import com.unity3d.player.GoogleVrVideo;
import com.unity3d.player.GoogleVrApi;
import android.app.Activity;
import android.os.Bundle;
import android.util.Log;
import android.view.Surface;
public class GoogleAVRPlayer implements GoogleVrVideo.GoogleVrVideoCallbacks {
private static final String TAG = GoogleAVRPlayer.class.getSimpleName();
private MyOwnVideoPlayer videoPlayer;
private boolean canPlayVideo = false;
private boolean isSceneLoaded = false;
// API you present to your C# code to handle initialization of your
// video system.
public void initVideoPlayer(UnityPlayerActivity activity) {
// Initialize Video player and any other support you need…
// Register this instance as the Google Vr Video Listener to get
// lifetime and control callbacks.
GoogleVrVideo gvrv = GoogleVrApi.getGoogleVrVideo();
if (gvrv != null) gvrv.registerGoogleVrVideoListener(this);
}
// API you present to your C# code to start your video system
// playing a video.
public void play()
{
if (canPlayVideo && videoPlayer != null && videoPlayer.isPaused())
videoPlayer.play();
}
// API you present to your C# code to stop your video system
// playing a video
public void pause()
{
if (canPlayVideo && videoPlayer != null && !videoPlayer.isPaused())
videoPlayer.pause();
}
// Google Vr Video Listener
@Override
public void onSurfaceAvailable(Surface surface) {
// Google Vr has a surface available for you to render into.
// Use this surface with your video player as needed.
if (videoPlayer != null){
videoPlayer.setSurface(surface);
canPlayVideo = true;
if (isSceneLoaded)
{
videoPlayer.play();
}
}
}
@Override
public void onSurfaceUnavailable() {
// The Google Vr Video Surface is going away. You need to remove
// it from anything you have holding it and stop your video player.
if (videoPlayer != null){
videoPlayer.pause();
canPlayVideo = false;
}
}
@Override
public void onFrameAvailable() {
// Handle Google Vr frame available callback
}
}
Unity C# Sample code:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using UnityEngine;
public class GoogleVRVideo : MonoBehaviour {
private AndroidJavaObject googleAvrPlayer = null;
private AndroidJavaObject googleVrVideo = null;
void Awake()
{
if (googleAvrPlayer == null)
{
googleAvrPlayer = new AndroidJavaObject("com.unity3d.samplevideoplayer.GoogleAVRPlayer");
}
AndroidJavaObject googleVrApi = new AndroidJavaClass("com.unity3d.player.GoogleVrApi");
if (googleVrApi != null) googleVrVideo = googleVrApi.CallStatic<AndroidJavaObject>("getGoogleVrVideo");
}
void Start()
{
if (googleVrVideo != null)
{
// We need to tell Google VR the location of the video suface in
// world space. Since there isn't a way to get at that info from
// Java, we can do it here and then pass the calculated matrix
// down to the api we expose on our UnityPlayerActivity subclass.
Matrix4x4 wm = transform.localToWorldMatrix;
wm = Camera.main.transform.parent.worldToLocalMatrix * wm;
wm = wm * Matrix4x4.Scale(new Vector3(0.5f, 0.5f, 1));
// Convert 4x4 Row Ordered matrix into a 16 element column ordered
// flat array. The transposition is to make sure that the matrix is
// in the order that Google uses and the we flatten it to make passing
// over the JNI boundary easier. The complication being that you have to
// then convert it back to an 4x4 matrix on the Java side.
float[] matrix = new float[16];
for (int i = 0; i < 4; i++)
{
for (int j = 0; j < 4; j++)
{
matrix[i * 4 + j] = wm[j,i];
}
}
googleVrVideo.Call("setVideoLocationTransform", matrix);
}
if (googleAvrPlayer != null)
{
AndroidJavaClass jc = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
AndroidJavaObject jo = jc.GetStatic<AndroidJavaObject>("currentActivity");
googleAvrPlayer.Call("initVideoPlayer", jo);
googleAvrPlayer.Call("play");
}
}
}