Video Streaming

WebRTC enables streaming video between peers. It can stream video rendered by Unity to multiple browsers at the same time.

Codec

There are two types of encoder for video streaming, one is using hardware for encoding and one is using software. Regarding different kinds of codecs, the hardware encoder uses H.264, and the software encoder uses VP8.

We can select the type of encoder by specifying the EncoderType in WebRTC.Initialize's method argument.

// Use a software encoder
WebRTC.Initialize(EncoderType.Software);

Note

This option selects whether or not to use hardware for encoding. Currently, there is no way to explicitly designate a codec.

The major browsers that support WebRTC can use H.264 and VP8, which means most browsers can receive video streaming from Unity.

Video Track

To implement video streaming, create a VideoStreamTrack instance.

// Create a track from the Camera
var camera = GetComponnent<Camera>();
var track = camera.CaptureStreamTrack(1280, 720);

There is also a way to directly assign a RenderTexture.

// Get a valid RendertextureFormat
var gfxType = SystemInfo.graphicsDeviceType;
var format = WebRTC.GetSupportedRenderTextureFormat(gfxType);

// Create a track from the RenderTexture
var rt = new RenderTexture(width, height, 0, format);
var track = new VideoStreamTrack("video", renderTexture);

Add Track

Add the created video track to the PeerConnection instance. The track can be added by calling the AddTrack method. Next, call the PeerConnection's CreateOffer or CreateAnswer to create an SDP.

// Add the track
peerConnection.AddTrack(track);

// Create the SDP
RTCAnswerOptions options = default;
var op = pc.CreateAnswer(ref options);
yield return op;

Multi track

It's possible to use multiple video tracks simultaneously. Simply call the PeerConnection's AddTrack method multiple times and add the tracks.

foreach(var track in listTrack)
{
    peerConnection.AddTrack(track);
}

When using hardware encoding, the number of tracks that can be used simultaneously may be limited depending on the graphic device's limitations. Generally, on desktop GPUs, up to two tracks can be used simultaneously on an NVIDIA Geforce card (On server-grade GPUs this is typically 4). For details, see the NVIDIA Codec SDK documentation.

See the section on Streamless tracks under PeerConnection.addTrack in the MDN documentation for information on simultaneously receiving multiple tracks in the browser.

Bitrate control

To control the bitrate of video streaming, use SetParameter method of RTCRtpSender instance. The instance of RTCRtpSender is obtained from RTCPeerConnection.

var senders = peerConnection.GetSenders();

Or, obtained from AddTrack method as its return value.

var sender = peerConnection.AddTrack(track);

After obtained the instance of RTCRtpSender, To get the settings about the sending stream, call the GetParameter method is able. And call the SetParameter method with customized settings. as a result, the settings are reflected.

var parameters = sender.GetParameters();
foreach (var encoding in parameters.Encodings)
{
    encoding.maxBitrate = bitrate;
}
sender.SetParameters(parameters);

Note

Currently not supported maxFramerate in values of the settings. Also, scaleResolutionDownBy parameter only works on software encoder.

It is possible to check the current bitrate on browsers. If using Google Chrome, shows statistics of WebRTC by accessing the URL chrome://webrtc-internals. Check the graph showing the received bytes per unit time in the category RTCInboundRTPVideoStream of statistics.