Version: 2018.1
Experimental: this API is experimental and might be changed or removed in the future.

AudioSampleProvider

class in UnityEngine.Experimental.Audio

Switch to Manual

Description

Provides access to the audio samples generated by Unity objects such as VideoPlayer.

This class is meant to be usable outside the main thread and the implementation assumes a single consumer at a time invoking AudioSampleProvider.ConsumeSampleFrames. Audio data can be accessed both from C# and from C++, depending on what has to be done with this data.

The following is an example of how to access samples from C# using an accessor obtained from the VideoPlayer:

using UnityEngine;
using Unity.Collections;
using UnityEngine.Experimental.Video;
using UnityEngine.Experimental.Audio;
using UnityEngine.Video;

public class ManagedAudioOutput : MonoBehaviour { AudioSampleProvider provider;

void Start() { VideoPlayer vp = GetComponent<VideoPlayer>(); vp.audioOutputMode = VideoAudioOutputMode.APIOnly; vp.prepareCompleted += Prepared; vp.Prepare(); }

void Prepared(VideoPlayer vp) { provider = vp.GetAudioSampleProvider(0); provider.sampleFramesAvailable += SampleFramesAvailable; provider.enableSampleFramesAvailableEvents = true; provider.freeSampleFrameCountLowThreshold = provider.maxSampleFrameCount / 4; vp.Play(); }

void SampleFramesAvailable(AudioSampleProvider provider, uint sampleFrameCount) { using (NativeArray<float> buffer = new NativeArray<float>( (int)sampleFrameCount * provider.channelCount, Allocator.Temp)) { var sfCount = provider.ConsumeSampleFrames(buffer); Debug.LogFormat("SetupSoftwareAudioOut.Available got {0} sample frames.", sfCount); // Do something with the samples here... } } }

The following is an example of how to access samples from C++. The setup has to be done in C# and then Unity's core and the native plug-in can call into each other without having to go through managed code.

using System;
using System.Runtime.InteropServices;
using UnityEngine;
using UnityEngine.Experimental.Video;
using UnityEngine.Experimental.Audio;
using UnityEngine.Video;

public class NativeAudioOutput : MonoBehaviour { AudioSampleProvider provider; AudioSampleProvider.SampleFramesEventNativeFunction sampleFramesAvailableNativeHandler = SampleFramesAvailable;

void Start() { VideoPlayer vp = GetComponent<VideoPlayer>(); vp.audioOutputMode = VideoAudioOutputMode.APIOnly; vp.prepareCompleted += Prepared; vp.Prepare(); }

void Prepared(VideoPlayer vp) { provider = vp.GetAudioSampleProvider(0); provider.freeSampleFrameCountLowThreshold = provider.maxSampleFrameCount - 1024;

SetConsumeSampleFramesFunction( AudioSampleProvider.consumeSampleFramesNativeFunction, provider.id, provider.channelCount, provider.sampleRate); provider.SetSampleFramesAvailableNativeHandler( sampleFramesAvailableNativeHandler, (IntPtr)0);

vp.Play(); }

private const string pluginName = #if UNITY_IPHONE "__Internal" #else "NativeAudioOutputPlugin" #endif ;

[DllImport(pluginName, ExactSpelling = true, CallingConvention = CallingConvention.Cdecl)] private static extern void SetConsumeSampleFramesFunction( AudioSampleProvider.ConsumeSampleFramesNativeFunction cb, uint id, ushort channelCount, uint sampleRate);

[AOT.MonoPInvokeCallback(typeof(AudioSampleProvider.SampleFramesEventNativeFunction))] [DllImport(pluginName, ExactSpelling = true, CallingConvention = CallingConvention.Cdecl)] private static extern void SampleFramesAvailable(IntPtr userData, uint id, uint sampleFrameCount); }

And here is the accompanying C++ plug-in:

#include <algorithm>
#include <stdint.h>

typedef uint32_t(__cdecl *ConsumeSampleFramesFunction)( uint32_t providerId, float* interleavedSampleFrames, uint32_t sampleFrameCount);

ConsumeSampleFramesFunction ConsumeSampleFrames = NULL; uint32_t providerId = -1; float* buffer = NULL; uint32_t bufferSampleFrameCount = 0; uint32_t availableSampleFrameCount = 0;

extern "C" __cdecl void SetConsumeSampleFramesFunction( ConsumeSampleFramesFunction function, uint32_t id, uint16_t channelCount, uint32_t sampleRate) { ConsumeSampleFrames = function; providerId = id; delete[] buffer; buffer = new float[channelCount * sampleRate]; // 1s worth of sample frames. bufferSampleFrameCount = sampleRate; }

extern "C" __cdecl void SampleFramesAvailable(void* userData, uint32_t id, uint32_t sampleFrameCount) { if (ConsumeSampleFrames == NULL) return;

// We consume the sample frames from the handler that tells us that there are some available. // But we could also invoke this regularly from another thread, for example the thread providing // samples to an audio device. const uint32_t consumedSampleFrameCount = ConsumeSampleFrames( providerId, buffer, std::min(bufferSampleFrameCount, sampleFrameCount)); // Do something with the samples here... }

Static Variables

consumeSampleFramesNativeFunctionPointer to the native function that provides access to audio sample frames.

Variables

availableSampleFrameCountNumber of sample frames available for consuming with AudioSampleProvider.ConsumeSampleFrames.
channelCountThe number of audio channels per sample frame.
enableSampleFramesAvailableEventsEnables the AudioSampleProvider.sampleFramesAvailable events.
enableSilencePaddingIf true, buffers produced by ConsumeSampleFrames will get padded when silence if there are less available than asked for. Otherwise, the extra sample frames in the buffer will be left unchanged.
freeSampleFrameCountNumber of sample frames that can still be written to by the sample producer before overflowing.
freeSampleFrameCountLowThresholdThen the free sample count falls below this threshold, the AudioSampleProvider.sampleFramesAvailable event and associated native is emitted.
idUnique identifier for this instance.
maxSampleFrameCountThe maximum number of sample frames that can be accumulated inside the internal buffer before an overflow event is emitted.
ownerObject where this provider came from.
sampleRateThe expected playback rate for the sample frames produced by this class.
trackIndexIndex of the track in the object that created this provider.
validTrue if the object is valid.

Public Functions

ClearSampleFramesAvailableNativeHandlerClear the native handler set with AudioSampleProvider.SetSampleFramesAvailableNativeHandler.
ClearSampleFramesOverflowNativeHandlerClear the native handler set with AudioSampleProvider.SetSampleFramesOverflowNativeHandler.
ConsumeSampleFramesConsume sample frames from the internal buffer.
DisposeRelease internal resources. Inherited from IDisposable.
SetSampleFramesAvailableNativeHandlerSet the native event handler for events emitted when the number of available sample frames crosses the threshold.
SetSampleFramesOverflowNativeHandlerSet the native event handler for events emitted when the internal sample frame buffer overflows.

Events

sampleFramesAvailableInvoked when the number of available sample frames goes beyond the threshold set with AudioSampleProvider.freeSampleFrameCountLowThreshold.
sampleFramesOverflowInvoked when the number of available sample frames goes beyond the maximum that fits in the internal buffer.

Delegates

ConsumeSampleFramesNativeFunctionType that represents the native function pointer for consuming sample frames.
SampleFramesEventNativeFunctionType that represents the native function pointer for handling sample frame events.
SampleFramesHandlerDelegate for sample frame events.