Version: 2023.1
Experimental: this API is experimental and might be changed or removed in the future.

AudioSampleProvider

class in UnityEngine.Experimental.Audio

切换到手册

描述

提供对 Unity 对象(如 VideoPlayer)生成的音频样本的访问。

此类可在主线程之外使用,该实现的前提是一次只有一位消费者调用 AudioSampleProvider.ConsumeSampleFrames。通过 C# 和 C++ 均可访问音频数据,具体取决于此数据的用途。

以下示例显示了如何使用从 VideoPlayer 获取的访问器来访问 C# 中的样本:

using UnityEngine;
using Unity.Collections;
using UnityEngine.Experimental.Video;
using UnityEngine.Experimental.Audio;
using UnityEngine.Video;

public class ManagedAudioOutput : MonoBehaviour { AudioSampleProvider provider;

void Start() { VideoPlayer vp = GetComponent<VideoPlayer>(); vp.audioOutputMode = VideoAudioOutputMode.APIOnly; vp.prepareCompleted += Prepared; vp.Prepare(); }

void Prepared(VideoPlayer vp) { provider = vp.GetAudioSampleProvider(0); provider.sampleFramesAvailable += SampleFramesAvailable; provider.enableSampleFramesAvailableEvents = true; provider.freeSampleFrameCountLowThreshold = provider.maxSampleFrameCount / 4; vp.Play(); }

void SampleFramesAvailable(AudioSampleProvider provider, uint sampleFrameCount) { using (NativeArray<float> buffer = new NativeArray<float>( (int)sampleFrameCount * provider.channelCount, Allocator.Temp)) { var sfCount = provider.ConsumeSampleFrames(buffer); Debug.LogFormat("SetupSoftwareAudioOut.Available got {0} sample frames.", sfCount); // Do something with the samples here... } } }

以下是有关如何访问 C++ 中的样本的示例:设置必须在 C# 中完成,之后 Unity 的核心插件和原生插件无需借助托管代码即可相互调用。

using System;
using System.Runtime.InteropServices;
using UnityEngine;
using UnityEngine.Experimental.Video;
using UnityEngine.Experimental.Audio;
using UnityEngine.Video;

public class NativeAudioOutput : MonoBehaviour { AudioSampleProvider provider; AudioSampleProvider.SampleFramesEventNativeFunction sampleFramesAvailableNativeHandler = SampleFramesAvailable;

void Start() { VideoPlayer vp = GetComponent<VideoPlayer>(); vp.audioOutputMode = VideoAudioOutputMode.APIOnly; vp.prepareCompleted += Prepared; vp.Prepare(); }

void Prepared(VideoPlayer vp) { provider = vp.GetAudioSampleProvider(0); provider.freeSampleFrameCountLowThreshold = provider.maxSampleFrameCount - 1024;

SetConsumeSampleFramesFunction( AudioSampleProvider.consumeSampleFramesNativeFunction, provider.id, provider.channelCount, provider.sampleRate); provider.SetSampleFramesAvailableNativeHandler( sampleFramesAvailableNativeHandler, (IntPtr)0);

vp.Play(); }

private const string pluginName = #if UNITY_IPHONE "__Internal" #else "NativeAudioOutputPlugin" #endif ;

[DllImport(pluginName, ExactSpelling = true, CallingConvention = CallingConvention.Cdecl)] private static extern void SetConsumeSampleFramesFunction( AudioSampleProvider.ConsumeSampleFramesNativeFunction cb, uint id, ushort channelCount, uint sampleRate);

[AOT.MonoPInvokeCallback(typeof(AudioSampleProvider.SampleFramesEventNativeFunction))] [DllImport(pluginName, ExactSpelling = true, CallingConvention = CallingConvention.Cdecl)] private static extern void SampleFramesAvailable(IntPtr userData, uint id, uint sampleFrameCount); }

以下是随附的 C++ 插件:

#include <algorithm>
#include <stdint.h>

typedef uint32_t(__cdecl *ConsumeSampleFramesFunction)( uint32_t providerId, float* interleavedSampleFrames, uint32_t sampleFrameCount);

ConsumeSampleFramesFunction ConsumeSampleFrames = NULL; uint32_t providerId = -1; float* buffer = NULL; uint32_t bufferSampleFrameCount = 0; uint32_t availableSampleFrameCount = 0;

extern "C" __cdecl void SetConsumeSampleFramesFunction( ConsumeSampleFramesFunction function, uint32_t id, uint16_t channelCount, uint32_t sampleRate) { ConsumeSampleFrames = function; providerId = id; delete[] buffer; buffer = new float[channelCount * sampleRate]; // 1s worth of sample frames. bufferSampleFrameCount = sampleRate; }

extern "C" __cdecl void SampleFramesAvailable(void* userData, uint32_t id, uint32_t sampleFrameCount) { if (ConsumeSampleFrames == NULL) return;

// We consume the sample frames from the handler that tells us that there are some available. // But we could also invoke this regularly from another thread, for example the thread providing // samples to an audio device. const uint32_t consumedSampleFrameCount = ConsumeSampleFrames( providerId, buffer, std::min(bufferSampleFrameCount, sampleFrameCount)); // Do something with the samples here... }

静态变量

consumeSampleFramesNativeFunction指向原生函数(提供了对音频样本帧的访问)的指针。

变量

availableSampleFrameCount可与 AudioSampleProvider.ConsumeSampleFrames 结合使用的样本帧数。
channelCount每个样本帧的音频声道数。
enableSampleFramesAvailableEvents启用 AudioSampleProvider.sampleFramesAvailable 事件。
enableSilencePadding如果此属性为 true,且可用数据量少于请求数据量,则在静音时填充 ConsumeSampleFrames 生成的缓冲区。否则,缓冲区中的额外样本帧将保持不变。
freeSampleFrameCount样本生成器在溢出之前仍然可以写入的样本帧数。
freeSampleFrameCountLowThreshold然后,空闲样本数低于此阈值,这将发出 AudioSampleProvider.sampleFramesAvailable 事件和相关原生事件。
id此实例的唯一标识符。
maxSampleFrameCount发出溢出事件之前可在内部缓冲区内累积的最大样本帧数。
owner此提供程序来自的对象。
sampleRate此类生成的样本帧的预期播放速率。
trackIndex创建此提供程序的对象中的轨道索引。
valid如果对象有效,则此属性为 true。

公共函数

ClearSampleFramesAvailableNativeHandler清除通过 AudioSampleProvider.SetSampleFramesAvailableNativeHandler 设置的原生处理程序。
ClearSampleFramesOverflowNativeHandler清除通过 AudioSampleProvider.SetSampleFramesOverflowNativeHandler 设置的原生处理程序。
ConsumeSampleFrames使用内部缓冲区中的样本帧。
Dispose释放内部资源。继承自 IDisposable。
SetSampleFramesAvailableNativeHandler当可用的样本帧数超过阈值时,为发出的事件设置原生事件处理程序。
SetSampleFramesOverflowNativeHandler当内部样本帧缓冲区溢出时,为发出的事件设置原生事件处理程序。

Events

sampleFramesAvailable在可用样本帧数超过通过 AudioSampleProvider.freeSampleFrameCountLowThreshold 设置的阈值时调用。
sampleFramesOverflow在可用样本帧数超过内部缓冲区所允许的最大值时调用。

委托

ConsumeSampleFramesNativeFunction表示用于使用样本帧的原生函数指针的类型。
SampleFramesEventNativeFunction表示用于处理样本帧事件的原生函数指针的类型。
SampleFramesHandler样本帧事件的委托。