class in UnityEngine.Experimental.Audio
/
Implemented in:UnityEngine.AudioModule
Thank you for helping us improve the quality of Unity Documentation. Although we cannot accept all submissions, we do read each suggested change from our users and will make updates where applicable.
CloseFor some reason your suggested change could not be submitted. Please <a>try again</a> in a few minutes. And thank you for taking the time to help us improve the quality of Unity Documentation.
CloseProvides access to the audio samples generated by Unity objects such as VideoPlayer.
This class is meant to be usable outside the main thread and the implementation assumes a single consumer at a time invoking AudioSampleProvider.ConsumeSampleFrames. Audio data can be accessed both from C# and from C++, depending on what has to be done with this data.
The following is an example of how to access samples from C# using an accessor obtained from the VideoPlayer:
using UnityEngine; using Unity.Collections; using UnityEngine.Experimental.Video; using UnityEngine.Experimental.Audio; using UnityEngine.Video;
public class ManagedAudioOutput : MonoBehaviour { AudioSampleProvider provider;
void Start() { VideoPlayer vp = GetComponent<VideoPlayer>(); vp.audioOutputMode = VideoAudioOutputMode.APIOnly; vp.prepareCompleted += Prepared; vp.Prepare(); }
void Prepared(VideoPlayer vp) { provider = vp.GetAudioSampleProvider(0); provider.sampleFramesAvailable += SampleFramesAvailable; provider.enableSampleFramesAvailableEvents = true; provider.freeSampleFrameCountLowThreshold = provider.maxSampleFrameCount / 4; vp.Play(); }
void SampleFramesAvailable(AudioSampleProvider provider, uint sampleFrameCount) { using (NativeArray<float> buffer = new NativeArray<float>( (int)sampleFrameCount * provider.channelCount, Allocator.Temp)) { var sfCount = provider.ConsumeSampleFrames(buffer); Debug.LogFormat("SetupSoftwareAudioOut.Available got {0} sample frames.", sfCount); // Do something with the samples here... } } }
The following is an example of how to access samples from C++. The setup has to be done in C# and then Unity's core and the native plug-in can call into each other without having to go through managed code.
using System; using System.Runtime.InteropServices; using UnityEngine; using UnityEngine.Experimental.Video; using UnityEngine.Experimental.Audio; using UnityEngine.Video;
public class NativeAudioOutput : MonoBehaviour { AudioSampleProvider provider; AudioSampleProvider.SampleFramesEventNativeFunction sampleFramesAvailableNativeHandler = SampleFramesAvailable;
void Start() { VideoPlayer vp = GetComponent<VideoPlayer>(); vp.audioOutputMode = VideoAudioOutputMode.APIOnly; vp.prepareCompleted += Prepared; vp.Prepare(); }
void Prepared(VideoPlayer vp) { provider = vp.GetAudioSampleProvider(0); provider.freeSampleFrameCountLowThreshold = provider.maxSampleFrameCount - 1024;
SetConsumeSampleFramesFunction( AudioSampleProvider.consumeSampleFramesNativeFunction, provider.id, provider.channelCount, provider.sampleRate); provider.SetSampleFramesAvailableNativeHandler( sampleFramesAvailableNativeHandler, (IntPtr)0);
vp.Play(); }
private const string pluginName = #if UNITY_IPHONE "__Internal" #else "NativeAudioOutputPlugin" #endif ;
[DllImport(pluginName, ExactSpelling = true, CallingConvention = CallingConvention.Cdecl)] private static extern void SetConsumeSampleFramesFunction( AudioSampleProvider.ConsumeSampleFramesNativeFunction cb, uint id, ushort channelCount, uint sampleRate);
[AOT.MonoPInvokeCallback(typeof(AudioSampleProvider.SampleFramesEventNativeFunction))] [DllImport(pluginName, ExactSpelling = true, CallingConvention = CallingConvention.Cdecl)] private static extern void SampleFramesAvailable(IntPtr userData, uint id, uint sampleFrameCount); }
And here is the accompanying C++ plug-in:
#include <algorithm> #include <stdint.h>
typedef uint32_t(__cdecl *ConsumeSampleFramesFunction)( uint32_t providerId, float* interleavedSampleFrames, uint32_t sampleFrameCount);
ConsumeSampleFramesFunction ConsumeSampleFrames = NULL; uint32_t providerId = -1; float* buffer = NULL; uint32_t bufferSampleFrameCount = 0; uint32_t availableSampleFrameCount = 0;
extern "C" __cdecl void SetConsumeSampleFramesFunction( ConsumeSampleFramesFunction function, uint32_t id, uint16_t channelCount, uint32_t sampleRate) { ConsumeSampleFrames = function; providerId = id; delete[] buffer; buffer = new float[channelCount * sampleRate]; // 1s worth of sample frames. bufferSampleFrameCount = sampleRate; }
extern "C" __cdecl void SampleFramesAvailable(void* userData, uint32_t id, uint32_t sampleFrameCount) { if (ConsumeSampleFrames == NULL) return;
// We consume the sample frames from the handler that tells us that there are some available. // But we could also invoke this regularly from another thread, for example the thread providing // samples to an audio device. const uint32_t consumedSampleFrameCount = ConsumeSampleFrames( providerId, buffer, std::min(bufferSampleFrameCount, sampleFrameCount)); // Do something with the samples here... }
consumeSampleFramesNativeFunction | Pointer to the native function that provides access to audio sample frames. |
availableSampleFrameCount | Number of sample frames available for consuming with AudioSampleProvider.ConsumeSampleFrames. |
channelCount | The number of audio channels per sample frame. |
enableSampleFramesAvailableEvents | Enables the AudioSampleProvider.sampleFramesAvailable events. |
enableSilencePadding | If true, buffers produced by ConsumeSampleFrames will get padded when silence if there are less available than asked for. Otherwise, the extra sample frames in the buffer will be left unchanged. |
freeSampleFrameCount | Number of sample frames that can still be written to by the sample producer before overflowing. |
freeSampleFrameCountLowThreshold | Then the free sample count falls below this threshold, the AudioSampleProvider.sampleFramesAvailable event and associated native is emitted. |
id | Unique identifier for this instance. |
maxSampleFrameCount | The maximum number of sample frames that can be accumulated inside the internal buffer before an overflow event is emitted. |
owner | Object where this provider came from. |
sampleRate | The expected playback rate for the sample frames produced by this class. |
trackIndex | Index of the track in the object that created this provider. |
valid | True if the object is valid. |
ClearSampleFramesAvailableNativeHandler | Clear the native handler set with AudioSampleProvider.SetSampleFramesAvailableNativeHandler. |
ClearSampleFramesOverflowNativeHandler | Clear the native handler set with AudioSampleProvider.SetSampleFramesOverflowNativeHandler. |
ConsumeSampleFrames | Consume sample frames from the internal buffer. |
Dispose | Release internal resources. Inherited from IDisposable. |
SetSampleFramesAvailableNativeHandler | Set the native event handler for events emitted when the number of available sample frames crosses the threshold. |
SetSampleFramesOverflowNativeHandler | Set the native event handler for events emitted when the internal sample frame buffer overflows. |
sampleFramesAvailable | Invoked when the number of available sample frames goes beyond the threshold set with AudioSampleProvider.freeSampleFrameCountLowThreshold. |
sampleFramesOverflow | Invoked when the number of available sample frames goes beyond the maximum that fits in the internal buffer. |
ConsumeSampleFramesNativeFunction | Type that represents the native function pointer for consuming sample frames. |
SampleFramesEventNativeFunction | Type that represents the native function pointer for handling sample frame events. |
SampleFramesHandler | Delegate for sample frame events. |
Did you find this page useful? Please give it a rating:
Thanks for rating this page!
What kind of problem would you like to report?
Thanks for letting us know! This page has been marked for review based on your feedback.
If you have time, you can provide more information to help us fix the problem faster.
Provide more information
You've told us this page needs code samples. If you'd like to help us further, you could provide a code sample, or tell us about what kind of code sample you'd like to see:
You've told us there are code samples on this page which don't work. If you know how to fix it, or have something better we could use instead, please let us know:
You've told us there is information missing from this page. Please tell us more about what's missing:
You've told us there is incorrect information on this page. If you know what we should change to make it correct, please tell us:
You've told us this page has unclear or confusing information. Please tell us more about what you found unclear or confusing, or let us know how we could make it clearer:
You've told us there is a spelling or grammar error on this page. Please tell us what's wrong:
You've told us this page has a problem. Please tell us more about what's wrong:
Thank you for helping to make the Unity documentation better!
Your feedback has been submitted as a ticket for our documentation team to review.
We are not able to reply to every ticket submitted.
When you visit any website, it may store or retrieve information on your browser, mostly in the form of cookies. This information might be about you, your preferences or your device and is mostly used to make the site work as you expect it to. The information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies. Click on the different category headings to find out more and change our default settings. However, blocking some types of cookies may impact your experience of the site and the services we are able to offer.
More information
These cookies enable the website to provide enhanced functionality and personalisation. They may be set by us or by third party providers whose services we have added to our pages. If you do not allow these cookies then some or all of these services may not function properly.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site. They help us to know which pages are the most and least popular and see how visitors move around the site. All information these cookies collect is aggregated and therefore anonymous. If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
These cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising. Some 3rd party video providers do not allow video views without targeting cookies. If you are experiencing difficulty viewing a video, you will need to set your cookie preferences for targeting to yes if you wish to view videos from these providers. Unity does not control this.
These cookies are necessary for the website to function and cannot be switched off in our systems. They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences, logging in or filling in forms. You can set your browser to block or alert you about these cookies, but some parts of the site will not then work. These cookies do not store any personally identifiable information.