Recorded Testing

Recorded Playbacks (recordings) can be used to "drive" a Unity Test, combining the simplicity of recorded playback creation with the flexibility and power of the Unity Test Framework.

Automatic Recorded Test Creation

Once you have one or more recordings, use the Test Generation feature to quickly create Unity Tests that validate the proper playback of a recording. To Generate tests for each recording in the Assets/Recordings directory, navigate to Automated QA > Test Generation... (from the Unity menu bar).

Managing & Running Recorded Tests

Recorded Tests, like all Unity Tests, can be managed and run from the Unity Test Runner: navigate to Window > General > Test Runner and then select the PlayMode tab. From the Test Runner UI, Unity Tests can be run in the Unity Editor or on a local device (e.g. on a phone or tablet).

Running Recorded Tests on a Cloud Device Farm

Run Recorded Tests (and other Unity Tests) on iOS and Android devices managed by Unity. By default, every Unity organization has a free trial of 10 runs each run containing up to 10 devices per run. Email us at AutomatedQA@unity3d.com if you would like more free cloud device testing credits or with any questions.

[Required for Android Only] Configure your Project for Cloud Testing on Android Devices

  1. Open your project in the Unity Editor with Android as the build target
  2. Navigate to File > Build Settings and make sure Build App Bundle (Google Play) is not checked.
  3. Add a custom Android Manifest to your project to bypass any device permission dialogs that may occur during testing. Navigate to Edit > Project Settings... > Player > Publishing Settings and ensure Custom Main Manifest is checked.

Replace the source of the Android Manifest (/Assets/Plugins/Android/AndroidManifest.xml) with the below:

<?xml version="1.0" encoding="utf-8"?>
<!-- GENERATED BY UNITY. REMOVE THIS COMMENT TO PREVENT OVERWRITING WHEN EXPORTING AGAIN-->
<manifest
    xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.unity3d.player"
    xmlns:tools="http://schemas.android.com/tools">
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
    <application>
        <activity android:name="com.unity3d.player.UnityPlayerActivity"
                  android:theme="@style/UnityThemeSelector">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
            <meta-data android:name="unityplayer.UnityActivity" android:value="true" />
            <meta-data android:name="unityplayer.SkipPermissionsDialog" android:value="true" />
        </activity>
    </application>
</manifest>

Run Tests in the Cloud

Once you have one or more Recorded Tests (or any Unity Tests) that you'd like to run on a cloud iOS or Android device:

  1. Designate the Unity Tests that you would like to run in the cloud by adding [CloudTest] above the test definition. Note: this is done by default for any tests automatically generated from recorded playback.
  2. Navigate to Automated QA > Cloud Test Runner... and then press "Build & Upload". This will create a build bundled with all of your tests tagged with [CloudTest] and upload it to our cloud testing service.
  3. The Build ID should be automatically populated in the buildid field. Press "Run on Device Farm".
  4. The Job ID should be automatically populated in the jobid field. Press "Refresh" to check the status of your tests. Cloud tests run sequentially and in real time (i.e. timescale 1).
  5. Press "Get Test Results" to view the results of your tests in an HTML file.

Supported Devices

By default, we run tests on 5 of randomly selected iOS or Android devices (depending on your build target).

Please reach out to us at AutomatedQA@unity3d.com if you would like to run tests on specific [devices and operating systems.

Advanced Features

Manual Recorded Test Creation

Recorded Playbacks can be used "drive" any Play Mode UnityTest. To manually create a test driven by a recording, create a Unity Test and then add [RecordedTest("[relative path of recording (relative to /Assets)]")] right after the test's [UnityTest] attribute in the source code of the test.

For example, to validate that a tutorial loads as expected:

  1. Create a recorded playback that ends with the tutorial loading
  2. Rename the resulting recording-[timestamp].json file to something more descriptive like "load_tutorial.json"
  3. Create a new Unity Test with [RecordedTest("Recordings/load_tutorial.json")] below the [UnityTest] attribute
  4. Inside the Unity Test: a. Load the scene where your recording begins, e.g. SceneManager.LoadScene("0_Title"); b. Once playback is complete or some period of time has passed, Assert that some key condition is true

Example RecordedTest usage:

namespace Tests
{
    public class SmokeTests : RecordedTestSuite
    {
        [UnityTest]
        [RecordedTest("Recordings/load_tutorial.json.json")]
        public IEnumerator validate_tutorial_loads()
        {
           SceneManager.LoadScene("0_Title");
           ...

           //wait until the Tutorial scene loads
           while (SceneManager.GetActiveScene().name != "Tutorial")
           {
               yield return null;
           }

           Assert.NotNull(GameObject.Find("Tutorial_Manager");
        }

    }
}