Recorded Testing

Recorded Playbacks (recordings) can be used to "drive" a Unity Test, combining the simplicity of recorded playback creation with the flexibility and power of the Unity Test Framework.

Automatic Recorded Test Creation

Once you have one or more recordings, use the Test Generation feature to quickly create Unity Tests that validate the proper playback of a recording. To Generate tests for each recording in the Assets/Recordings directory, navigate to Automated QA > Generate Recorded Tests (from the Unity menu bar).

Managing & Running Recorded Tests

Recorded Tests, like all Unity Tests, can be managed and run from the Unity Test Runner: navigate to Window > General > Test Runner and then select the PlayMode tab. From the Test Runner UI, Unity Tests can be run in the Unity Editor or on a local device (e.g. on a phone or tablet).

[Opt In Required for Access] Running Recorded Tests on a Cloud Device Farm

Run Recorded Tests (and other Unity Tests) on Android Phones hosted on AWS Device Farm - no AWS account needed.

Email us at a AutomatedTesting@unity3d.com for access!

Configure your Project for Cloud Testing

  1. Open your project in the Unity Editor with Android as the build target
  2. Navigate to File > Build Settings and make sure Build App Bundle (Google Play) is not checked.
  3. Add a custom Android Manifest to your project to bypass any device permission dialogs that may occur during testing. Navigate to Edit > Project Settings... > Player > Publishing Settings and ensure Custom Main Manifest is checked.

Replace the source of the Android Manifest (/Assets/Plugins/Android/AndroidManifest.xml) with the below:

<?xml version="1.0" encoding="utf-8"?>
<!-- GENERATED BY UNITY. REMOVE THIS COMMENT TO PREVENT OVERWRITING WHEN EXPORTING AGAIN-->
<manifest
    xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.unity3d.player"
    xmlns:tools="http://schemas.android.com/tools">
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
    <application>
        <activity android:name="com.unity3d.player.UnityPlayerActivity"
                  android:theme="@style/UnityThemeSelector">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
            <meta-data android:name="unityplayer.UnityActivity" android:value="true" />
            <meta-data android:name="unityplayer.SkipPermissionsDialog" android:value="true" />
        </activity>
    </application>
</manifest>

Run Tests in the Cloud

Once you have one or more Recorded Tests (or any Unity Tests) that you'd like to run on a cloud Android device:

  1. [If you're running any Recorded Tests] Navigate to Automated QA > Recording Upload... and then press "Upload All"
  2. Navigate to Automated QA > Cloud Test Runner... and then press "Build & Upload". This will create a build bundled with all of your play mode tests and upload it to our cloud testing service.
  3. The Build ID should be automatically populated in the buildid field. Press "Run on Device Farm".
  4. The Job ID should be automatically populated in the jobid field. Press "Refresh" to check the status of your tests. Cloud tests usually take 15-30 minutes to complete.
  5. Press "Get Test Results" to view the results of your tests in an HTML file.

Supported Devices

Currently, we run tests by default on 5 of the most popular Android devices:

  • Samsung Galaxy S9 (OS 9)
  • Samsung Galaxy S20 (OS 10)
  • Samsung Galaxy Note 10 (OS 9),
  • Google Pixel 4 XL (OS 10)
  • Google Pixel XL (OS 8.0.0).

Please reach out to us at AutomatedTesting@unity3d.com if you would like to run tests on other devices offered by AWS Device Farm.

Advanced Features

Manual Recorded Test Creation

Recorded Playbacks can be used "drive" any Play Mode UnityTest. To manually create a test driven by a recording, create a Unity Test and then add [RecordedTest("[relative path of recording (relative to /Assets)]")] right after the test's [UnityTest] attribute in the source code of the test.

For example, to validate that a tutorial loads as expected:

  1. Create a recorded playback that ends with the tutorial loading
  2. Rename the resulting recording-[timestamp].json file to something more descriptive like "load_tutorial.json"
  3. Create a new Unity Test with [RecordedTest("Recordings/load_tutorial.json")] below the [UnityTest] attribute
  4. Inside the Unity Test: a. Load the scene where your recording begins, e.g. SceneManager.LoadScene("0_Title"); b. Once playback is complete or some period of time has passed, Assert that some key condition is true

Example RecordedTest usage:

namespace Tests
{
    public class SmokeTests : RecordedTestSuite
    {
        [UnityTest]
        [RecordedTest("Recordings/load_tutorial.json.json")]
        public IEnumerator validate_tutorial_loads()
        {
           SceneManager.LoadScene("0_Title");
           ...

           //wait until the Tutorial scene loads
           while (SceneManager.GetActiveScene().name != "Tutorial")
           {
               yield return null;
           }

           Assert.NotNull(GameObject.Find("Tutorial_Manager");
        }

    }
}