Class TestRunnerApi

The TestRunnerApi retrieves and runs tests programmatically from code inside the project, or inside other packages. TestRunnerApi is a ScriptableObject. You can initialize the API like this:

var testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();

You can subscribe and receive test results in one instance of the API, even if the run starts from another instance. The TestRunnerApi supports the following workflows:

Note: Non-static methods in this class will become obsolete in future versions so you should use the static methods (e.g. RegisterTestCallback, ExecuteTestRun) rather than their non-static equivalents (e.g. RegisterCallbacks, Execute).

Inheritance
object
Object
ScriptableObject
TestRunnerApi
Inherited Members
Namespace: UnityEditor.TestTools.TestRunner.Api
Assembly: solution.dll
Syntax
public class TestRunnerApi : ScriptableObject
Examples

Run tests

Run tests by calling ExecuteTestRun and providing some execution settings in the form of a Filter. The Filter specifies what tests to run. The following is an example of how to run all Play Mode tests in a project:

var filter = new Filter()
{
    testMode = TestMode.PlayMode
};
    TestRunnerApi.ExecuteTestRun(new ExecutionSettings(filter));

Multiple filter values

You can specify a more specific filter by filling out the fields on the Filter class in more detail. Many of the fields allow for multiple values. The runner runs any test that matches at least one of the specified values. In this example, the API runs tests with full names that match either of the two names provided:

TestRunnerApi.ExecuteTestRun(new ExecutionSettings(new Filter()
{
    testNames = new[] {"MyTestClass.NameOfMyTest", "SpecificTestFixture.NameOfAnotherTest"}
}));

Multiple filter fields

If using multiple different fields on the filter, then the runner runs tests that match each of the different fields. In this example, a test is run if it matches either of the two test names and belongs to an assembly with the specified name:

TestRunnerApi.ExecuteTestRun(new ExecutionSettings(new Filter()
{
    assemblyNames = new [] {"MyTestAssembly"},
    testNames = new [] {"MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest"}
}));

Multiple constructor filters

ExecutionSettings takes one or more filters in its constructor. If there is no filter provided, then it runs all Edit Mode tests by default. If there are multiple filters provided, then a test runs if it matches any of the filters. In this example, it runs any tests in the assembly named MyTestAssembly and any test with a full name matching either of the two provided test names:

TestRunnerApi.ExecuteTestRun(new ExecutionSettings(
    new Filter()
    {
        assemblyNames = new[] { "MyTestAssembly" },
    },
    new Filter()
    {
        testNames = new[] { "MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest" }
    }
));

Get test results

You can receive callbacks when the active test run, or an individual test, starts and finishes. You can register callbacks by invoking RegisterTestCallback with an instance of a class that implements ICallbacks. There are four ICallbacks methods for the start and finish of both the whole run and each level of the test tree. An example of how listeners can be set up:

Note: Listeners receive callbacks from all test runs, regardless of the registered TestRunnerApi for that instance.

public void SetupListeners()
{
    TestRunnerApi.RegisterTestCallback(new MyCallbacks());
}

private class MyCallbacks : ICallbacks
{
    public void RunStarted(ITestAdaptor testsToRun)
    {

    }

    public void RunFinished(ITestResultAdaptor result)
    {

    }

    public void TestStarted(ITestAdaptor test)
    {

    }

    public void TestFinished(ITestResultAdaptor result)
    {
       if (!result.HasChildren && result.ResultState != "Passed")
       {
           Debug.Log(string.Format("Test {0} {1}", result.Test.Name, result.ResultState));
       }
    }
}

Note: The registered callbacks are not persisted on domain reloads. So it is necessary to re-register the callback after a domain reload, usually with InitializeOnLoad.

It's possible to provide a priority as an integer as the second argument when registering a callback. This influences the invocation order of different callbacks. The default value is zero. It's also possible to provide RegisterTestCallback with a class instance that implements IErrorCallbacks that is an extended version of ICallbacks. IErrorCallbacks also has a callback method for OnError that invokes if the run fails to start, for example, due to compilation errors or if an IPrebuildSetup throws an exception.

Retrieve list of tests

You can retrieve the test tree by invoking RetrieveTestTree with the desired ExecutionSettings and a callback action, with an ITestAdaptor representing the test tree.

var filter = new Filter()
{
    assemblyNames = new [] {"myTestAssembly"}
};
TestRunnerApi.RetrieveTestTree(new ExecutionSettings(filter), (testRoot) =>
{
    Debug.Log(string.Format("Tree contains {0} tests.", testRoot.TestCaseCount));
});

Methods

Name Description
CancelAllTestRuns()

Cancels all running test runs. Currently only supports EditMode tests.
CancelTestRun(string)

Cancel the test run with a given guid. The guid can be retrieved when executing the test run. Currently only supports EditMode tests.
Execute(ExecutionSettings)

Starts a test run with a given set of executionSettings.
ExecuteTestRun(ExecutionSettings)

Starts a test run with a given set of executionSettings.
GetActiveRunGuids()

Provides a list of guids for all test runs that are currently active.
GetCustomRunnerNames()

Get all the names of the currently registered custom runners.
GetExecutionSettings(string)

Gets the execution settings for a test run with a specific guid.
IsAnyRunActive()

Checks if any current test run is currently running.
IsRunActive(string)

Checks if a test run with a given guid is active.
RegisterCallbacks<T>(T, int)

Sets up a given instance of ICallbacks to be invoked on test runs.
RegisterCustomRunner(CustomRunnerBase)

Registers an implementation of CustomRunnerBase with the test framework, allowing for running custom test frameworks. Since custom runners are identified by name, the provided custom runner must use a name that has not already been registered.
RegisterTestCallback<T>(T, int)

Sets up a given instance of ICallbacks to be invoked on test runs.
RetrieveTestList(TestMode, Action<ITestAdaptor>)

Retrieve the full test tree as ITestAdaptor for a given test mode.
RetrieveTestTree(ExecutionSettings, Action<ITestAdaptor>)

Retrieve a list of tests in a tree structure matching a given execution settings.
SaveResultToFile(ITestResultAdaptor, string)

Save a given set of ITestResultAdaptor to a file at the provided file path.
UnregisterCallbacks<T>(T)

Unregister an instance of ICallbacks to no longer receive callbacks from test runs.
UnregisterCustomRunner(CustomRunnerBase)

Unregisters a custom runner from the test framework.
UnregisterTestCallback<T>(T)

Unregister an instance of ICallbacks to no longer receive callbacks from test runs.