AR development in Unity

Augmented Reality (AR) involves a new set of design challenges compared to VR or traditional real-time 3D applications. By definition, an augmented reality app overlays its content on the real world around the user. To place an object in the real world, you must first determine where to place for it. For example, you may want to place a virtual painting on a physical wall. If you place a virtual potted plant, you may want it on a physical table or the floor. An AR app receives information about the world from the user’s device, such as the locations of planar surfaces, the detection of objects, people, faces, and so on; and must decide how to use this information to create a good experience for the user.

When you open a typical AR scene in Unity, you will not find many 3D objects in the scene or the Hierarchy view. Instead, most GameObjects in the scene define the settings and logic of the app. 3D content is typically created as prefabs that are added to the scene at runtime in response to AR-related events.

A typical AR scene in the Unity Editor

Basic AR scene elements

A basic AR scene contains the following GameObjects and components:

• AR Session GameObject
  • AR Session component
  • AR Input Manager component
• XR Origin (Mobile) GameObject
  • XR Origin component
  • Camera Offset GameObject
    • AR Camera Manager component
    • AR Camera Background component

If you have the XR Interaction Toolkit installed, the XR Origin option for AR applications changes to XR Origin (AR), which adds GameObecjts for representing hand-held controllers and the toolkit components to interaction with objects in the scene.

• XR Origin (AR) GameObject
  • XR Origin component
  • Camera Offset GameObject
    • AR Camera Manager component
    • AR Camera Background component
    • LeftHand/RightHand Controller GameObjects
      • XR Controller (Action-based) component
      • XR Ray Interactor component
      • Line Renderer component
      • XR Interactor Line Visual component
      • Sorting Group component

Tip: Use the GameObject > XR menu to add these GameObjects and their associated components to a scene. (You can also open the menu by right-clicking in the Hierarchy window.)

In addition to these session GameObjects, you need the corresponding AR manager component for each type of AR feature that your application uses.

Refer to Set up an XR scene for an overview of how to set up any XR scene.

Refer to the AR Foundation package documentation, including Scene set up for more in-depth information on creating AR applications.

AR packages

To build AR apps in unity, you can install the AR Foundation package along with the XR provider plug-ins for the devices you want to support. Unity provides additional packages, including Unity Mars and the XR Interaction Toolkit to make it easier and faster to develop AR experiences.

AR provider plug-ins

The AR provider plug-ins supported by Unity include:

• Apple ARKit XR Plug-in on iOS
• Google ARCore XR Plug-in on Android
• OpenXR for any AR device with an OpenXR runtime, including HoloLens 2 and others.

Use the XR Plug-in Management system to add and enable one or more of these plug-ins. See XR Project set up for instructions.

Note: Depending on the platform or device, you might need to install additional packages to along with OpenXR. For example, to build an AR app for HoloLens 2, you must install the Microsoft’s Mixed Reality OpenXR Plugin.

AR Foundation

The AR Foundation package supports AR development in Unity.

AR Foundation enables you to create multi-platform AR apps with Unity. In an AR Foundation project, you choose which AR features to enable by adding the corresponding manager components to your scene. When you build and run your app on an AR device, AR Foundation enables these features using the platform’s native AR SDK, so you can create once and deploy to the world’s leading AR platforms.

A device can be AR-capable without supporting all possible AR features. Available functionality depends on both the device platform and the capabilities of the specific device. For example, ARCore, Google’s AR platform for Android, does not currently support body tracking, so body tracking can’t be used when you build your app for the Android platform. Even on the same platform, capabilities can vary from device to device. For example, a specific device model might support AR through its world-facing camera, but not its user-facing camera.

For more information about AR Foundation, see the AR Foundation package documentation.

Unity Mars

Unity Mars provides purpose-built authoring tools and better workflows for creating AR applications.

• Plain-language authoring: You can define simple rules that can be stated in plain language like, “on every horizontal surface, create grass,” to specify how your app content should augment a scene based on the AR features detected by a user’s device. See A plain-language approach to authoring AR for more information.
• Proxy-based workflow: you add proxies to a scene to represent real-world AR features. You can set conditions and actions on your proxies to specify how your app should respond when a matching object is detected in the real world. For example, you can add a proxy to a scene that activates whenever a suitable horizontal plane is detected.
• In-Editor simulation: Unity Mars provides a simulation mode, along with pre-built sample environments, that helps you test your AR logic inside the Editor.
• Customizable building-blocks: Unity Mars contains templates and other building blocks for creating all or part of an AR application. For example, the Training Template provides UI and logic that you can use to build an AR tutorial that walks a trainee through a series of steps.

Unity Mars requires a compatible license. The Unity Pro, Unity Enterprise, and Unity Industrial Collection plans include Mars. You can also obtain a license for Unity Mars separately if you have a different plan. See Unity Mars for more information.

XR Interaction Toolkit

The Unity XR Interaction Toolkit provides tools for building both AR and VR interactions. The AR functionality provided by the XR Interaction Toolkit includes:

• AR gesture system to map screen touches to gesture events
• AR placement Interactable component to help place virtual objects in the real world
• AR gesture Interactor and Interactable components to support object manipulations such as place, select, translate, rotate, and scale
• AR annotations to inform users about AR objects placed in the real world

AR template

Unity’s AR Project Template provides a starting point for virtual reality development in Unity. The template configures project settings, pre-installs the right packages, and includes a sample scene with various pre-configured example assets to demonstrate how to set up a project that is ready for AR. Access the AR template through the Unity Hub when you create a new project. Refer to Create a new project for information about creating a project with the template.

For more information about the template assets and how the sample scene is set up, refer to About the AR Project Template.