Bounding box detection

Apple's RoomPlan framework allows users to scan environments and detect objects through room capture functionality. It also provides the bounding box properties of captured objects during and after the room capture process. Users must complete a room capture before retrieving the bounding boxes of the detected objects. To learn more about the RoomPlan framework, refer to Apple's RoomPlan documentation.

Tip

When developing an AR app, refer to both the AR Foundation documentation as well as the required packages for each platform you support.

Requirements

This feature is only available on iOS devices equipped with a LiDAR scanner running iOS 17 and newer.

Note

To check whether a device has a LiDAR scanner, refer to Apple's Tech Specs.

Bounding box classifications

This package maps Apple's native semantic labels to AR Foundation's BoundingBoxClassifications. Apple supports multiple classifications per bounding box.

Refer to the following table to understand the mapping between AR Foundation's classifications and Apple's semantic labels:

AR Foundation Label	Apple Label
Couch	sofa
Table	table
Bed	bed
Screen	television
Storage	storage
Bathtub	bathtub
Chair	chair
Dishwasher	dishwasher
Fireplace	fireplace
Oven	oven
Refrigerator	refrigerator
Sink	sink
Stairs	stairs
Stove	stove
Toilet	toilet
WasherDryer	washerDryer

Room capture instructions

Apple provides instructions to guide the user through the room capture process. Refer to the following table for the set of instruction enumerations:

ARKit Label	Apple Label	Description
Normal	normal	Room capturing normally and the user needs no coaching.
MoveCloseToWall	moveCloseToWall	The user should move closer to the wall.
MoveAwayFromWall	moveAwayFromWall	The user should move further from the wall.
TurnOnLight	turnOnLight	The user should increase the amount of light in the room.
SlowDown	slowDown	The user should move slower.
LowTexture	lowTexture	The process doesn't detect distinguishable features.

Check room capture support

The following example method checks whether the RoomPlanBoundingBoxSubsystem is available. This method is used by the other code examples on this page.

bool CheckRoomCaptureSupport(out RoomPlanBoundingBoxSubsystem subsystem)
{
    // check if roomplan bounding box subsystem is available
    subsystem = m_BoundingBoxManager.subsystem as RoomPlanBoundingBoxSubsystem;
    if (subsystem == null)
    {
        Debug.LogError("RoomPlanBoundingBoxSubsystem is not available.");
        return false;
    }

    return true;
}

Take a room capture

Use the following methods to take a room capture:

Life cycle event	Description
SetupRoomCapture	Set up room capture before the process.
StartRoomCapture	Start room capture process.
StopRoomCapture	Stop room capture process.
IsRoomCapturing	Check the room capture status.
GetRoomCaptureInstruction	Retrieve the instructions for accurate room capture.