Namespace UnityEngine.Perception.GroundTruth

Classes

BoundingBox2DLabeler

Produces 2d bounding box annotations for all visible objects each frame.

BoundingBox3DLabeler

Produces 3d bounding box ground truth for all visible and Labeling objects each frame.

CameraLabeler

Abstract class for defining custom annotation and metric generation to be run by PerceptionCamera. Instances of CameraLabeler on labelers will be invoked each frame the camera is set to capture data (see ShouldCaptureThisFrame).

DatasetCapture

Global manager for frame scheduling and output capture for simulations. Data capture follows the schema defined in TODO: Expose schema publicly

Ego

Marker indicating the GameObject is the root of the ego for a set of sensors. Works with PerceptionCamera. Custom sensors can use the EgoHandle property to register themselves under this ego.

GroundTruthLabelSetupSystem

System which notifies the registered IGroundTruthGenerator about Labeling additions.

HUDPanel

Heads up display panel used to publish a key value pair on the screen. Items added to this need to have their values updated every frame, or else, they will be determined to be stale and removed from the view and re-used for a new entry.

IdLabelConfig

A definition for how a Labeling should be resolved to a single label and id for ground truth generation.

LabelConfig<T>

A definition for how a Labeling should be resolved to a single label and id for ground truth generation.

Labeling

Defines a set of labels associated with the object and its descendants. A Labeling component will override any Labeling components on the object's ancestors.

ObjectCountLabeler

Labeler which produces object counts for each label in the associated IdLabelConfig each frame.

PerceptionCamera

Captures ground truth from the associated Camera.

RenderedObjectInfoGenerator

A CPU-based pass which computes bounding box and pixel counts per-object from instance segmentation images

RenderedObjectInfoLabeler

Labeler which produces label id, instance id, and visible pixel count in a single metric each frame for each object which takes up one or more pixels in the camera's frame.

RenderTextureReader<T>

RenderTextureReader reads a RenderTexture from the GPU each frame and passes the data back through a provided callback.

SemanticSegmentationLabelConfig

A definition for how a Labeling should be resolved to a single label and color for semantic segmentation generation.

SemanticSegmentationLabeler

Labeler which generates a semantic segmentation image each frame. Each object is rendered to the semantic segmentation image using the color associated with it based on the given SemanticSegmentationLabelConfig. Semantic segmentation images are saved to the dataset in PNG format.

Only one SemanticSegmentationLabeler can render at once across all cameras.

Structs

Annotation

A handle to an annotation. Can be used to report metrics on the annotation.

AnnotationDefinition

A metric type, used to define a kind of annotation. RegisterAnnotationDefinition(String, String, String, Guid).

AsyncAnnotation

A handle to an async annotation, used to report values for an annotation after the frame for the annotation has past. See ReportAnnotationAsync(AnnotationDefinition)

AsyncMetric

Handle to a metric whose values may be reported in a subsequent frame.

BoundingBox3DLabeler.BoxData

Each 3D bounding box data record maps a tuple of (instance, label) to translation, size and rotation that draws a 3D bounding box, as well as velocity and acceleration (optional) of the 3D bounding box. All location data is given with respect to the sensor coordinate system.

bounding_box_3d label_id (int): Integer identifier of the label label_name (str): String identifier of the label instance_id (str): UUID of the instance. translation (float, float, float): 3d bounding box's center location in meters as center_x, center_y, center_z with respect to global coordinate system. size (float, float, float): 3d bounding box size in meters as width, length, height. rotation (float, float, float, float): 3d bounding box orientation as quaternion: w, x, y, z. velocity (float, float, float) [optional]: 3d bounding box velocity in meters per second as v_x, v_y, v_z. acceleration (float, float, float) [optional]: 3d bounding box acceleration in meters per second^2 as a_x, a_y, a_z.

EgoHandle

An ego, which is used to group multiple sensors under a single frame of reference.

GroundTruthInfo

Information regarding a Labeling instance. Generated by GroundTruthLabelSetupSystem

IdLabelEntry

An entry for IdLabelConfig mapping a label to an integer id.

MetricDefinition

A metric type, used to define a kind of metric. RegisterMetricDefinition(String, String, Guid).

RenderedObjectInfo

Describes an instance of an object in an instance segmentation frame. Generated by RenderedObjectInfoGenerator.

SemanticSegmentationLabelEntry

LabelEntry for SemanticSegmentationLabelConfig. Maps a label to a color.

SemanticSegmentationLabeler.ImageReadbackEventArgs

Event information for imageReadback

SensorHandle

A handle to a sensor managed by the DatasetCapture. It can be used to check whether the sensor is expected to capture this frame and report captures, annotations, and metrics regarding the sensor.

SensorSpatialData

Container holding the poses of the ego and sensor. Also optionally contains the ego velocity and acceleration.

Interfaces

IGroundTruthGenerator

Interface for setting up MeshRenderers for ground truth generation via GroundTruthLabelSetupSystem.

ILabelEntry

Required interface for entries in a LabelConfig<T>. Exposes the string label which is the "key" for the entry.

Enums

BoundingBoxOrigin

The origin to use for bounding box calculation

StartingLabelId

Selector for whether label ids should start at zero or one. startingLabelId.