Namespace UnityEngine.Perception.GroundTruth

Classes

BoundingBox2DLabeler

Produces 2d bounding box annotations for all visible objects each frame.

BoundingBox3DLabeler

Produces 3d bounding box ground truth for all visible and Labeling objects each frame.

Abstract class for defining custom annotation and metric generation to be run by PerceptionCamera. Instances of CameraLabeler on labelers will be invoked each frame the camera is set to capture data (see ShouldCaptureThisFrame).

DatasetCapture

Global manager for frame scheduling and output capture for simulations. Data capture follows the schema defined in TODO: Expose schema publicly

Ego

Marker indicating the GameObject is the root of the ego for a set of sensors. Works with PerceptionCamera. Custom sensors can use the EgoHandle property to register themselves under this ego.

GroundTruthLabelSetupSystem

System which notifies the registered IGroundTruthGenerator about Labeling additions.

HUDPanel

Heads up display panel used to publish a key value pair on the screen. Items added to this need to have their values updated every frame, or else, they will be determined to be stale and removed from the view and re-used for a new entry.

IdLabelConfig

A definition for how a Labeling should be resolved to a single label and id for ground truth generation.

InstanceIdToColorMapping

Static class to procedurally generate a unique color for an instance ID. This algorithm is deterministic, and will always return the same color for a ID, and the same ID for a color. ID 0 is reserved to be an invalid ID and is mapped to color black (0,0,0,255). Invalid IDs always map to black, and black always maps to ID 0. In order to try to create visually contrasting colors for IDs, there are a subset of IDs reserved (1-65) to be generated by applying the golden ration to find the next color in the HSL spectrum. All of these colors, and only theses colors, will be in the alpha channel 255. After the first 65 IDs, the color will be determined by iterating through all available RGB values in the alpha channels from 264 - 1. Alpha channel 0 is marked as invalid. This service will support over 4 billion unique IDs => colors [(256^4) - (256*2) + 64]

InstanceSegmentationLabeler

Produces instance segmentation for each frame.

LabelConfig<T>

A definition for how a Labeling should be resolved to a single label and id for ground truth generation.

Labeling

Defines a set of labels associated with the object and its descendants. A Labeling component will override any Labeling components on the object's ancestors.

ObjectCountLabeler

Labeler which produces object counts for each label in the associated IdLabelConfig each frame.

OverlayPanel

Some labeler's result in a full screen image per frame. The overlay panel controls which of these labeler's image is currently shown.

PerceptionCamera

Captures ground truth from the associated Camera.

RenderTextureReader<T>

RenderTextureReader reads a RenderTexture from the GPU each frame and passes the data back through a provided callback.

RenderedObjectInfoGenerator

A CPU-based pass which computes bounding box and pixel counts per-object from instance segmentation images

RenderedObjectInfoLabeler

Labeler which produces label id, instance id, and visible pixel count in a single metric each frame for each object which takes up one or more pixels in the camera's frame.

SemanticSegmentationLabelConfig

A definition for how a Labeling should be resolved to a single label and color for semantic segmentation generation.

SemanticSegmentationLabeler

Labeler which generates a semantic segmentation image each frame. Each object is rendered to the semantic segmentation image using the color associated with it based on the given SemanticSegmentationLabelConfig. Semantic segmentation images are saved to the dataset in PNG format.

Only one SemanticSegmentationLabeler can render at once across all cameras.

Structs

Annotation

A handle to an annotation. Can be used to report metrics on the annotation.

AnnotationDefinition

A metric type, used to define a kind of annotation. RegisterAnnotationDefinition(string, string, string, Guid).

AsyncAnnotation

A handle to an async annotation, used to report values for an annotation after the frame for the annotation has past. See ReportAnnotationAsync(AnnotationDefinition)

AsyncMetric

Handle to a metric whose values may be reported in a subsequent frame.

BoundingBox3DLabeler.BoxData

Each 3D bounding box data record maps a tuple of (instance, label) to translation, size and rotation that draws a 3D bounding box, as well as velocity and acceleration (optional) of the 3D bounding box. All location data is given with respect to the sensor coordinate system.

EgoHandle

An ego, which is used to group multiple sensors under a single frame of reference.

GroundTruthInfo

Information regarding a Labeling instance. Generated by GroundTruthLabelSetupSystem

IdLabelEntry

An entry for IdLabelConfig mapping a label to an integer id.

MetricDefinition

A metric type, used to define a kind of metric. RegisterMetricDefinition(string, string, Guid).

RenderedObjectInfo

Describes an instance of an object in an instance segmentation frame. Generated by RenderedObjectInfoGenerator.

SemanticSegmentationLabelEntry

LabelEntry for SemanticSegmentationLabelConfig. Maps a label to a color.

SemanticSegmentationLabeler.ImageReadbackEventArgs

Event information for imageReadback

SensorHandle

A handle to a sensor managed by the DatasetCapture. It can be used to check whether the sensor is expected to capture this frame and report captures, annotations, and metrics regarding the sensor.

SensorSpatialData

Container holding the poses of the ego and sensor. Also optionally contains the ego velocity and acceleration.

Namespace UnityEngine.Perception.GroundTruth

Classes

Structs

Interfaces

Enums