Class KeypointLabeler

Produces keypoint annotations for a humanoid model. This labeler supports generic KeypointTemplate. Template values are mapped to rigged . Custom joints can be created by applying JointLabel to empty game objects at a body part's location.

Keypoints are recorded by this labeler with a state value describing if they are present on the model, present but not visible, or visible. A keypoint can be listed as not visible for three reasons: it is outside of the camera's view frustum, it is occluded by another object in the scene, or it is occluded by itself, for example a raised arm in front a model's face could occlude its eyes from being visible. To calculate self occlusion values, the keypoint labeler uses tolerances per keypoint to determine if the keypoint is blocked. The initial tolerance value for each keypoint is set per keypoint in the KeypointTemplate file. The tolerance of a custom keypoints can be set with the JointLabel used to create the keypoint. Finally, a KeypointOcclusionOverrides component be added to a model to apply a universal scaling override to all of the keypoint tolerances defined in a keypoint template.

Inheritance
Object
CameraLabeler
KeypointLabeler
Inherited Members
Namespace: UnityEngine.Perception.GroundTruth
Syntax
public sealed class KeypointLabeler : CameraLabeler

Constructors

KeypointLabeler()

Creates a new key point labeler. This constructor creates a labeler that is not valid until a IdLabelConfig and KeypointTemplate are assigned.

Declaration
public KeypointLabeler()

KeypointLabeler(IdLabelConfig, KeypointTemplate)

Creates a new key point labeler.

Declaration
public KeypointLabeler(IdLabelConfig config, KeypointTemplate template)
Parameters
Type Name Description
IdLabelConfig config

The Id label config for the labeler
KeypointTemplate template

The active keypoint template

Fields

activeTemplate

The active keypoint template. Required to annotate keypoint data.

Declaration
public KeypointTemplate activeTemplate
Field Value
Type Description
KeypointTemplate

animationPoseConfigs

Array of animation pose labels which map animation clip times to ground truth pose labels.

Declaration
public List<AnimationPoseConfig> animationPoseConfigs
Field Value
Type Description
List<AnimationPoseConfig>

annotationId

The GUID id to associate with the annotations produced by this labeler.

Declaration
public string annotationId
Field Value
Type Description
String

idLabelConfig

The IdLabelConfig which associates objects with labels.

Declaration
public IdLabelConfig idLabelConfig
Field Value
Type Description
IdLabelConfig

objectFilter

Controls which objects will have keypoints recorded in the dataset. KeypointObjectFilter

Declaration
public KeypointObjectFilter objectFilter
Field Value
Type Description
KeypointObjectFilter

Properties

description

A human-readable description of the labeler

Declaration
public override string description { get; }
Property Value
Type Description
String
Overrides

labelerId

The GUID id to associate with the data produced by this labeler.

Declaration
public override string labelerId { get; }
Property Value
Type Description
String
Overrides

supportsVisualization

Labelers should set this in their setup to define if they support realtime visualization of their data.

Declaration
protected override bool supportsVisualization { get; }
Property Value
Type Description
Boolean
Overrides

Methods

OnEndRendering(ScriptableRenderContext)

Called just after the camera renders each frame the the labeler is enabled and ShouldCaptureThisFrame is true.

Declaration
protected override void OnEndRendering(ScriptableRenderContext scriptableRenderContext)
Parameters
Type Name Description
ScriptableRenderContext scriptableRenderContext

The current context from the Scriptable Render Pipeline.
Overrides

OnVisualize()

Labeling pass to display labeler's visualization components, if applicable. Important note, all labeler's visualizations need to use Unity's Immediate Mode GUI (IMGUI) system. This called is triggered from call. This call happens immediately before OnVisualizeAdditionalUI() so that the visualization components are drawn below the UI elements.

Declaration
protected override void OnVisualize()
Overrides

Setup()

Called just before the first call to OnUpdate() or OnBeginRendering(ScriptableRenderContext). Implement this to initialize state.

Declaration
protected override void Setup()
Overrides

Events

KeypointsComputed

Action that gets triggered when a new frame of key points are computed.

Declaration
public event Action<int, List<KeypointComponent>> KeypointsComputed
Event Type
Type Description
Action<Int32, List<KeypointComponent>>