Class KeypointLabeler

Produces keypoint annotations for a humanoid model. This labeler supports generic KeypointTemplate. Template values are mapped to rigged Animator Avatar. Custom joints can be created by applying JointLabel to empty game objects at a body part's location.

Keypoints are recorded by this labeler with a state value describing if they are present on the model, present but not visible, or visible. A keypoint can be listed as not visible for three reasons: it is outside of the camera's view frustum, it is occluded by another object in the scene, or it is occluded by itself, for example a raised arm in front a model's face could occlude its eyes from being visible. To calculate self occlusion values, the keypoint labeler uses tolerances per keypoint to determine if the keypoint is blocked. The initial tolerance value for each keypoint is set per keypoint in the KeypointTemplate file. The tolerance of a custom keypoints can be set with the JointLabel used to create the keypoint. Finally, a KeypointOcclusionOverrides component be added to a model to apply a universal scaling override to all of the keypoint tolerances defined in a keypoint template.

Inheritance

object

CameraLabeler

KeypointLabeler

Inherited Members

CameraLabeler.enabled

CameraLabeler.hudPanel

CameraLabeler.overlayPanel

CameraLabeler.visualizationEnabled

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: UnityEngine.Perception.GroundTruth.Labelers

Assembly: Unity.Perception.Runtime.dll

Syntax

[Serializable]
[MovedFrom("UnityEngine.Perception.GroundTruth")]
public sealed class KeypointLabeler : CameraLabeler

Constructors

KeypointLabeler()

Creates a new key point labeler. This constructor creates a labeler that is not valid until a IdLabelConfig and KeypointTemplate are assigned.

Declaration

public KeypointLabeler()

KeypointLabeler(IdLabelConfig, KeypointTemplate)

Creates a new key point labeler.

Declaration

public KeypointLabeler(IdLabelConfig config, KeypointTemplate template)

Parameters

Type	Name	Description
IdLabelConfig	config	The Id label config for the labeler
KeypointTemplate	template	The active keypoint template

Fields

activeTemplate

The active keypoint template. Required to annotate keypoint data.

Declaration

public KeypointTemplate activeTemplate

Field Value

Type	Description
KeypointTemplate

animationPoseConfigs

Array of animation pose labels which map animation clip times to ground truth pose labels.

Declaration

public List<AnimationPoseConfig> animationPoseConfigs

Field Value

Type	Description
List<AnimationPoseConfig>

annotationId

The GUID id to associate with the annotations produced by this labeler.

Declaration

public string annotationId

Field Value

Type	Description
string

idLabelConfig

The IdLabelConfig which associates objects with labels.

Declaration

public IdLabelConfig idLabelConfig

Field Value

Type	Description
IdLabelConfig

objectFilter

Controls which objects will have keypoints recorded in the dataset. KeypointObjectFilter

Declaration

public KeypointObjectFilter objectFilter

Field Value

Type	Description
KeypointObjectFilter

visualizeOccludedPoints

Should the visualizer draw occluded points.

Declaration

[Tooltip("If checked on the visualizer will draw an empty black circle for occluded points. If unchecked the point will not be drawn.")]
public bool visualizeOccludedPoints

Field Value

Type	Description
bool

Properties

description

A human-readable description of the labeler

Declaration

public override string description { get; }

Property Value

Type	Description
string

Overrides

CameraLabeler.description

labelerId

The GUID id to associate with the data produced by this labeler.

Declaration

public override string labelerId { get; }

Property Value

Type	Description
string

Overrides

CameraLabeler.labelerId

supportsVisualization

Labelers should set this in their setup to define if they support realtime visualization of their data.

Declaration

protected override bool supportsVisualization { get; }

Property Value

Type	Description
bool

Overrides

CameraLabeler.supportsVisualization

Methods

OnEndRendering(ScriptableRenderContext)

Called just after the camera renders each frame the the labeler is enabled and ShouldCaptureThisFrame is true.

Declaration

protected override void OnEndRendering(ScriptableRenderContext scriptableRenderContext)

Parameters

Type	Name	Description
ScriptableRenderContext	scriptableRenderContext

Overrides

CameraLabeler.OnEndRendering(ScriptableRenderContext)

OnVisualize()

Labeling pass to display labeler's visualization components, if applicable. Important note, all labeler's visualizations need to use Unity's Immediate Mode GUI (IMGUI) https://docs.unity3d.com/Manual/GUIScriptingGuide.html system. This called is triggered from OnGUI() call. This call happens immediately before OnVisualizeAdditionalUI() so that the visualization components are drawn below the UI elements.

Declaration

protected override void OnVisualize()

Overrides

CameraLabeler.OnVisualize()

Setup()

Called just before the first call to OnUpdate() or OnBeginRendering(ScriptableRenderContext). Implement this to initialize state.

Declaration

protected override void Setup()

Overrides

CameraLabeler.Setup()

Events

KeypointsComputed

Action that gets triggered when a new frame of key points are computed.

Declaration

public event Action<int, List<KeypointComponent>> KeypointsComputed

Event Type

Type	Description
Action<int, List<KeypointComponent>>