Class CameraSensingComponent

This class implement the sensing part of the photosensor.

This implementation relies on a Unity camera to measure depth and intensity. Thus, all incoming rays have to converge to a single point, the camera effective pinhole. Consequences:

  • Origin of individual beams (e.g. for multi-beam lidars) will be ignored and assumed to be at the camera pinhole.
  • Photosensor translation cannot be interpolated; if the lidar is moving in the scene and multiple sampling requests are resolved in the same Unity update, they will all use the final lidar position, introducing translational error.
  • Lidars usually have constant angular difference between consecutive beams, while cameras have constant translational difference between consecutive pixels. Thus, some approximation must be made, leading to slight depth inaccuracies compared to a ray tracing implementation.

For the last point, use the Max Approx Error field to configure the maximum approximation angular error, that is, the maximum angle between the desired lidar beam and the closest camera ray going through a pixel center. This is used to compute the texture size used by the camera.

The code also uses a few assumptions:

  • Beams in samplingOffsets are roughly oriented along the positive Z axis.
  • Photosensor angular speed is constant, which avoid having to look at all the stamps.
  • Photosensor has no roll rotation, otherwise bounding boxes will be incorrect.
  • Photosensor turns less than 120 degrees between consecutive updates, otherwise some lidar points may be missing.

Warning: this will not work for lidar using beam patterns (e.g. MEMS) instead of physically moving the photosensor, since it violates the first assumption.

Inheritance
Object
Object
Component
Behaviour
MonoBehaviour
PhotosensorSensingComponent
CameraSensingComponent
Inherited Members
UnityEngine.Object.FindObjectsOfType<T>(System.Boolean)
UnityEngine.Object.FindObjectOfType<T>(System.Boolean)
Namespace: Mechatronics.SensorSDK
Syntax
[Serializable]
public class CameraSensingComponent : PhotosensorSensingComponent

Fields

_customPassDescriptor

Keep custom pass added to the camera volume. Used to hook a unity camera just before the post processing step. We take the buffer and do the post processing step ourself in the graph.

Declaration
protected CustomPassDescriptor _customPassDescriptor
Field Value
Type Description
CustomPassDescriptor

maxApproxErrorDeg

Maximum approximation angular error, i.e. maximum angle between the desired lidar beam and the closest camera ray going through a pixel center. This is used to compute the texture size used by the camera.

Declaration
[Tooltip("Maximum angular error introduced by the camera-based implementation")]
public float maxApproxErrorDeg
Field Value
Type Description
Single

Methods

Activate(PhotosensorData, Action<CustomPassContext, PhotosensorOutputBuffer>)

Initializes the sensing component.

Declaration
public override void Activate(PhotosensorData photosensorData, Action<CustomPassContext, PhotosensorOutputBuffer> callback)
Parameters
Type Name Description
PhotosensorData photosensorData

Information define on the photosensor node.
Action<CustomPassContext, PhotosensorOutputBuffer> callback

Provides a callback to invoke when the scene sampling is completed.
Overrides

Deactivate()

Releases the sampler and other GPU resources.

Declaration
public override void Deactivate()
Overrides

InitializeComputeShader(String, String)

Load and find the kernel in a compute buffer.

Declaration
protected void InitializeComputeShader(string kernelName = "CastRaysToCameraTexture", string computeShaderName = "PhotosensorCameraBased")
Parameters
Type Name Description
String kernelName

Kernel name to find.
String computeShaderName

Compute shader to load.

UpdateSamplingOffsets(List<OrientedPoint>)

Updates the sensing component with new sampling offsets.

Declaration
public override void UpdateSamplingOffsets(List<OrientedPoint> samplingOffsets)
Parameters
Type Name Description
List<OrientedPoint> samplingOffsets

The sampling offsets to update the sensing component with.
Overrides

UpdateSensingComponent(SamplingStamps)

Updates the child that implements the sensing component.

Declaration
public override void UpdateSensingComponent(SamplingStamps samplingStamps)
Parameters
Type Name Description
SamplingStamps samplingStamps

List of sampling stamps collected on the photosensor node.
Overrides