Parallel stereo camera

The parallel stereo camera emulates a depth-sensing camera based on stereoscopic vision.

Implementation details

This camera uses two red cameras (bound to game objects of the scene) and triangulate pixels to figure out a depth for each pixel.

The baseline between cameras is computed once at the start of execution and is assumed constant. The transformation from left view to right view must be a pure non-zero translation along the positive X axis, otherwise an error is reported.

The pixel matching is sensitive to variations in the frame. A flat featureless surface will result in failure to find a good depth.

To reduce noise, a speckle filter is applied after pixel matching. The filter invalidates the pixel if there is too much disparity variation in its neighborhood. Disparity variation is defined as the difference between the minimum and maximum disparities inside the speckle filter window.

The current implementation assumes pixels are square, and thus requires the image aspect ratio (ResolutionH/ResolutionV) to be the same as the sensor aspect ratio (SensorWidth/SensorHeight).

Inputs

Input	Description
ResolutionH	Horizontal number of photocells on the photosensor array.
ResolutionV	Vertical number of photocells on the photosensor array.
SensorWidth	Physical width of the photosensor array, in millimeters.
SensorHeight	Physical height of the photosensor array, in millimeters.
FramesPerSecond	Number of captured frames per second. Automatically capped to 1 / ExposureTime.
ExposureTime	The amount of time the photosensor is capturing light, in seconds.
Gain	Amplification factor applied to the captured light. 1.0 is the default gain; more than 1.0 is brighter; less than 1.0 is darker.
PatchSize	Kernel side width used to make the pixel's pattern matching. Between 3 and 31. Must be odd.
DisparityMin	Distance, from the pixel we want to match, to start the pattern matching.
DisparityMax	Maximum distance from the pixel we want to match to execute the pattern matching.
FocalLength	Distance between the lens and the sensor, in millimeters.
SpeckleWindow	The size of the window, in pixels, for the speckle filter. Must be odd. Set to 0 to disable.
DisparityVariation	The maximum allowed disparity variation, in pixels, inside the speckle filter window.

Outputs

Output	Description
Left	A render texture of the left view, resident on the GPU with intensity stored in lumens.
Right	A render texture of the right view, resident on the GPU with intensity stored in lumens.
Depth	A render texture of the depth (Z coordinate), resident on the GPU.
PointCloud	A render texture with the (x, y, z) position for each pixels.
OutTranscode	Provides a context for executing sampling custom passes, which then passes through the GPU processing pipeline.