Lenses

SensorSDK can emulate camera lenses in the Unity editor. Lenses use a LensComponent and bind to a compatible SystemGraph node, such as PhotoSensorArray.

Thin lens

The thin lens implements a lens component that reproduces a thin lens model. This lens is commonly used in game development.

Fish-Eye lens

This post processing lens is a consumer of a CubemapSampler, which renders the scene in up to six directions corresponding to the faces of a cube. Each of the renders are done at 90 degree field of view and stitched together using the fish-eye projection mapping. This lens uses the equidistant mapping (also called tru-theta mapping). See Paul Bourke's page on Fisheye lens correction.

In the following, all coordinates are normalized device coordinates in the image plane.

In the equidistant projection, angular distances are preserved, which means that the radial distance of a pixel in the image plane is directly proportional to the angle the object projected onto that pixel would make with the optical axis, in short: $r = f\theta$ .

For ease of calculation, consider parameter F which is half of the sensor field of view, such that $r = theta over F$ .

For a given pixel in the output image denoted by $x_o$ :

//: # "&r^2 = a_o^2 + b_o^2 = \left( \frac{\theta}{F} \right) ^2 \" //: # "&\phi = \text{atan2}(b_o, a_o)" //: # "\end{align}"

The light ray from the camera for the corresponding pixel can be calculated using:

//: # "x &= cos(\phi) \sin(\theta) & &= \frac{a_o}{r} \sin(Fr) \" //: # "y &= sin(\phi) \sin(\theta) & &= \frac{b_o}{r} \sin(Fr) \" //: # "z &= cos(\theta) & &= cos(Fr)" //: # "\end{align*}"

Distortion is added to this model using the following polynomial model to alter the theta, bending the ray either away or towards the center based on the distortion:

where ![bullet_d](../images/latex/bullet_d.svg) subscript implies the distorted value and ![bullet_u](../images/latex/bullet_u.svg) subscript is the corresponding undistorted value. ![K1-K5](../images/latex/K1K5.svg) are the parameters that control the type and level of distortion. Note that this distortion removes the angular distance preserving property.

When calculating the vector $ar$ , multiple values of $a_o$ and $b_o$ , slightly offset from each other, are used for antialiasing. You can configure this number using the Samples Per Pixel property.

Brown-Conrady lens

This lens applies a post processing effect on the camera. It renders at a higher resolution and higher field of view to support distortion without creating artifacts or black borders around the image.

The implementation is a Brown-Conrady model used to calculate the distorted pixels from undistorted pixels. The lens uses the following formula:

with the following variables:

$x_c$ is the center of distortion.
$k_1-k_4$ are the radial distortion parameters and $x_d, x_u$ are the distorted and undistorted points respectively.
$r = \lVert x_d - x_c \rVert$ is a tangential distortion parameter.
$p_1-p_4$ is a tangential distortion parameter.
$T$ represents the following function:

//: # "T^{0}(x_d, x_c) = [p_1 [r^2 + 2(x_d^0 - x_c^0)^2] + 2p_2(x_d^0 - x_c^0)(x_d^1 - x_c^1)](1 + p_3r^2 + p_4r^4) \" //: # "T^{1}(x_d, x_c) = [p_2 [r^2 + 2(x_d^1 - x_c^1)^2] + 2p_1(x_d^0 - x_c^0)(x_d^1 - x_c^1)](1 + p_3r^2 + p_4r^4)" //: # "\end{align}}"

$\bullet^{n}$ refers to the nth coordinate of the corresponding vector.

Segmentation lens

This lens generates a label value for each object in the scene, instead of an intensity or color image. This lens is solely used as a ground truth. For example, you can use this to train a machine learning algorithm that segments a scene into objects.

For an example of how to use the segmentation lens, see the segmentation camera page.