Warning

Warning: Unity Simulation is deprecated as of December 2023, and is no longer available.

Stream data to an external process

Some simulation use cases may require data to be streamed to and from an external process. This can come in handy in cases of hardware-in-the-loop simulations, where the data or rendered frames need to be further streamed to an external process like NVIDIA Isaac, a projected display or ROS.

To stream frames, the produced frames can be saved to disk using the sample FrameCaptureEvent component provided in the Distributed Rendering package along with the onClientFrameBegin event. To stream custom data, the onClientInjectCustomData and onServerReceivedCustomData events can be used, which can then be received externally.

Streaming to an external process: Rendered frames

Distributed Rendering enables users to capture each frame once it is rendered on the Clients by triggering a Frame Capture using the FrameCaptureEvent component in the package. This component enables users to save their frames as images to their disk, from where these frames can further be streamed to an external process.

To capture Client frames the FrameCaptureEvent needs to be added to a GameObject in the scene. This component contains the RGB capture API in the TriggerFrameCapture function. The frame capture itself must be enabled by checking the "Enable Capture" box on the FrameCaptureEvent component.

Passing the method to the On Client Frame Begin event is optional since this method is automatically passed into this event using the [DistributedRendering(EventType.OnClientFrameBegin)] attribute.

The captures from this event are not cleaned up on exit, and the unique path to the captures is logged into the player log for each corresponding Client. Below we have the results of the FrameCaptureEvent method on a scene with a spinning capsule in the center.

Captures of the spinning Capsule scene at the a) 0th Frame b) 499th Frame c) 999th Frame

To access the saved images, navigate to the capture path as mentioned in the log file. By default logs are stored to the following locations:

  • Linux: ~/.config/unity3d/CompanyName/ProductName/Player.log

  • macOS: ~/Library/Logs/Company Name/Product Name/Player.log

  • Windows: C:\Users\username\AppData\LocalLow\CompanyName\ProductName\Player.log

These logs can also be accessed by adding the -logFile <path_to_store_log_file> argument while running the Server or Client on Terminal or Command Line, or '-logFile -' to log the output directly to Terminal (not applicable to Windows). More information on log files can be found here.

A detailed tutorial for frame capture can be found in this demo.