In this section, we will look at the demo structure in more detail. The demo uses four cameras and a DNN based object detection node. Below is a block diagram of this demo:
A graphical editor tool, known as the DW Graph UI tool, can also be used to open the graphlet JSON file for visualization. Please use the released graphical editor tool released to open CGFDemo.graphlet.json file. An example of the graphical output of this demo is shown below:
Zooming into camera graphlet in the compute graph:
As described earlier, the cameraSensor graphlet contains two nodes, dwCameraNode and ISPNode. By double clicking on the cameraSensor box, a new tab in the GUI tool will be created showing the nodes inside the camera graphlet shown as below:
The division of the processes is defined in CGFDemo.schedule.json file. The processes are divided as the following:
There are four instances of camera pipelines: cameraPipeline0 - 3. Each pipeline is defined in the CameraPipeline graphlet JSON file. In the CameraPipeline graphlet JSON, it includes three sub-graphlets: CameraSensor, CameraPreProcessingDemo, and CameraObjectDetectorDemo. To view the graphlet in graphical form, you can use the GUI editor tool or simply check the connections section of JSON to determine how the graphlets are connected.
The DNN implementation is based on the sample application "sample_object_detector_tracker". It uses tensorRT_model.bin from the sample app to perform object detection on four 2MP camera recordings. The C++ implementation of this node follows the code structure defined previously in the custom node section.
To render the output of the detection node onto display, an input, CUSTOM_OBJECT_DETECTION_ARRAY, exists in RenderingCGFDemoNode to receive the output of CameraObjectDetectorCGFDemoNode node. The connection takes in an array of detected bounding boxes.
To launch the demo, copy nvsciipc.cfg from targets/<x86_64-Linux or aarch64-Linux>/config/nvsciipc.cfg into /etc/ folder and run script in bin/config/run_cgf.sh. Use the following command to launch the demo:
"sudo ./run_cgf.sh -p /usr/local/driveworks/src/cgf/graphs/descriptions/systems/CGFDemo.system.json -s /usr/local/driveworks/src/cgf/graphs/CGFDemo.stm"I
The default view will show the following:
To view feature tracker rendering, use the following key sequence "nvidia", then 'h' key. To view self-calibration status, press F12 button
For each runnable defined in CGFDemo pipeline to be modeled as an STM runnable, the workload must satisfy the following requirements:
In the following, we will take nvidia_computegraphframework_linux-amd64-ubuntu/apps/roadrunner-2.0/graphs/descriptions/systems/CGFDemo.schedule.json as an example for further elaboration.
{
"resources": {
"TegraA": {
"CPU": ["CPU0", "CPU1", "CPU2", "CPU3", "CPU4", "CPU5"],
"GPU": ["dGPU"]
},
"TegraB": {
}
},
"hyperepochs": {
"cameraHyperepoch": {
"resources": [
"TegraA.CPU0",
"TegraA.CPU1",
"TegraA.CPU2",
"TegraA.CPU3",
"TegraA.dGPU",
"camera_master.TegraA.CUDA_STREAM0",
"camera_master.TegraA.CUDA_MUTEX_LOCK",
"camera_pipeline0.TegraA.CUDA_STREAM0",
"camera_pipeline0.TegraA.CUDA_MUTEX_LOCK",
"camera_pipeline1.TegraA.CUDA_STREAM0",
"camera_pipeline1.TegraA.CUDA_MUTEX_LOCK",
"camera_pipeline2.TegraA.CUDA_STREAM0",
"camera_pipeline2.TegraA.CUDA_MUTEX_LOCK",
"camera_pipeline3.TegraA.CUDA_STREAM0",
"camera_pipeline3.TegraA.CUDA_MUTEX_LOCK"
],
"period": 33000000,
"epochs": {
"renderEpoch": {
"period": 33000000,
"frames": 1,
"passes": [
"arender"
]
},
"cameraEpoch": {
"period": 33000000,
"frames": 1,
"passes": [
"cameraEpochSync",
"cameraPipeline0",
"cameraPipeline1",
"cameraPipeline2",
"cameraPipeline3",
"selfCalibration"
]
},
"radarDopplerMotionEpoch": {
"period": 33000000,
"frames": 1,
"passes": [
"radarDopplerMotion"
]
}
}
},
"imuHyperepoch": {
Hyperepochs and epochs define the timing boundaries for tasks (runnables). Clients define the data boundaries. A client is an operating system process that contains software resources (like CUDA streams) and runnables.
"clients": { "render": { "executable": "LoaderLite", "dwProgramMaster": "false", "subcomponents": [ "arender" ], "resources": {} }, "camera_pipeline0": { "executable": "LoaderLite", "dwProgramMaster": "false", "subcomponents": [ "cameraPipeline0" ], "resources": { "CUDA_STREAM": ["CUDA_STREAM0:dGPU"], "MUTEX": ["CUDA_MUTEX_LOCK"] }, "resourcesAssignment": { "CUDA_MUTEX_LOCK": [ "cameraPipeline0.cameraSensor.cameraNode.PASS_RAW_OUTPUT_CPU_SYNC", "cameraPipeline0.cameraSensor.cameraNode.PASS_PROCESSED_OUTPUT_CPU_SYNC" ] } }, "camera_pipeline1": { "executable": "LoaderLite", "dwProgramMaster": "false", "subcomponents": [ "cameraPipeline1" ], "resources": { "CUDA_STREAM": ["CUDA_STREAM0:dGPU"], "MUTEX": ["CUDA_MUTEX_LOCK"] }, "resourcesAssignment": { "CUDA_MUTEX_LOCK": [ "cameraPipeline1.cameraSensor.cameraNode.PASS_RAW_OUTPUT_CPU_SYNC", "cameraPipeline1.cameraSensor.cameraNode.PASS_PROCESSED_OUTPUT_CPU_SYNC" ] } }, "camera_pipeline2": { "executable": "LoaderLite", "dwProgramMaster": "false", "subcomponents": [ "cameraPipeline2" ], "resources": { "CUDA_STREAM": ["CUDA_STREAM0:dGPU"], "MUTEX": ["CUDA_MUTEX_LOCK"] }, "resourcesAssignment": { "CUDA_MUTEX_LOCK": [
NvSciStream channel feature in the release, that supports a single producer and a single consumer, is enabled. Some of the channels with a single producer and a single consumer have been updated to NvSciStream in the demo. In graphlet file CGFDemo.graphlet.json, in the connection section, type parameter were changed from socket to nvsci:
"connections": [
{
"src": "cameraPipeline0.NEXT_IMAGE_TIMESTAMP",
"dests": {"cameraEpochSync.SENSOR_TIMESTAMP[0]": {}},
"params": {"type": "nvsci", "indirect": true}
},
...
]
In addition, nvsciipc.cfg file under directory /etc was modified to add additional ipc slots for NvSciStream to use. Each additional line is a slot allocated for each producer/consumer NvSciStream channel:
INTER_PROCESS cgf_0_p cgf_0_c 16 24576
The naming convention is cgf_<connection number>_<producer/consumer>
Please make sure there are 32 slots allocated inside the nvsciipc.cfg file. After the file has been updated, it is necessary to reboot the system for changes to take effect. To make sure the changes has been properly applied, please check with the following command:
sudo service nv_nvsciipc_init status should print message below:
nv_nvsciipc_init.service - NvSciIpc initialization
Loaded: loaded (/lib/systemd/system/nv_nvsciipc_init.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Wed 2021-06-16 20:36:43 UTC; 52min ago
Process: 758 ExecStart=/bin/sh /etc/systemd/scripts/nv_nvsciipc_init.sh (code=exited, status=0/SUCCESS)
Main PID: 758 (code=exited, status=0/SUCCESS)
If not, please double check your nvsciipc.cfg file.
1.9.4