System Task Manager (STM)

Large-scale deployment of autonomous vehicles (AVs) requires the reconciliation of two competing goals: performance and safety. AV systems also need to be deterministic so that their worst-case behavior can be validated for satisfying timing guarantees. On a complex SoC like the NVIDIA? DRIVE? Orin? SoC, the behavior exhibited by any software component depends on the state of the SoC as exhibited by two factors: the state of other software components (locks, memory contention, etc.), and the state of the hardware (caches, physical memory, etc.).

Heterogenous AV Workload

To solve the problems associated with dynamic scheduling, we present a static, centrally monitored, OS agnostic, non-preemptive scheduler that manages work across hardware engines on NVIDIA DRIVE Orin? SoC. Our scheduling framework also offers methodologies and tools to both characterize and tune the interactions between tasks on different hardware engines.

STM Framework

Supported Hardware Engines

STM provides the ability to schedule Runnables across the various hardware engines on the NVIDIA DRIVE Orin? SoC as well as any discrete GPUs attached to NVIDIA DRIVE Orin? SoC. Below is a short overview of the supported hardware engines and software libraries.

• CPU: STM can schedule Runnables across the different CPUs in NVIDIA DRIVE Orin? SoC’s CCPLEX. A CPU Runnable is represented as a callback function (C/C++) that is registered with the STM run-time. STM maintains a pool of worker threads that will call the clients registered functions at the appropriate offset in the schedule.
• GPU: STM can schedule GPU Runnables on NVIDIA DRIVE Orin? SoC’s iGPU as well as any discrete dGPUs attached to NVIDIA DRIVE Orin? SoC. If running on an x86 machine, only dGPUs are supported. STM only supports scheduling of CUDA? compute tasks and any higher-level libraries that are built on top of Cuda (TensorRT?, cuDNN, cuBLAS, DriveWorks, etc.).
• PVA: STM can schedule PVA Runnables on NVIDIA DRIVE Orin SoC’s PVA engine using the APIs in the wrappers provided with STM. The PVA's capabilities are a good good match in the algorithmic domains where we need to have predictable processing, at low power and low latency.