System Task Manager (STM)

Large-scale deployment of autonomous vehicles (AVs) requires the reconciliation of two competing goals: performance and safety. AV systems also need to be deterministic so that their worst-case behavior can be validated for satisfying timing guarantees. On a complex SoC like the NVIDIA? DRIVE? Orin? SoC, the behavior exhibited by any software component depends on the state of the SoC as exhibited by two factors: the state of other software components (locks, memory contention, etc.), and the state of the hardware (caches, physical memory, etc.). Dynamic scheduling and preemption of tasks by engine-specific schedulers lead to an explosion in the size of the state space that must be validated and certified for timing determinism.

Heterogenous AV Workload

To solve these problems associated with dynamic scheduling, STM offers a static, centrally monitored, OS agnostic, non-preemptive scheduler that manages work across hardware engines on NVIDIA DRIVE Orin? SoC. The STM scheduling framework also offers methodologies and tools to both characterize and tune the interactions between tasks on different hardware engines.

STM Framework

Supported Hardware Engines

STM provides the ability to schedule tasks across the various hardware engines on the NVIDIA DRIVE Orin? SoC as well as any discrete GPUs attached to NVIDIA DRIVE Orin? SoC or x86 Platforms. Below is a short overview of the supported hardware engines and software libraries.

• CPU: STM can schedule tasks across the different CPUs in NVIDIA DRIVE Orin? SoC’s CCPLEX.
• GPU: STM can schedule GPU tasks on NVIDIA DRIVE Orin? SoC’s iGPU as well as any discrete GPUs attached to NVIDIA DRIVE Orin? SoC or x86 platforms. If running on an x86 machine, only discrete GPUs of compute capability 7.5 or higher are supported. STM only supports scheduling of CUDA? compute tasks and any higher-level libraries that are built on top of Cuda (TensorRT?, cuDNN, cuBLAS, DriveWorks, etc.).
• PVA: STM can schedule PVA tasks on NVIDIA DRIVE Orin SoC’s PVA engine using the APIs in the wrappers provided with STM. The PVA's capabilities are a good match in the algorithmic domains where we need to have predictable processing, at low power and low latency.