NvDisplay
NvDisplay is the display architecture supported on Linux starting in NVIDIA DRIVE? OS 6.0.
Display State Manager
Drive Setmode
When driver setmode is enabled, display setup + modeset will happen during the resmgr init phase instead of deferring this to when the first display client comes up. So, enabling driver setmode can improve "power-on to first pixel visible on screen" latency. On QNX safety, driver setmode is enabled by default. On QNX Standard, driver setmode can be enabled using Device Tree. Driver setmode can be enabled only for the configuration which uses DP Serializer, on other configurations enabling it will fail to load display driver.
\ {
display@13800000 {
nvidia,driver-setmode;
};
};
OpenWFD Modeset
- When a OpenWFD client first initializes the WFD library, OpenWFD queries NvKms for the
current mode using NVKMS_IOCTL_GET_CURRENT_MODE
- If current mode is not NULL:
-
OpenWFD exposes the sole active mode to clients and skips invoking the NvKms APIs for mode query and verification
-
-
Any client requests for modesets (via wfdSetPortMode) will be ignored by the OpenWFD driver (with a message logged in slog2info. This is not an error message)
- This enables existing client apps to work as-is without any changes.
-
-
- If current mode is NULL:
-
OpenWFD uses NvKms' mode query and validation IOCTLs to build a list of available modes and set one out of these modes as the preferred one (this is usually reported by NvKms)
-
At runtime OpenWFD performs a modeset iff the WFD client invokes wfdSetPortMode followed by a wfdDeviceCommit. This is the legacy way of doing modesets and all current WFD apps are already following this approach.
-
- If current mode is not NULL:
Components
The following are NvDisplay components and their libraries:
- OpenWFD (
libtegrawfd.so
) - NvKms (
nvidia-modeset.ko
)
OpenWFD
OpenWFD (Open Windowing Foundation Display) is a Khronos API that provides a low-level hardware abstraction interface for windowing systems and applications to make use of display hardware.
Applications that need to interact with NvDisplay should do so via the OpenWFD API.
The specification of the OpenWFD API is available at Khronos' registry at https://www.khronos.org/registry/OpenWF/specs/OpenWF_Display_1_0_Specification.pdf (hereby referred to as the spec).
The NvDisplay architecture on NVIDIA DRIVE? OS includes an
OpenWFD driver (libtegrawfd.so
– hereby referred to as the WFD driver) that
supports a subset of the core OpenWFD APIs listed in the spec along with additional NVIDIA
specific extensions.
Usage guidelines for the core OpenWFD APIs can be obtained from the spec and any deviations from the spec or additional details will be described in this document.
Supported OpenWFD APIs
The following table lists all the core OpenWFD APIs and the available support for them on the WFD driver.
API |
Support in libtegrawfd.so |
---|---|
wfdGetStrings | Supported on Linux build |
wfdIsExtensionSupported | Supported on Linux build |
wfdGetError | Supported on Linux build |
wfdEnumerateDevices | Supported on Linux build |
wfdCreateDevice | Supported on Linux build |
wfdDestroyDevice | Supported on Linux build |
wfdDeviceCommit | Supported on Linux build |
wfdGetDeviceAttribi | Supported on Linux build |
wfdSetDeviceAttribi | Supported on Linux build |
wfdCreateEvent | Not supported |
wfdDestroyEvent | Not supported |
wfdGetEventAttribi | Not supported |
wfdDeviceEventAsync | Not supported |
wfdDeviceEventWait | Not supported |
wfdDeviceEventFilter | Not supported |
wfdEnumeratePorts | Supported on Linux build |
wfdCreatePort | Supported on Linux build |
wfdDestroyPort | Supported on Linux build |
wfdGetPortModes | Supported on Linux build |
wfdGetPortModeAttrib{i/f} | Supported on Linux build |
wfdSetPortMode | Supported on Linux build |
wfdGetCurrentPortMode | Supported on Linux build |
wfdGetPortAttribi | Supported on Linux build |
wfdGetPortAttribf | Supported on Linux build |
wfdGetPortAttribiv | Supported on Linux build |
wfdGetPortAttribfv | Supported on Linux build |
wfdSetPortAttribi | Supported on Linux build |
wfdSetPortAttribf | Supported on Linux build |
wfdSetPortAttribiv | Supported on Linux build |
wfdSetPortAttribfv | Supported on Linux build |
wfdBindPipelineToPort | Supported on Linux build |
wfdGetDisplayDataFormats | Not supported |
wfdGetDisplayData | Not supported |
wfdEnumeratePipelines | Supported on Linux build |
wfdCreatePipeline | Supported on Linux build |
wfdDestroyPipeline | Supported on Linux build |
wfdCreateSourceFromImage | Supported on Linux build |
wfdCreateSourceFromStream | Not supported |
wfdDestroySource | Supported on Linux build |
wfdCreateMaskFromImage | Not supported |
wfdCreateMaskFromStream | Not supported |
wfdDestroyMask | Not supported |
wfdBindSourceToPipeline | Supported on Linux build |
wfdBindMaskToPipeline | Not supported |
wfdGetPipelineAttribi | Supported on Linux build |
wfdGetPipelineAttribf | Supported on Linux build |
wfdGetPipelineAttribiv | Supported on Linux build |
wfdGetPipelineAttribfv | Supported on Linux build |
wfdSetPipelineAttribi | Supported on Linux build |
wfdSetPipelineAttribf | Supported on Linux build |
wfdSetPipelineAttribiv | Supported on Linux build |
wfdSetPipelineAttribfv | Supported on Linux build |
wfdGetPipelineTransparency | Supported on Linux build |
wfdSetPipelineTSColor | Not supported |
wfdGetPipelineLayerOrder | Supported on Linux build |
The following OpenWFD attributes have limited support for querying:
- WFD_PORT_BACKGROUND_COLOR – Cannot be queried via wfdGetPortAttribi API. wfdGetPortAttribiv and wfdGetPortAttribfv APIs can still be used for querying WFD_PORT_BACKGROUND_COLOR.
Supported OpenWFD Extensions
The following OpenWFD extensions are supported by the OpenWFD Driver.
WFD_NVX_create_source_from_nvscibuf
WFD_NVX_commit_non_blocking
WFD_NVX_nvscisync
WFD_NVX_port_mode_timings
These extensions are described in the NVIDIA DRIVE? OS 6.0 SDK API Reference.
OpenWFD Usage Guidelines
- The macro
WFD_NVX_create_source_from_nvscibuf
needs to be defined before including the headerswfd.h
andwfdext.h
for enabling the WFD_NVX_create_source_from_nvscibuf extension. - The macro
WFD_WFDEXT_PROTOTYPES
must be defined before including thewfd.h
andwfdext.h
headers. - To clear the contents on a
WFDPipeline
(performing a null flip),wfdBindSourceToPipeline
must be called with0
as the argument forWFDSource
before callingwfdDeviceCommit
on theWFDPipeline
. This is demonstrated in theopenwfd_nvsci_sample
application present in the SDK. - During the deinitialization of OpenWFD applications, you must follow these steps:
- Perform a null flip on all
WFDPorts
andWFDPipelines
that were previously bound to aWFDSource
.- If nonblocking commits are being used and
WFD_PIPELINE_POSTFENCE_SCANOUT_BEGIN_NVX
is set toWFD_TRUE
, the post-flip fence of the null flip commit should be waited on. - If nonblocking commits are being used and
WFD_PIPELINE_POSTFENCE_SCANOUT_BEGIN_NVX
is set toWFD_FALSE
, the post-flip fence of thewfdDeviceCommit
call before the null flip commit should be waited on.
- If nonblocking commits are being used and
- Destroy all
WFDSource
objects by invokingwfdDestroySource
on them only after all null flip commits are complete. BlockingwfdDeviceCommit
calls ensures that the null flip is complete when thewfdDeviceCommit
call returns. For nonblockingwfdDeviceCommit
calls, synchronization must be handled by the OpenWFD client as described previously. - Destroy all
WFDPipeline
andWFDPort
objects by usingwfdDestroyPipeline
andwfdDestroyPort
, respectively, before finally destroying theWFDDevice
object usingwfdDestroyDevice
.
- Perform a null flip on all
- For committing to multiple
WFDPipeline
handles bound to the sameWFDPort
, it is recommended to callwfdDeviceCommit
* withWFDCommitType
set toWFD_COMMIT_ENTIRE_PORT
. - When you use
WFD_COMMIT_ENTIRE_PORT
in awfdDeviceCommit
call, all theWFDPipeline
objects part of the commit must either have a surface bound(normal flip) or have no surfaces bound (null flip). A mix of normal and a null flip in oneWFD_COMMIT_ENTIRE_PORT
wfdDeviceCommit
call is not allowed due to a limitation on the NvDisplay resource manager. To ensure that any errors are caught, the OpenWFD APIwfdGetError
must be invoked after calls to the following OpenWFD APIs that take in aWFDDevice
handle as one of the inputs and return void:- wfdDeviceCommit
- wfdDeviceCommitWithNvSciSyncFenceNVX
- wfdDestroyPort
- wfdSetPortMode
- wfdGetPortAttrib{iv/fv}
- wfdSetPortAttrib{i/v/iv/fv}
- wfdBindPipelineToPort
- wfdDestroyPipeline
- wfdDestroySource
- wfdBindSourceToPipeline
- wfdGetPipelineAttrib{iv/fv}
- wfdSetPipelineAttrib{i/v/iv/fv}
-
When flipping to display, you should use a minimum of two
WFDSource
handles (double buffering) to prevent tearing or corruption.For more details, see section 8.1 in the OpenWF Display Specification.
-
The following prerequisites must be fulfilled for use cases to suspend to RAM (SC7 mode):
- Before entering suspend-to-RAM (SC7) mode, it is recommended to perform a null flip to display.
- To exit SC7 mode, a
modeset
must be performed before flipping to display.
Display Serializer
The following sections describe the display serializer.
Configuring Video Timings
For both SST and MST mode, the mode timings that are used for each stream must be configured in Device Tree. Only one mode timing can be specified at a time for each video stream. The timings that are exposed in the EDIDs of the serializer and the panels connected to the downstream deserializer are completely ignored.
An example Device Tree fragment is shown below. In this example, a standard 1920x1080 at 60 Hz timing is specified for the first video stream, and a 1280x720 at 60 Hz timing is specified for the second video stream:
\ {
display@13800000 {
display-timings {
display-connector-0 {
dcb-index = <0>;
stream-0 {
timings-phandle = <&mode0>;
};
stream-1 {
timings-phandle = <&mode1>;
};
};
};
mode0: 1920-1080-60Hz {
clock-frequency-khz = <148500>;
hactive = <1920>;
vactive = <1080>;
hfront-porch = <88>;
hback-porch = <148>;
hsync-len = <44>;
vfront-porch = <4>;
vback-porch = <36>;
vsync-len = <5>;
rrx1k = <60000>;
pps-data = [
11 00 00 89 30 80 04 38
07 80 04 38 03 c0 03 c0
02 00 03 58 00 20 73 3e
00 0d 00 0f 00 1d 00 0e
18 00 10 f0 03 0c 20 00
06 0b 0b 33 0e 1c 2a 38
46 54 62 69 70 77 79 7b
7d 7e 01 02 01 00 09 40
09 be 19 fc 19 fa 19 f8
1a 38 1a 78 22 b6 2a b6
2a f6 2a f4 43 34 63 74
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 ];
};
mode1: 1280-720-60Hz {
clock-frequency-khz = <74250>;
hactive = <1280>;
vactive = <720>;
hfront-porch = <110>;
hback-porch = <220>;
hsync-len = <40>;
vfront-porch = <5>;
vback-porch = <20>;
vsync-len = <5>;
rrx1k = <60000>;
};
};
};
- "display@13800000" is the overall parent node for the entire display device.
This node already exists today.
- "display-timings" is used to specify which timings are used for each
stream.
- "display-connector-0" specifies the timing information for
the first display connector. If there are multiple display
connectors present on the board that require fixed timings,
then a new "display-connector" node must be created for each
connector.
- "dcb-index" specifies the logical index X of the DCB -> Display Devices -> Display Device X entry in the display DCB blob that this connector entry applies to. If there is only one display connector on the board, then "dcb-index" defaults to 0.
- The "stream" nodes specify the phandle of the mode timing node that applies to the given video stream.
- "display-connector-0" specifies the timing information for
the first display connector. If there are multiple display
connectors present on the board that require fixed timings,
then a new "display-connector" node must be created for each
connector.
- Each "mode" node contains the actual mode timing parameters that
will be used for a given video stream.
- "clock-frequency-khz": Pixel clock frequency in KHz
- "hactive": Horizontal active
- "vactive": Vertical active
- "hfront-porch": Horizontal front porch
- "hback-porch": Horizontal back porch
- "hsync-len": Horizontal sync width
- "vfront-porch": Vertical front porch
- "vback-porch": Vertical back porch
- "vsync-len": Vertical sync width
- "rrx1k": Refresh rate in units of 0.001Hz
- "pps-data": All 128B of the DSC PPS
This property should be specified if DSC will be enabled for the given timing.
- "display-timings" is used to specify which timings are used for each
stream.
Each "display-connector" can only have up to two (2) "stream" nodes. Note that it is fine to specify two (2) "stream" nodes even if the display serializer operates in SST mode because only the first "stream" node is consumed by the display driver. The extra node is ignored.
Configuring the Serializer Driver
If you are using the NVIDIA reference Maxim serializer driver, there are various ways to configure the driver by Device Tree. An example Device Tree fragment is shown below that configures the Maxim serializer chip in MST mode:
i2c@31e0000 {
status = "okay";
maxim_ser: max_gmsl_dp_ser@40 {
compatible = "maxim,max_gmsl_dp_ser";
reg = <0x40>;
status = "okay";
max_gmsl_dp_ser-pwrdn = <&tegra_main_gpio TEGRA234_MAIN_GPIO(G, 3) GPIO_ACTIVE_HIGH>;
ser-errb = <&tegra_main_gpio TEGRA234_MAIN_GPIO(G, 7) 0>;
dprx-link-rate = <0x1e>;
dprx-lane-count = <0x4>;
enable-mst;
mst-payload-ids = <0x1 0x3 0x2 0x4>;
gmsl-stream-ids = <0x0 0x1 0x2 0x3>;
gmsl-link-select = <0x0 0x0 0x1 0x1>; enable-dp-fec;
enable-dsc = <1 0>;
enable-gmsl-fec = <1 0>;
};
};
A description of each of the preceding properties:
- Required properties:
- compatible: Must be "maxim,max_gmsl_dp_ser".
- reg: I2C address of the Maxim display serializer.
- max_gmsl_dp_ser-pwrdn: GPIO pin number of the PWRDN pin. This pin is used to power up the Maxim display serializer chip.
-
gmsl-link-select: This property is an array of four unsigned 8-bit values that determines the GMSL output link to enable for each video pipe X, Y, Z, and U. The possible values for each pipe are:
- 0x0 (Link A)
- 0x1 (Link B)
- 0x2 (Link A + B)
- Optional properties:
-
dprx-link-rate: Configures the DP link rate of the serializer chip.
The default value is 0x1E (HBR3). The possible values are:- 0xA (HBR)
- 0x14 (HBR2)
- 0x1E (HBR3)
-
dprx-lane-count: Configures the DP lane count of the serializer chip.
The default value is 0x4. The possible values are:- 0x1
- 0x2
- 0x4
-
ser-errb: GPIO pin number of the ERRB pin. This pin is used for error and fault reporting by the serializer chip.
-
enable-mst: This is a Boolean property. If this property is present, the driver will enable MST mode.
-
mst-payload-ids: This property is an array of four unsigned 8-bit values, which represent MST payload IDs of pipe X, Y, Z, U. This property is mandatory if enable-mst property is mentioned in dt.
-
gmsl-stream-ids: This property is an array of four unsigned 8-bit values, which represent GMSL stream IDs of pipe X, Y, Z, U. This property is mandatory if enable-mst property is mentioned in dt.
- enable-dp-fec: This is a boolean property. When this property is present, the driver will enable FEC on the DP link if the serializer chip supports it.
- enable-dsc: This property is an array of two 32-bit values, where each value indicates whether DSC is enabled or not. The first entry corresponds to video pipe X, and the second entry corresponds to video pipe Y. DSC is only supported on pipe X currently.
- enable-gmsl-fec: This property is an array of two 32-bit values, where each value indicates whether FEC is enabled on the GMSL link. The first entry corresponds to GMSL Link A, and the second entry corresponds to GMSL Link B.
-
Modeset Limitations
The Maxim display serializers do not support dynamic mode changes in MST mode without requiring a reset of the serializer chip in-between. In this context, a mode change refers to changing the number of video streams and/or the display timings that are used for each stream. As such, if the Maxim display serializers are configured in MST mode, it is recommended to always enable all video streams that will be used at once.
Head to Window Assignment
The following sections describe head to window assignment.
Configuring Head to Window Assignment
Configure head to window assignment in the Device Tree. If an assignment is not specified in the DT, the driver assigns windows (2N) and (2N + 1) to HEAD N. In DT, specify the assignment using 64 bit mask, which is interpreted as:
Head-Bitmask | Window-Number |
---|---|
BITMASK(0-7) | 0 |
BITMASK(8-15) | 1 |
BITMASK(16-23) | 2 |
BITMASK(24-31) | 3 |
BITMASK(32-39) | 4 |
BITMASK(40-47) | 5 |
BITMASK(48-55) | 6 |
BITMASK(56-63) | 7 |
The display driver fails to load if an invalid assignment is specified in the DT. The specified assignment must adhere to the conditions below:
- The specified window number must be supported by hardware.
- The specified head number must be supported by hardware.
- The same window must not be assigned simultaneously for multiple heads.
- At least one window must be assigned to at least one head (that is, specified window-head mask should not be 0).
The display driver culls the head with no windows assigned, and all the heads above it. For example, if hardware supports three heads and the user uses the assignment mask to assign valid windows to head-0 and head-2 but no windows to head-1, then head-1 and head-2 is culled.
An example Device Tree fragment is shown below. In this example, window-0, window-1, and window-2 is assigned to head-0 and window-3 is assigned to head-1.
\ {
display@13800000 {
nvidia,window-head-mask = <0x00000000 0x02010101>;
};
};
In the preceding example:
-
display@13800000
is the overall parent node for the entire display device. This node already exists today.nvidia,window-head-mask
is used to specify the 64-bit window head assignment mask.
Restrictions
NvDisplay architecture has the following restrictions:
- Only one NvDisplay client process can be active at a time on the NVIDIA DRIVE
Orin? platforms.
- On the NVIDIA DRIVE? OS Linux NVIDIA Orin platforms, nvidia-drm should not be installed when you run non-DRM applications.
- NVIDIA DRIVE? OS does not support passive DP-to-HDMI (DP++) cables, dongles, or adapters.