Name
NV_stream_remote
NV_stream_cross_object
NV_stream_cross_display
NV_stream_cross_process
NV_stream_cross_partition
NV_stream_cross_system
Name Strings
EGL_NV_stream_remote
Contributors
Daniel Kartch
Bogdan Naodovic
James Jones
Zander Clucas
Miguel A. Vico
Contacts
Daniel Kartch, NVIDIA (dkartch 'at' nvidia.com)
Status
Draft
Version
Version 5 - July 06, 2016
Number
EGL Extension #114
Extension Type
EGL display extension
Dependencies
Requires EGL_KHR_stream
Interacts with EGL_KHR_stream_cross_process_fd
All EGL_NV_stream_cross_* extensions require EGL_NV_stream_remote,
but are independent of each other.
Overview
This extension provides a generic framework for establishing stream
connections when the producer and consumer endpoints are not
associated with the same EGLStream object. The cross-process streams
allowed by the EGL_KHR_stream_cross_process_fd extension are one
example of such a connection. Other potential examples include
streams between objects associated with different EGLDisplays in a
single process, streams between processes residing in separate
virtual machine partitions on a single system, or streams between
entirely separate systems connected via the internet.
The approach used by EGL_KHR_stream_cross_process_fd has several
drawbacks:
- It creates a new pair of stream creation and query functions
which are specific to both the type of stream and the method
used for communicating the stream's identity. Extending this
methodology would require new pairs of functions for every new
type of stream connection.
- It allows creation of the connected stream objects without any
indication of which side will be the consumer and which will be
the producer. It even allows, although this was probably not
intended, a cross-process stream to be established, but have
both producer and consumer exist in the same process, with the
other process acting as a voyeur that can observe state and
perhaps modify attributes, but not operate on frames.
- The use of file descriptors generated by EGL in one process
and accessed by EGL in another to establish communication has
potential security flaws, and may not be available at all on
some real-time operating systems.
Rather than implement new connection types using this model, we
propose a more generic approach that does not require a
proliferation of new interfaces and avoids any ambiguities in how
the stream will be used. These extensions can coexist with
EGL_KHR_stream_cross_process_fd, but allow for more control of
stream communication by the applications.
These extensions provide the framework with which arbitrary stream
object pairs can be established, but by themselves are insufficient
to create such pairs. In addition, a separate extension must be
used to determine the protocol by which the two objects will
communicate. See for example the EGL_NV_stream_socket extension.
Several optional types of separate stream objects are defined,
with successive levels of remoteness. It should be assumed that
a more remote type will be less efficient in transfering images,
and therefore applications should choose the least remote type
available that is compatible with their use cases.
New Types
None
New Functions
None
New Tokens
Returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when
attribute is EGL_STREAM_STATE_KHR:
EGL_STREAM_STATE_INITIALIZING_NV 0x3240
Accepted as attribute names in eglCreateStreamKHR,
eglCreateStreamAttribKHR, eglQueryStreamKHR, and
eglQueryStreamAttribKHR:
EGL_STREAM_TYPE_NV 0x3241
EGL_STREAM_PROTOCOL_NV 0x3242
EGL_STREAM_ENDPOINT_NV 0x3243
Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and
returned by eglQueryStreamKHR and eglQueryStreamAttribKHR, as value
when attribute is EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV or
EGL_STREAM_ENDPOINT_NV:
EGL_STREAM_LOCAL_NV 0x3244
When EGL_NV_stream_cross_object is present, accepted by
eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
attribute EGL_STREAM_TYPE_NV:
EGL_STREAM_CROSS_OBJECT_NV 0x334D
When EGL_NV_stream_cross_display is present, accepted by
eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
attribute EGL_STREAM_TYPE_NV:
EGL_STREAM_CROSS_DISPLAY_NV 0x334E
When EGL_NV_stream_cross_process is present, accepted as
value for attribute EGL_STREAM_TYPE_NV by eglCreateStreamKHR
and eglCreateStreamAttribKHR. When either
EGL_NV_stream_cross_process or EGL_KHR_stream_cross_process_fd
is present, returned by eglQueryStreamKHR and
eglQueryStreamAttribKHR for attribute EGL_STREAM_TYPE_NV:
EGL_STREAM_CROSS_PROCESS_NV 0x3245
When EGL_NV_stream_cross_partition is present, accepted by
eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
attribute EGL_STREAM_TYPE_NV:
EGL_STREAM_CROSS_PARTITION_NV 0x323F
When EGL_NV_stream_cross_system is present, accepted by
eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
attribute EGL_STREAM_TYPE_NV:
EGL_STREAM_CROSS_SYSTEM_NV 0x334F
If EGL_KHR_stream_cross_process_fd is present, returned by
eglQueryStreamKHR and eglQueryStreamAttribKHR when attribute is
EGL_STREAM_PROTOCOL_NV:
EGL_STREAM_PROTOCOL_FD_NV 0x3246
Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and
returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when
attribute is EGL_STREAM_ENDPOINT_NV:
EGL_STREAM_PRODUCER_NV 0x3247
EGL_STREAM_CONSUMER_NV 0x3248
Add to "Glossary" of EGL_KHR_stream:
Local stream
A stream implemented with both consumer and producer attached to a
single EGLStream object.
Remote stream
A stream implemented with two EGLStream objects that communicate
with each other, with the consumer attached to one object and the
producer attached to the other.
Add to section "3.10.1 Creating an EGLStream" in EGL_KHR_stream:
If a remote EGLStream is created, as described in section "3.10.5.x
Remote streams", and is paired with an EGLStream object which has
conflicting values for some attributes, creation will succeed, but
the stream will enter state EGL_STREAM_STATE_DISCONNECTED_KHR once
communication is established.
Add to list of failures in section "3.10.1 Creating an EGLStream" in
EGL_KHR stream:
- EGL_BAD_MATCH is generated if the value of any one of the
EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV, or
EGL_STREAM_ENDPOINT_NV attributes is EGL_STREAM_LOCAL_NV and
any other of these attributes is neither EGL_STREAM_LOCAL_NV
nor EGL_DONT_CARE.
Add to section "3.10.2 Connecting an EGLStream to a consumer" in
EGL_KHR_stream:
Any function which connects a consumer to an EGLStream will fail and
generate an EGL_BAD_ACCESS error if the value of the EGLStream's
EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_PRODUCER_NV.
Add to section "3.10.3 Connecting an EGLStream to a producer" in
EGL_KHR_stream:
Any function which connects a producer to an EGLStream will fail and
generate an EGL_BAD_ACCESS error if the value of the EGLStream's
EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_CONSUMER_NV.
Add to "Table 3.10.4.4 EGLStream Attributes" in EGL_KHR_stream:
Attribute Read/Write Type Section
-------------------------- ---------- ------ ----------
EGL_STREAM_TYPE_NV io EGLint 3.10.4.x
EGL_STREAM_PROTOCOL_NV io EGLint 3.10.4.x+1
EGL_STREAM_ENDPOINT_NV io EGLint 3.10.4.x+2
Add to beginning of state list in section "3.10.4.3 EGL_STREAM_STATE_KHR
Attribute" in EGL_KHR_stream:
- EGL_STREAM_STATE_INITIALIZING_NV - The EGLStream object
represents one endpoint of a remote stream and has not yet
established communication with the opposite endpoint.
Replace the first entry in the state transition table in section
"3.10.4.3 EGL_STREAM_STATE_KHR Attribute" in EGL_KHR_stream:
-> EGL_STREAM_STATE_INITIALIZING_NV
A new remote EGLStream is created in this state.
EGL_STREAM_STATE_INITIALIZING_NV ->
EGL_STREAM_STATE_CREATED_KHR
Occurs when a remote EGLStream object establishes communication with
the opposite endpoint.
-> EGL_STREAM_STATE_CREATED_KHR
A new local EGLStream or EGLStream of undetermined type is created
in this state.
Add new subsections to the end of section "3.10.4 EGLStream Attributes"
in EGL_KHR_stream:
3.10.4.x EGL_STREAM_TYPE_NV Attribute
The EGL_STREAM_TYPE_NV attribute may be set when the stream
is created, and indicates the type of connection between the
EGLStream objects to which the consumer and producer are attached.
Legal values are EGL_DONT_CARE and EGL_STREAM_LOCAL_NV. The
default value is EGL_DONT_CARE.
A value of EGL_STREAM_LOCAL_NV indicates that this is a local
stream with both consumer and producer connected to a single
EGLStream object.
If EGL_DONT_CARE is initially specified, the value of the attribute
will automatically be changed to the appropriate value once both
consumer and producer are attached, depending on the functions used.
Other possible values for EGL_STREAM_TYPE_NV may be defined by
additional extensions to indicate a remote stream. Where used,
the EGL_STREAM_PROTOCOL_NV attribute must also be set to a
communication protocol compatible with the stream type.
3.10.4.x+1 EGL_STREAM_PROTOCOL_NV Attribute
The EGL_STREAM_PROTOCOL_NV attribute may be set when the stream is
created, and indicates the manner in which communication is
established between the EGLStream objects to which the consumer
and producer are attached. Legal values are EGL_DONT_CARE and
EGL_STREAM_LOCAL_NV. The default value is EGL_DONT_CARE.
A value of EGL_STREAM_LOCAL_NV indicates that this is a local
stream with both consumer and producer connected to a single
EGLStream object, so no communication protocol is required.
If EGL_DONT_CARE is initially specified, the value of the attribute
will automatically be changed to the appropriate value once both
consumer and producer are attached, depending on the functions used.
Other possible values for EGL_STREAM_PROTOCOL_NV may be defined by
additional extensions to indicate the communication protocol to be
used for a remote stream. Not all communication protocols are
compatible with all stream types, and vice versa. If incompatible
types and protocols are specified, an EGL_BAD_MATCH error will be
generated.
3.10.4.x+2 EGL_STREAM_ENDPOINT_NV Attribute
The EGL_STREAM_ENDPOINT_NV attribute may be set when the stream is
created, and indicates the endpoints which will be attached to the
EGLStream object. Legal values are EGL_DONT_CARE,
EGL_STREAM_LOCAL_NV, EGL_STREAM_CONSUMER_NV, and
EGL_STREAM_PRODUCER_NV. The default value is EGL_DONT_CARE.
A value of EGL_STREAM_LOCAL_NV indicates that this is a local
stream with both consumer and producer connected to a single
EGLStream object.
A value of EGL_STREAM_CONSUMER_NV indicates that the EGLStream
object represents the consumer side of a remote stream.
A value of EGL_STREAM_PRODUCER_NV indicates that the EGLStream
object represents the producer side of a remote stream.
If EGL_DONT_CARE is initially specified, the value of the attribute
will automatically be changed to the appropriate value once both
consumer and producer are attached, depending on the functions used.
If EGL_NV_stream_cross_object is present, in section 3.10.4.x above,
add EGL_STREAM_CROSS_OBJECT_NV to the list of legal values, and insert
A value of EGL_STREAM_CROSS_OBJECT_NV indicates that the stream
object represents one endpoint of a remote stream whose other
endpoint is obtained from the same EGLDisplay.
If EGL_NV_stream_cross_display is present, in section 3.10.4.x above,
add EGL_STREAM_CROSS_DISPLAY_NV to the list of legal values, and insert
A value of EGL_STREAM_CROSS_DISPLAY_NV indicates that the stream
object represents one endpoint of a remote stream whose other
endpoint may be obtained from a different EGLDisplay in the same
process.
If EGL_NV_stream_cross_process or EGL_KHR_stream_cross_process_fd are
present, in section "3.10.4.x" above, add EGL_STREAM_CROSS_PROCESS_NV
to the list of legal values, and insert
A value of EGL_STREAM_CROSS_PROCESS_NV indicates that the stream
object represents one endpoint of a remote stream whose other
endpoint may reside in a separate process.
If EGL_KHR_stream_cross_process_fd is present and
EGL_NV_stream_cross_process is not present, in section "3.10.4.x" above,
insert
EGL_STREAM_CROSS_PROCESS_NV is not accepted by eglCreateStreamKHR or
eglCreateStreamAttribKHR as value of EGL_STREAM_TYPE_NV. It will be
returned by eglQueryStreamKHR or eglQueryStreamAttribKHR for
EGL_STREAM_TYPE_NV if an EGLStream of undetermined type is created
and a file descriptor is subsequently obtained with
eglGetStreamFileDescriptorKHR.
If EGL_NV_stream_cross_partition is present, in section 3.10.4.x
above, add EGL_STREAM_CROSS_PARTITION_NV to the list of legal values,
and insert
A value of EGL_STREAM_CROSS_PARTITION_NV indicates that the stream
object represents one endpoint of a remote stream whose other
endpoint may reside in a separate virtual machine partition on
the same system. The partitions are not required to be using the
same operating systems, but must support compatible communication
protocols.
If EGL_NV_stream_cross_system is present, in section 3.10.4.x above,
add EGL_STREAM_CROSS_SYSTEM_NV to the list of legal values, and insert
A value of EGL_STREAM_CROSS_SYSTEM_NV indicates that the stream
object represents one endpoint of a remote stream whose other
endpoint may reside on an independent hardware system with no
directly sharable memory resources.
If EGL_KHR_stream_cross_process_fd is present, in section 3.10.4.x+1
above, add EGL_STREAM_PROTOCOL_FD_NV to the list of legal values, and
insert
A value of EGL_STREAM_PROTOCOL_FD_NV indicates that the stream is
a remote stream whose communication is established using a file
descriptor. The details of what this file descriptor represents
are implementation dependent.
EGL_STREAM_PROTOCOL_FD_NV is not accepted by eglCreateStreamKHR or
eglCreateStreamAttribKHR as value of EGL_STREAM_PROTOCOL_NV. It will
be returned by eglQueryStreamKHR or eglQueryStreamAttribKHR for
EGL_STREAM_PROTOCOL_NV if an EGLStream of undetermined type is
created and a file descriptor is subsequently obtained with
eglGetStreamFileDescriptorKHR.
Add a new subsection to the end of "3.10.5 EGLStream operation":
3.10.5.x Remote streams
An EGLStream object may be created as the endpoint of a remote
stream by specifying EGL_STREAM_PRODUCER_NV or
EGL_STREAM_CONSUMER_NV as the value for its EGL_STREAM_ENDPOINT_NV
attribute. Valid and compatible EGL_STREAM_TYPE_NV and
EGL_STREAM_PROTOCOL_NV values other than EGL_DONT_CARE or
EGL_STREAM_LOCAL_NV must also be specified.
If a value for EGL_STREAM_ENDPOINT_NV is not specified, the stream
is assumed to be local, but other extensions (see for example
EGL_KHR_stream_cross_process_fd) may allow it to be converted to a
remote stream before the producer has been attached.
When an EGLStream object is created as remote, any unspecified
attribute will be initially set to a value of EGL_DONT_CARE. Pairs
of EGLStream objects representing opposite endpoints of a stream are
not required to specify the same attribute lists, but their
attributes must be compatible. When communication is established
between the endpoints, they will exchange attribute settings:
- If both endpoints have a value of EGL_DONT_CARE for an
attribute, the default value will be assigned.
- If one endpoint has a value of EGL_DONT_CARE for an attribute,
it will be set to the other endpoint's value.
- If neither endpoint has a value of EGL_DONT_CARE for an
attribute, their values must agree. In most cases, this means
the values must be identical, but this may not be true for all
attributes. In particular, one endpoint must specify an
EGL_STREAM_ENDPOINT_NV value of EGL_STREAM_CONSUMER_NV, and
the other must specify a value of EGL_STREAM_PRODUCER_NV.
If the values for any attribute are not compatible, the stream will
transition to the EGL_STREAM_STATE_DISCONNECTED_KHR state.
Additionally, if the two EGLStream objects representing a remote
stream are created for EGLDisplays which cannot share resources,
the stream will transition to the EGL_STREAM_STATE_DISCONNECTED_KHR
state.
When using remote streams, there may be latency in communicating
state changes between the EGLStream objects representing the two
endpoints. For instance, when a new frame is inserted into the
stream by the producer, the consumer endpoint may not immediately
transition to EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR. Similarly,
when a frame is acquired by the consumer, the producer endpoint may
not immediately transition to
EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR. Applications should rely
only on the state of the endpoint on which they are operating to
determine the expected results of that operation.
In section "3.10.1.1 Duplicating an EGLStream from a file descriptor" of
the EGL_KHR_stream_cross_process_fd extension, add to the failure list
for eglGetStreamFileDescriptorKHR
- EGL_BAD_ACCESS is generated if the EGL_STREAM_TYPE_NV
attribute of <stream> is anything other than EGL_DONT_CARE or
EGL_STREAM_CROSS_PROCESS_NV.
- EGL_BAD_ACCESS is generated if the EGL_STREAM_PROTOCOL_NV
attribute of <stream> is anything other than EGL_DONT_CARE or
EGL_STREAM_PROTOCOL_FD_NV.
- EGL_BAD_ACCESS is generated if the EGL_STREAM_ENDPOINT_NV
attribute of <stream> is EGL_STREAM_LOCAL_NV.
Issues
None
Revision History
#5 (July 06, 2016) Miguel A. Vico
- Minor corrections to EGL_KHR_stream_cross_process_fd
interactions.
#4 (June 01, 2016) Daniel Kartch
- Removed sync object definition. It will be provided by a
separate extension.
- Folded in cross-partition extension.
- Added types for cross-object, cross-display, and
cross-system streams.
- General cleanup in preparation for publication.
#3 (September 16, 2015) Zander Clucas
- Removed dependence of the CROSS_PROCESS type on the
EGL_NV_cross_process_fd extension
- Added CROSS_PROCESS to the list of STREAM_TYPE legal values
- Added CROSS_PROCESS requirement for cross_process_fd protocol
#2 (December 11, 2014) Daniel Kartch
- Rewrote as NV draft for earlier release.
- Added section for creation of sync object.
- Reserved enum values.
#1 (October 10, 2014) Daniel Kartch
- Initial EXT draft
