Name
KHR_stream_consumer_gltexture
Name Strings
EGL_KHR_stream_consumer_gltexture
Contributors
Acorn Pooley
Jamie Gennis
Marcus Lorentzon
Contacts
Acorn Pooley, NVIDIA (apooley 'at' nvidia.com)
Notice
Copyright (c) 2011-2013 The Khronos Group Inc. Copyright terms at
http://www.khronos.org/registry/speccopyright.html
Status
Complete.
Approved by the Khronos Board of Promoters on December 2, 2011.
Version
Version 11, June 18, 2012
Number
EGL Extension #33
Dependencies
Requires EGL 1.2.
Requires OpenGL ES 1.1 or OpenGL ES 2.0.
Requires the EGL_KHR_stream extension.
Requires the GL_NV_EGL_stream_consumer_external extension.
Overview
This extension allows an OpenGL(ES) texture to be connected to an
EGLStream as its consumer. Image frames from the EGLStream can be
'latched' into the texture as the contents of the texture. This
is equivalent to copying the image into the texture, but on most
implementations a copy is not needed so this is faster.
New Procedures and Functions
EGLBoolean eglStreamConsumerGLTextureExternalKHR(
EGLDisplay dpy,
EGLStreamKHR stream)
EGLBoolean eglStreamConsumerAcquireKHR(
EGLDisplay dpy,
EGLStreamKHR stream);
EGLBoolean eglStreamConsumerReleaseKHR(
EGLDisplay dpy,
EGLStreamKHR stream);
New Tokens
Accepted as an attribute in the <attrib_list> parameter of
eglCreateStreamKHR and as the <attribute> parameter of
eglStreamAttribKHR and eglQueryStreamKHR
EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR 0x321E
Replace section "3.10.2.1 No way to connect consumer to EGLStream" in
the EGL_KHR_stream extension with this:
3.10.2.1 GL Texture External consumer
Call
EGLBoolean eglStreamConsumerGLTextureExternalKHR(
EGLDisplay dpy,
EGLStreamKHR stream)
to connect the texture object currently bound to the active
texture unit's GL_TEXTURE_EXTERNAL_OES texture target in the
OpenGL or OpenGL ES context current to the calling thread as the
consumer of <stream>.
(Note: Before this can succeed a GL_TEXTURE_EXTERNAL_OES texture
must be bound to the active texture unit of the GL context current
to the calling thread. To create a GL_TEXTURE_EXTERNAL_OES
texture and bind it to the current context, call glBindTexture()
with <target> set to GL_TEXTURE_EXTERNAL_OES and <texture> set to
the name of the GL_TEXTURE_EXTERNAL_OES (which may or may not have
previously been created). This is described in the
GL_NV_EGL_stream_consumer_external extension.)
On failure EGL_FALSE is returned and an error is generated.
- EGL_BAD_STATE_KHR is generated if <stream> is not in state
EGL_STREAM_STATE_CREATED_KHR.
- EGL_BAD_ACCESS is generated if there is no GL context
current to the calling thread.
- EGL_BAD_ACCESS is generated unless a nonzero texture object
name is bound to the GL_TEXTURE_EXTERNAL_OES texture target
of the GL context current to the calling thread.
- EGL_BAD_STREAM_KHR is generated if <stream> is not a valid
EGLStreamKHR created for <dpy>.
- EGL_BAD_DISPLAY is generated if <dpy> is not a valid,
initialized EGLDisplay.
On success the texture is connected to the <stream>, <stream> is
placed in the EGL_STREAM_STATE_CONNECTING_KHR state, and EGL_TRUE is
returned.
If the texture is later deleted, connected to a different
EGLStream, or connected to an EGLImage, then <stream> will be
placed into the EGL_STREAM_STATE_DISCONNECTED_KHR state.
If the <stream> is later destroyed then the texture will be
"incomplete" until it is connected to a new EGLStream, connected
to a new EGLImage, or deleted.
Call
EGLBoolean eglStreamConsumerAcquireKHR(
EGLDisplay dpy,
EGLStreamKHR stream);
to "latch" the most recent image frame from <stream> into the
texture that is the consumer of <stream>. The GLES context
containing the texture must be bound to the current thread. If
the GLES texture is also used in shared contexts current to other
threads then the texture must be re-bound in those contexts to
guarantee the new texture is used.
eglStreamConsumerAcquireKHR will block until either the timeout
specified by EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR expires, or the
value of EGL_BAD_STATE_KHR is neither EGL_STREAM_STATE_EMPTY_KHR nor
EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR (whichever comes first).
Blocking effectively waits until a new image frame (that has never
been consumed) is available in the EGLStream. By default the
timeout is zero and the function does not block.
eglStreamConsumerAcquireKHR returns EGL_TRUE if an image frame was
successfully latched into the texture object.
If the producer has not inserted any new image frames since the
last call to eglStreamConsumerAcquireKHR then
eglStreamConsumerAcquireKHR will "latch" the same image frame it
latched last time eglStreamConsumerAcquireKHR was called. If the
producer has inserted one new image frame since the last call to
eglStreamConsumerAcquireKHR then eglStreamConsumerAcquireKHR will
"latch" the newly inserted image frame. If the producer has
inserted more than one new image frame since the last call to
eglStreamConsumerAcquireKHR then all but the most recently
inserted image frames are discarded and the
eglStreamConsumerAcquireKHR will "latch" the most recently
inserted image frame.
The application can use the value of EGL_CONSUMER_FRAME_KHR to
identify which image frame was actually latched.
On failure the texture becomes "incomplete", eglStreamConsumerAcquireKHR
returns EGL_FALSE, and an error is generated.
- EGL_BAD_STATE_KHR is generated if <stream> is not in state
EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR or
EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR.
- EGL_BAD_ACCESS is generated if there is no GL context
current to the calling thread, or if the GL context current
to the calling thread does not contain a texture that is
connected as the consumer of the EGLStream.
- EGL_BAD_STREAM_KHR is generated if <stream> is not a valid
EGLStream created for <dpy>.
- EGL_BAD_DISPLAY is generated if <dpy> is not a valid,
initialized EGLDisplay.
After using the texture call
EGLBoolean eglStreamConsumerReleaseKHR(
EGLDisplay dpy,
EGLStreamKHR stream);
to release the image frame back to the stream.
eglStreamConsumerReleaseKHR() will prevent the EGLStream and
producer from re-using and/or modifying the image frame until all
preceding GL commands that use the image frame as a texture have
completed. If eglStreamConsumerAcquireKHR() is called twice on the
same EGLStream without an intervening call to
eglStreamConsumerReleaseKHR() then eglStreamConsumerReleaseKHR() is
implicitly called at the start of eglStreamConsumerAcquireKHR().
After successfully calling eglStreamConsumerReleaseKHR the texture
becomes "incomplete".
If eglStreamConsumerReleaseKHR is called twice without a successful
intervening call to eglStreamConsumerAcquireKHR, or called with no
previous call to eglStreamConsumerAcquireKHR, then the call does
nothing and the texture remains in "incomplete" state. This is
not an error.
If eglStreamConsumerReleaseKHR fails EGL_FALSE is returned and an error is
generated.
- EGL_BAD_STATE_KHR is generated if <stream> is not in state
EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR or
EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR.
- EGL_BAD_ACCESS is generated if there is no GL context
current to the calling thread, or if the GL context current
to the calling thread does not contain the texture to which
the EGLStream is connected.
- EGL_BAD_STREAM_KHR is generated if <stream> is not a valid
EGLStream created for <dpy>.
- EGL_BAD_DISPLAY is generated if <dpy> is not a valid,
initialized EGLDisplay.
The application should estimate the time that will elapse from the
time a new frame becomes available (i.e. the state becomes
EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR) and the time the frame
is presented to the user. The application should set this as the
value of the EGL_CONSUMER_LATENCY_USEC attribute by calling
eglStreamAttribKHR(). The value will depend on the complexity of
the scene being rendered and the platform that the app is running
on. It may be difficult to estimate except by experimentation on
a specific platform. The default value is implementation
dependent and may be a good enough estimate for some situations.
If the estimate changes over time the application may modify the
value of EGL_CONSUMER_LATENCY_USEC.
If the EGLStream is deleted while an image frame is acquired (i.e.
after calling eglStreamConsumerAcquireKHR and before calling
eglStreamConsumerReleaseKHR) then the EGLStream resources will not
be freed until the acquired image frame is released. However it
is an error to call eglStreamConsumerReleaseKHR after deleting the
EGLStream because <stream> is no longer a valid handle. In this
situation the image can be released (and the EGLStream resources
freed) by doing any one of
- deleting the GL_TEXTURE_EXTERNAL (call glDeleteTextures)
- connecting the GL_TEXTURE_EXTERNAL to another EGLStream
(call eglStreamConsumerGLTextureExternalKHR)
- connecting the GL_TEXTURE_EXTERNAL to an EGLImage (if the
GL_OES_EGL_image_external extension is supported, call
glEGLImageTargetTexture2DOES)
Add a new subsection 3.10.4.6 at the end of section "3.10.4 EGLStream
Attributes" in the EGL_KHR_stream extension spec:
3.10.4.6 EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR Attribute
This attribute is read/write. The default value is 0. It
indicates the maximum amount of time (in microseconds) that
eglStreamConsumerAcquireKHR should block. If 0 (the default) it
will not block at all. If negative it will block indefinitely.
Issues
1. How to notify the app when a new image is available
- callback?
- pro: easy to use
- con: introduces extra threads into EGL which does not define such
behavior now - would have to define a lot of semantics (e.g. what
can you call from the callback?)
- EGL_KHR_reusable_sync signaled?
- this is how EGL_KHR_stream_consumer_endpoint does it
- pro: simpler to specify
- pro: easy to use if that is all you are waiting for
- con: difficult to wait on this AND other events simultaneously?
- blocking call to eglStreamConsumerAcquireKHR?
RESOLVED: Use the EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR to make
eglStreamConsumerAcquireKHR blocking if desired. Additional
mechanisms can be added as layered extensions.
2. What to call this extension?
EGL_NV_stream_consumer_gltexture
EGL_EXT_stream_consumer_gltexture
EGL_KHR_stream_consumer_gltexture
EGL_KHR_stream_consumer_gltexture_external
RESOLVED: EGL_KHR_stream_consumer_gltexture
3. Should it be possible to connect an EGLStream to this consumer
(texture), and then later reconnect the same stream to a different
consumer?
RESOLVED: no
There may be reasons to allow this later, but for the time being
there is no use for this. Adding this functionality can be
considered in the future with a layered extension.
4. Do we need both this extension and
GL_NV_EGL_stream_consumer_external? Should we just have one
extension that takes the place of both? If so should it be an
EGL or a GL extension?
UNRESOLVED
SUGGESTION: need both
See issue 1 in GL_NV_EGL_stream_consumer_external.txt
5. What happens if the EGLStream is deleted while the consumer
has an image acquired?
This case is a problem because after the EGLStream is deleted
the EGLStreamKHR handle is no longer valid, which means
eglStreamConsumerReleaseKHR cannot be called (because it would
return EGL_BAD_STREAM).
Possible resolutions:
A) Do not allow the EGLStream to be deleted while an image is
acquired.
B) Allow the EGLStream to be deleted. Allow the EGLStreamKHR
handle to be used in a call to eglStreamConsumerReleaseKHR()
after it has been deleted.
C) Allow the EGLStream to be deleted. It is an error to call
eglStreamConsumerReleaseKHR() after the stream is deleted. To
release the image the app must
- delete the GL_TEXTURE_EXTERNAL texture object
or - connect another EGLStream to the GL_TEXTURE_EXTERNAL
texture object
or - connect an EGLImage to the GL_TEXTURE_EXTERNAL
texture object
D) Make the call to EGLStream implicitly call
eglStreamConsumerReleaseKHR if an image is acquired. This
requires the GL context is current to the thread that deletes
the EGLStream.
E) Make the call to EGLStream implicitly call
eglStreamConsumerReleaseKHR if an image is acquired, and state
that this has to work even if the GL context is current to a
different thread or not current to any thread.
Pros/cons:
- B violates EGL object handle lifetime policies
- E is hard/impossible to implement on some systems
- D makes deletion fail for complicated reasons
- A makes deletion fail for less complicated reasons
RESOLVED: option C
Revision History
#11 (June 18. 2012) Acorn Pooley
- Replace EGLStream with EGLStreamKHR in function prototypes.
#10 (October 12, 2011) Acorn Pooley
- Fix confusing error in eglStreamConsumerAcquireKHR description.
#9 (October 4, 2011) Acorn Pooley
- Convert from an NV extension to a KHR extension
#8 (September 30, 2011) Acorn Pooley
- Add issue 5 and clarify EGLStream deletion while image is
acquired.
#7 (September 27, 2011) Acorn Pooley
- Assign enum values (bug 8064)
#6 (Aug 3, 2011) Acorn Pooley
- rename GL_OES_EGL_stream_external to
GL_NV_EGL_stream_consumer_external
#5 (Aug 2, 2011) Acorn Pooley
- Add dependency on GL_OES_EGL_stream_external
#4 (Aug 2, 2011) Acorn Pooley
- Fix spelling and grammar
#3 (July 6, 2011) Acorn Pooley
- Rename EGL_KHR_image_stream to EGL_KHR_stream
#2 (June 29, 2011) Acorn Pooley
- change how texture is connected to stream to match
EGL_KHR_stream spec.
- Add EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_NV
#1 (April 20, 2011) Acorn Pooley
- initial draft
# vim:ai:ts=4:sts=4:expandtab:textwidth=70
