--- /dev/null
+Android Hardware OpenGLES emulation design overview
+===================================================
+
+Introduction:
+-------------
+
+Hardware GLES emulation in the Android platform is implemented with a mix
+of components, which are:
+
+ - Several host "translator" libraries. They implement the EGL, GLES 1.1 and
+ GLES 2.0 ABIs defined by Khronos, and translate the corresponding function
+ calls into calls to the appropriate desktop APIs, i.e.:
+
+ - GLX (Linux), AGL (OS X) or WGL (Windows) for EGL
+ - desktop GL 2.0 for GLES 1.1 and GLES 2.0
+
+ _________ __________ __________
+ | | | | | |
+ |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST
+ | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR
+ |_________| |__________| |__________| LIBRARIES
+ | | |
+ - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
+ | | |
+ ____v____ ____v_____ _____v____ HOST
+ | | | | | | SYSTEM
+ | GLX | | GL 2.0 | | GL 2.0 | LIBRARIES
+ |_________| |__________| |__________|
+
+
+
+ - Several system libraries inside the emulated guest system that implement
+ the same EGL / GLES 1.1 and GLES 2.0 ABIs.
+
+ They collect the sequence of EGL/GLES function calls and translate then
+ into a custom wire protocol stream that is sent to the emulator program
+ through a high-speed communication channel called a "QEMU Pipe".
+
+ For now, all you need to know is that the pipe is implemented with a
+ custom kernel driver, and provides for _very_ fast bandwidth. All read()
+ and writes() from/to the pipes are essentially instantaneous from the
+ guest's point of view.
+
+
+ _________ __________ __________
+ | | | | | |
+ |EMULATION| |EMULATION | |EMULATION | GUEST
+ | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM
+ |_________| |__________| |__________| LIBRARIES
+ | | |
+ - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
+ | | |
+ ____v____________________v____________________v____ GUEST
+ | | KERNEL
+ | QEMU PIPE |
+ |___________________________________________________|
+ |
+ - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - -
+ |
+ v
+ EMULATOR
+
+ - Specific code inside the emulator program that is capable of transmitting
+ the wire protocol stream to a special rendering library or process (called
+ the "renderer" here), which understands the format.
+
+ |
+ | PROTOCOL BYTE STREAM
+ _____v_____
+ | |
+ | EMULATOR |
+ |___________|
+ |
+ | UNMODIFIED PROTOCOL BYTE STREAM
+ _____v_____
+ | |
+ | RENDERER |
+ |___________|
+
+
+ - The renderer decodes the EGL/GLES commands from the wire
+ protocol stream, and dispatches them to the translator libraries
+ appropriately.
+
+ |
+ | PROTOCOL BYTE STREAM
+ _____v_____
+ | |
+ | RENDERER |
+ |___________|
+ | | |
+ +-----------------+ | +-----------------+
+ | | |
+ ____v____ ___v______ ____v_____
+ | | | | | | HOST
+ |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST
+ | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR
+ |_________| |__________| |__________| LIBRARIES
+
+
+
+ - In reality, the protocol stream flows in both directions, even though most
+ of the commands result in data going from the guest to the host. A complete
+ picture of the emulation would thus be:
+
+
+
+
+
+ _________ __________ __________
+ | | | | | |
+ |EMULATION| |EMULATION | |EMULATION | GUEST
+ | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM
+ |_________| |__________| |__________| LIBRARIES
+ ^ ^ ^
+ | | |
+ - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
+ | | |
+ ____v____________________v____________________v____ GUEST
+ | | KERNEL
+ | QEMU PIPE |
+ |___________________________________________________|
+ ^
+ |
+ - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - -
+ |
+ | PROTOCOL BYTE STREAM
+ _____v_____
+ | |
+ | EMULATOR |
+ |___________|
+ ^
+ | UNMODIFIED PROTOCOL BYTE STREAM
+ _____v_____
+ | |
+ | RENDERER |
+ |___________|
+ ^ ^ ^
+ | | |
+ +-----------------+ | +-----------------+
+ | | |
+ ____v____ ___v______ ____v_____
+ | | | | | |
+ |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST
+ | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR
+ |_________| |__________| |__________| LIBRARIES
+ ^ ^ ^
+ | | |
+ - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
+ | | |
+ ____v____ ____v_____ _____v____ HOST
+ | | | | | | SYSTEM
+ | GLX | | GL 2.0 | | GL 2.0 | LIBRARIES
+ |_________| |__________| |__________|
+
+ (NOTE: 'GLX' is for Linux only, replace 'AGL' on OS X, and 'WGL' on Windows).
+
+
+Note that, in the above graphics, only the host system libraries at the bottom
+are _not_ provided by Android.
+
+
+Design Requirements:
+--------------------
+
+The above design comes from several important requirements that were decided
+early in the project:
+
+1 - The ability to run the renderer in a separate process from the emulator
+ itself is important.
+
+ For various practical reasons, we plan to completely separate the core QEMU
+ emulation from the UI window by using two distinct processes. As such, the
+ renderer will be implemented as a library inside the UI program, but will
+ need to receive protocol bytes from the QEMU process.
+
+ The communication channel will be either a fast Unix socket or a Win32
+ named pipe between these two. A shared memory segment with appropriate
+ synchronization primitives might also be used if performance becomes
+ an issue.
+
+ This explains why the emulator doesn't alter or even try to parse the
+ protocol byte stream. It only acts as a dumb proxy between the guest
+ system and the renderer. This also avoids adding lots of GLES-specific
+ code inside the QEMU code base which is terribly complex.
+
+2 - The ability to use vendor-specific desktop EGL/GLES libraries is
+ important.
+
+ GPU vendors like NVidia, AMD or ARM all provide host versions of the
+ EGL/GLES libraries that emulate their respectivie embedded graphics
+ chipset.
+
+ The renderer library can be configured to use these instead of the
+ translator libraries provided with this project. This can be useful to
+ more accurately emulate the behaviour of specific devices.
+
+ Moreover, these vendor libraries typically expose vendor-specific
+ extensions that are not provided by the translator libraries. We cannot
+ expose them without modifying our code, but it's important to be able
+ to do so without too much pain.
+
+
+Code organization:
+------------------
+
+All source code for the components above is spread over multiple directories
+in the Android source trees:
+
+ - The emulator sources are under $ANDROID/external/qemu, which we'll
+ call $QEMU in the rest of this document.
+
+ - The guest libraries are under
+ $ANDROID/device/generic/goldfish/opengl, which we'll call $EMUGL_GUEST
+
+ - The host renderer and translator libraries are under
+ $QEMU/distrib/android-emugl, which we'll call $EMUGL_HOST
+
+ - The QEMU Pipe kernel driver is under $KERNEL/drivers/misc/qemupipe (3.4)
+ or $KERNEL/drivers/platform/goldfish/goldfish_pipe.c (3.10)
+
+Where $ANDROID is the top of the open-source Android source tree, and
+$KERNEL is the top of the qemu-specific kernel source tree (using one
+of the android-goldfish-xxxx branches here).
+
+The emulator sources related to this projects are:
+
+ $QEMU/hw/android/goldfish/pipe.c -> implements QEMU pipe virtual hardware.
+ $QEMU/android/opengles.c -> implements GLES initialization.
+ $QEMU/android/hw-pipe-net.c -> implements the communication channel
+ between the QEMU Pipe and the renderer library
+
+The other sources are:
+
+ $EMUGL_GUEST/system -> system libraries
+ $EMUGL_GUEST/shared -> guest copy of shared libraries
+ $EMUGL_GUEST/tests -> various test programs
+
+ $EMUGL_HOST/host -> host libraries (translator + renderer)
+ $EMUGL_HOST/shared -> host copy of shared libraries
+
+The reason the shared libraries aren't actually shared is historical: at one
+point both guest and host code lived in the same place. That turned out to be
+impractical with the way the Android SDK is branched, and didn't support the
+requirement that a single emulator binary be able to run several releases
+of Android.
+
+
+Translator libraries:
+---------------------
+
+There are three translator host libraries provided by this project:
+
+ libEGL_translator -> EGL 1.2 translation
+ libGLES_CM_translator -> GLES 1.1 translation
+ libGLES_V2_translator -> GLES 2.0 translation
+
+The full name of the library will depend on the host system.
+For simplicity, only the library name suffix will change (i.e. the
+'lib' prefix is not dropped on Windows), i.e.:
+
+ libEGL_translator.so -> for Linux
+ libEGL_translator.dylib -> for OS X
+ libEGL_translator.dll -> for Windows
+
+The source code for these libraries is located under the following
+path in the Android source tree:
+
+ $EMUGL_HOST/host/libs/Translator/EGL
+ $EMUGL_HOST/host/libs/Translator/GLES_CM
+ $EMUGL_HOST/host/libs/Translator/GLES_V2
+
+The translator libraries also use a common routines defined under:
+
+ $EMUGL_HOST/host/libs/Translator/GLcommon
+
+
+Wire Protocol Overiew:
+----------------------
+
+The "wire protocol" is implemented as follows:
+
+ - EGL/GLES function calls are described through several "specification"
+ files, which describes the types, function signatures and various
+ attributes for each one of them.
+
+ - These files are read by a tool called "emugen" which generates C
+ source files and headers based on the specification. These correspond
+ to both encoding, decoding and "wrappers" (more on this later).
+
+ - System "encoder" static libraries are built using some of these generated
+ files. They contain code that can serialize EGL/GLES calls into simple
+ byte messages and send it through a generic "IOStream" object.
+
+ - Host "decoder" static libraries are also built using some of these
+ generated files. Their code retrieves byte messages from an "IOStream"
+ object, and translates them into function callbacks.
+
+IOStream abstraction:
+- - - - - - - - - - -
+
+The "IOStream" is a very simple abstract class used to send byte messages
+both in the guest and host. It is defined through a shared header under
+$EMUGL/host/include/libOpenglRender/IOStream.h
+
+Note that despite the path, this header is included by *both* host and guest
+source code. The main idea around IOStream's design is that to send a message,
+one does the following:
+
+ 1/ call stream->allocBuffer(size), which returns the address of a
+ memory buffer of at least 'size' bytes.
+
+ 2/ write the content of the serialized command (usually a header + some
+ payload) directly into the buffer
+
+ 3/ call stream->commitBuffer() to send it.
+
+Alternatively, one can also pack several commands into a single buffer with
+stream->alloc() and stream->flush(), as in:
+
+ 1/ buf1 = stream->alloc(size1)
+ 2/ write first command bytes into buf1
+ 3/ buf2 = stream->alloc(size2)
+ 4/ write second command bytes into buf2
+ 5/ stream->flush()
+
+Finally, there are also explict read/write methods like stream->readFully()
+or stream->writeFully() which can be used when you don't want an intermediate
+buffer. This is used in certain cases by the implementation, e.g. to avoid
+an intermediate memory copy when sending texture data from the guest to the
+host.
+
+The host IOStream implementations are under $EMUGL/shared/OpenglCodecCommon/,
+see in particular:
+
+ $EMUGL_HOST/shared/OpenglCodecCommon/TcpStream.cpp
+ -> using local TCP sockets
+ $EMUGL_HOST/shared/OpenglCodecCommon/UnixStream.cpp
+ -> using Unix sockets
+ $EMUGL_HOST/shared/OpenglCodecCommon/Win32PipeStream.cpp
+ -> using Win32 named pipes
+
+The guest IOStream implementation uses the TcpStream.cpp above, as well as
+an alternative QEMU-specific source:
+
+ $EMUGL_GUEST/system/OpenglSystemCommon/QemuPipeStream.cpp
+ -> uses QEMU pipe from the guest
+
+The QEMU Pipe implementation is _significantly_ faster (about 20x) due to
+several reasons:
+
+ - all succesful read() and write() operations through it are instantaneous
+ from the guest's point of view.
+
+ - all buffer/memory copies are performed directly by the emulator, and thus
+ much faster than performing the same thing inside the kernel with emulated
+ ARM instructions.
+
+ - it doesn't need to go through a kernel TCP/IP stack that will wrap the
+ data into TCP/IP/MAC packets, send them to an emulated ethernet device,
+ which is itself connected to an internal firewall implementation that
+ will unwrap the packets, re-assemble them, then send them through BSD
+ sockets to the host kernel.
+
+However, would it be necessary, you could write a guest IOStream implementation
+that uses a different transport. If you do, please look at
+$EMUGL_GUEST/system/OpenglCodecCommon/HostConnection.cpp which contains the
+code used to connect the guest to the host, on a per-thread basis.
+
+
+Source code auto-generation:
+- - - - - - - - - - - - - -
+
+The 'emugen' tool is located under $EMUGL_HOST/host/tools/emugen. There is a
+README file that explains how it works.
+
+You can also look at the following specifications files:
+
+For GLES 1.1:
+ $EMUGL_HOST/host/GLESv1_dec/gl.types
+ $EMUGL_HOST/host/GLESv1_dec/gl.in
+ $EMUGL_HOST/host/GLESv1_dec/gl.attrib
+
+For GLES 2.0:
+ $EMUGL_HOST/host/GLESv2_dec/gl2.types
+ $EMUGL_HOST/host/GLESv2_dec/gl2.in
+ $EMUGL_HOST/host/GLESv2_dec/gl2.attrib
+
+For EGL:
+ $EMUGL_HOST/host/renderControl_dec/renderControl.types
+ $EMUGL_HOST/host/renderControl_dec/renderControl.in
+ $EMUGL_HOST/host/renderControl_dec/renderControl.attrib
+
+Note that the EGL specification files are under a directory named
+"renderControl_dec" and have filenames that begin with "renderControl"
+
+This is mainly for historic reasons now, but is also related to the fact that
+this part of the wire protocol contains support functions/calls/specifications
+that are not part of the EGL specification itself, but add a few features
+required to make everything works. For example, they have calls related to
+the "gralloc" system library module used to manage graphics surfaces at a
+lower level than EGL.
+
+Generally speaking, guest encoder sources are located under directories
+named $EMUGL_GUEST/system/<name>_enc/, while the corresponding host decoder
+sources will be under $EMUGL_HOST/host/libs/<name>_dec/
+
+However, all these sources use the same spec files located under the
+decoding directories.
+
+The encoder files are built from emugen and spec files located in $EMUGL_HOST
+and copied to the encoder directories in $EMUGL_GUEST by the gen-encoder.sh
+script. They are checked in, so that a given version of Android supports a
+specific version of the protocol, even if newer versions of the renderer (and
+future Android versions) support a newer protocol version. This step needs to
+be done manually when the protocol changes; these changes also need to be
+accompanied by changes in the renderer to handle the old version of the
+protocol.
+
+
+System libraries:
+-----------------
+
+Meta EGL/GLES system libraries, and egl.cfg:
+- - - - - - - - - - - - - - - - - - - - - -
+
+It is important to understand that the emulation-specific EGL/GLES libraries
+are not directly linked by applications at runtime. Instead, the system
+provides a set of "meta" EGL/GLES libraries that will load the appropriate
+hardware-specific libraries on first use.
+
+More specifically, the system libEGL.so contains a "loader" which will try
+to load:
+
+ - hardware-specific EGL/GLES libraries
+ - the software-based rendering libraries (called "libagl")
+
+The system libEGL.so is also capable of merging the EGL configs of both the
+hardware and software libraries transparently to the application. The system
+libGLESv1_CM.so and libGLESv2.so, work with it to ensure that the thread's
+current context will be linked to either the hardware or software libraries
+depending on the config selected.
+
+For the record, the loader's source code in under
+frameworks/base/opengl/libs/EGL/Loader.cpp. It depends on a file named
+/system/lib/egl/egl.cfg which must contain two lines that look like:
+
+ 0 1 <name>
+ 0 0 android
+
+The first number in each line is a display number, and must be 0 since the
+system's EGL/GLES libraries don't support anything else.
+
+The second number must be 1 to indicate hardware libraries, and 0 to indicate
+a software one. The line corresponding to the hardware library, if any, must
+always appear before the one for the software library.
+
+The third field is a name corresponding to a shared library suffix. It really
+means that the corresponding libraries will be named libEGL_<name>.so,
+libGLESv1_CM_<name>.so and libGLESv2_<name>.so. Moreover these libraries must
+be placed under /system/lib/egl/
+
+The name "android" is reserved for the system software renderer.
+
+The egl.cfg that comes with this project uses the name "emulation" for the
+hardware libraries. This means that it provides an egl.cfg file that contains
+the following lines:
+
+ 0 1 emulation
+ 0 0 android
+
+See $EMUGL_GUEST/system/egl/egl.cfg and more generally the following build
+files:
+
+ $EMUGL_GUEST/system/egl/Android.mk
+ $EMUGL_GUEST/system/GLESv1/Android.mk
+ $EMUGL_GUEST/system/GLESv2/Android.mk
+
+to see how the libraries are named and placed under /system/lib/egl/ by the
+build system.
+
+
+Emulation libraries:
+- - - - - - - - - - -
+
+The emulator-specific libraries are under the following:
+
+ $EMUGL_GUEST/system/egl/
+ $EMUGL_GUEST/system/GLESv1/
+ $EMUGL_GUEST/system/GLESv2/
+
+The code for GLESv1 and GLESv2 is pretty small, since it mostly link against
+the static encoding libraries.
+
+The code for EGL is a bit more complex, because it needs to deal with
+extensions dynamically. I.e. if an extension is not available on the host
+it shouldn't be exposed by the library at runtime. So the EGL code queries
+the host for the list of available extensions in order to return them to
+clients. Similarly, it must query the list of valid EGLConfigs for the
+current host system.
+
+
+"gralloc" module implementation:
+- - - - - - - - - - - - - - - - -
+
+In addition to EGL/GLES libraries, the Android system requires a
+hardware-specific library to manage graphics surfaces at a level lower than
+EGL. This library must be what is called in Android land as a "HAL module".
+
+A "HAL module" must provide interfaces defined by Android's HAL
+(Hardware Abstraction Library). These interface definitions can be found
+under $ANDROID/hardware/libhardware/include/
+
+Of all possible HAL modules, the "gralloc" one is used by the system's
+SurfaceFlinger to allocate framebuffers and other graphics memory regions,
+as well as eventually lock/unlock/swap them when needed.
+
+The code under $EMUGL/system/gralloc/ implements the module required by the
+GLES emulation project. It's not very long, but there are a few things to
+notice here:
+
+- first, it will probe the guest system to determine if the emulator that
+ is running the virtual device really supports GPU emulation. In certain
+ circumstances this may not be possible.
+
+ If this is the case, then the module will redirect all calls to the
+ "default" gralloc module that is normally used by the system when
+ software-only rendering is enabled.
+
+ The probing happens in the function "fallback_init" which gets called
+ when the module is first opened. This initializes the 'sFallback' variable
+ to a pointer to the default gralloc module when required.
+
+- second, this module is used by SurfaceFlinger to display "software surfaces",
+ i.e. those that are backed by system memory pixel buffers, and written to
+ directly through the Skia graphics library (i.e. the non-accelerated ones).
+
+ the default module simply copies the pixel data from the surface to the
+ virtual framebuffer i/o memory, but this project's gralloc module sends it
+ to the renderer through the QEMU Pipe instead.
+
+ It turns out that this results in _faster_ rendering/frame-rates overall,
+ because memory copies inside the guest are slow, while QEMU pipe transfers
+ are done directly in the emulator.
+
+
+Host Renderer:
+--------------
+
+The host renderer library is located under
+$EMUGL_HOST/host/libs/libOpenglRender, and it provides an interface described
+by the headers under $EMUGL_HOST/host/libs/libOpenglRender/render_api.h
+(e.g. for use by the emulator).
+
+In a nutshell, the rendering library is responsible for the following:
+
+ - Providing a virtual off-screen video surface where everything will get
+ rendered at runtime. Its dimensions are fixed by the call to
+ initOpenglRender() that must happen just after the library is
+ initialized.
+
+ - Provide a way to display the virtual video surface on a host application's
+ UI. This is done by calling createOpenGLSubWindow() which takes as argument
+ the window ID or handle of a parent window, some display dimensions and
+ a rotation angle. This allows the surface to be scaled/rotated when it is
+ displayed, even if the dimensions of the video surface do not change.
+
+ - Provide a way to listen to incoming EGL/GLES commands from the guest.
+ This is done by providing a so-called "port number" to initOpenglRender().
+
+ By default, the port number corresponds to a local TCP port number that the
+ renderer will bind to and listen. Every new connection to this port will
+ correspond to the creation of a new guest host connection, each such
+ connection corresponding to a distinct thread in the guest system.
+
+ For performance reasons, it is possible to listen to either Unix sockets
+ (on Linux and OS X), or to a Win32 named pipe (on Windows). To do so, one
+ had to call setStreamType() between library initialization
+ (i.e. initLibrary()) and construction (i.e. initOpenglRender()).
+
+ Note that in these modes, the port number is still used to differentiate
+ between several emulator instances. These details are normally handled by
+ the emulator code so you shouldn't care too much.
+
+Note that an earlier version of the interface allowed a client of the renderer
+library to provide its own IOStream implementation. However, this wasn't very
+convenient for a number of reasons. This maybe something that could be done
+again if it makes sense, but for now the performance numbers are pretty good.
+
+
+Host emulator:
+--------------
+
+The code under $QEMU/android/opengles.c is in charge of dynamically loading
+the rendering library and initializing / constructing it properly.
+
+QEMU pipe connections to the 'opengles' service are piped through the code
+in $QEMU/android/hw-pipe-net.c. Look for the openglesPipe_init() function,
+which is in charge of creating a connection to the renderer library
+(either through a TCP socket, or a Unix pipe depending on configuration.
+support for Win32 named pipes hasn't been implemented yet in the emulator)
+whenever a guest process opens the "opengles" service through /dev/qemu_pipe.
+
+There is also some support code for the display of the GLES framebuffer
+(through the renderer library's subwindow) under $QEMU/skin/window.
+
+Note that at the moment, scaling and rotation are supported. However,
+brightness emulation (which used to modify the pixel values from the
+hardware framebuffer before displaying them) doesn't work.
+
+Another issue is that it is not possible to display anything on top of the
+GL subwindow at the moment. E.g. this will obscure the emulated trackball
+image (that is normally toggled with Ctrl-T during emulation, or enabled
+by pressing the Delete key).
+
Code Review / iec.git / blobdiff