1 Android Hardware OpenGLES emulation design overview
2 ===================================================
7 Hardware GLES emulation in the Android platform is implemented with a mix
8 of components, which are:
10 - Several host "translator" libraries. They implement the EGL, GLES 1.1 and
11 GLES 2.0 ABIs defined by Khronos, and translate the corresponding function
12 calls into calls to the appropriate desktop APIs, i.e.:
14 - GLX (Linux), AGL (OS X) or WGL (Windows) for EGL
15 - desktop GL 2.0 for GLES 1.1 and GLES 2.0
17 _________ __________ __________
19 |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST
20 | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR
21 |_________| |__________| |__________| LIBRARIES
23 - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
25 ____v____ ____v_____ _____v____ HOST
27 | GLX | | GL 2.0 | | GL 2.0 | LIBRARIES
28 |_________| |__________| |__________|
32 - Several system libraries inside the emulated guest system that implement
33 the same EGL / GLES 1.1 and GLES 2.0 ABIs.
35 They collect the sequence of EGL/GLES function calls and translate then
36 into a custom wire protocol stream that is sent to the emulator program
37 through a high-speed communication channel called a "QEMU Pipe".
39 For now, all you need to know is that the pipe is implemented with a
40 custom kernel driver, and provides for _very_ fast bandwidth. All read()
41 and writes() from/to the pipes are essentially instantaneous from the
42 guest's point of view.
45 _________ __________ __________
47 |EMULATION| |EMULATION | |EMULATION | GUEST
48 | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM
49 |_________| |__________| |__________| LIBRARIES
51 - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
53 ____v____________________v____________________v____ GUEST
56 |___________________________________________________|
58 - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - -
63 - Specific code inside the emulator program that is capable of transmitting
64 the wire protocol stream to a special rendering library or process (called
65 the "renderer" here), which understands the format.
68 | PROTOCOL BYTE STREAM
74 | UNMODIFIED PROTOCOL BYTE STREAM
81 - The renderer decodes the EGL/GLES commands from the wire
82 protocol stream, and dispatches them to the translator libraries
86 | PROTOCOL BYTE STREAM
92 +-----------------+ | +-----------------+
94 ____v____ ___v______ ____v_____
96 |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST
97 | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR
98 |_________| |__________| |__________| LIBRARIES
102 - In reality, the protocol stream flows in both directions, even though most
103 of the commands result in data going from the guest to the host. A complete
104 picture of the emulation would thus be:
110 _________ __________ __________
112 |EMULATION| |EMULATION | |EMULATION | GUEST
113 | EGL | | GLES 1.1 | | GLES 2.0 | SYSTEM
114 |_________| |__________| |__________| LIBRARIES
117 - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
119 ____v____________________v____________________v____ GUEST
122 |___________________________________________________|
125 - - - - - - - - - - - - - -|- - - - - - - - - - - - - - - -
127 | PROTOCOL BYTE STREAM
133 | UNMODIFIED PROTOCOL BYTE STREAM
140 +-----------------+ | +-----------------+
142 ____v____ ___v______ ____v_____
144 |TRANSLATOR |TRANSLATOR| |TRANSLATOR| HOST
145 | EGL | | GLES 1.1 | | GLES 2.0 | TRANSLATOR
146 |_________| |__________| |__________| LIBRARIES
149 - - - | - - - - - - - - - | - - - - - - - - - | - - - - -
151 ____v____ ____v_____ _____v____ HOST
153 | GLX | | GL 2.0 | | GL 2.0 | LIBRARIES
154 |_________| |__________| |__________|
156 (NOTE: 'GLX' is for Linux only, replace 'AGL' on OS X, and 'WGL' on Windows).
159 Note that, in the above graphics, only the host system libraries at the bottom
160 are _not_ provided by Android.
166 The above design comes from several important requirements that were decided
167 early in the project:
169 1 - The ability to run the renderer in a separate process from the emulator
172 For various practical reasons, we plan to completely separate the core QEMU
173 emulation from the UI window by using two distinct processes. As such, the
174 renderer will be implemented as a library inside the UI program, but will
175 need to receive protocol bytes from the QEMU process.
177 The communication channel will be either a fast Unix socket or a Win32
178 named pipe between these two. A shared memory segment with appropriate
179 synchronization primitives might also be used if performance becomes
182 This explains why the emulator doesn't alter or even try to parse the
183 protocol byte stream. It only acts as a dumb proxy between the guest
184 system and the renderer. This also avoids adding lots of GLES-specific
185 code inside the QEMU code base which is terribly complex.
187 2 - The ability to use vendor-specific desktop EGL/GLES libraries is
190 GPU vendors like NVidia, AMD or ARM all provide host versions of the
191 EGL/GLES libraries that emulate their respectivie embedded graphics
194 The renderer library can be configured to use these instead of the
195 translator libraries provided with this project. This can be useful to
196 more accurately emulate the behaviour of specific devices.
198 Moreover, these vendor libraries typically expose vendor-specific
199 extensions that are not provided by the translator libraries. We cannot
200 expose them without modifying our code, but it's important to be able
201 to do so without too much pain.
207 All source code for the components above is spread over multiple directories
208 in the Android source trees:
210 - The emulator sources are under $ANDROID/external/qemu, which we'll
211 call $QEMU in the rest of this document.
213 - The guest libraries are under
214 $ANDROID/device/generic/goldfish/opengl, which we'll call $EMUGL_GUEST
216 - The host renderer and translator libraries are under
217 $QEMU/distrib/android-emugl, which we'll call $EMUGL_HOST
219 - The QEMU Pipe kernel driver is under $KERNEL/drivers/misc/qemupipe (3.4)
220 or $KERNEL/drivers/platform/goldfish/goldfish_pipe.c (3.10)
222 Where $ANDROID is the top of the open-source Android source tree, and
223 $KERNEL is the top of the qemu-specific kernel source tree (using one
224 of the android-goldfish-xxxx branches here).
226 The emulator sources related to this projects are:
228 $QEMU/hw/android/goldfish/pipe.c -> implements QEMU pipe virtual hardware.
229 $QEMU/android/opengles.c -> implements GLES initialization.
230 $QEMU/android/hw-pipe-net.c -> implements the communication channel
231 between the QEMU Pipe and the renderer library
233 The other sources are:
235 $EMUGL_GUEST/system -> system libraries
236 $EMUGL_GUEST/shared -> guest copy of shared libraries
237 $EMUGL_GUEST/tests -> various test programs
239 $EMUGL_HOST/host -> host libraries (translator + renderer)
240 $EMUGL_HOST/shared -> host copy of shared libraries
242 The reason the shared libraries aren't actually shared is historical: at one
243 point both guest and host code lived in the same place. That turned out to be
244 impractical with the way the Android SDK is branched, and didn't support the
245 requirement that a single emulator binary be able to run several releases
249 Translator libraries:
250 ---------------------
252 There are three translator host libraries provided by this project:
254 libEGL_translator -> EGL 1.2 translation
255 libGLES_CM_translator -> GLES 1.1 translation
256 libGLES_V2_translator -> GLES 2.0 translation
258 The full name of the library will depend on the host system.
259 For simplicity, only the library name suffix will change (i.e. the
260 'lib' prefix is not dropped on Windows), i.e.:
262 libEGL_translator.so -> for Linux
263 libEGL_translator.dylib -> for OS X
264 libEGL_translator.dll -> for Windows
266 The source code for these libraries is located under the following
267 path in the Android source tree:
269 $EMUGL_HOST/host/libs/Translator/EGL
270 $EMUGL_HOST/host/libs/Translator/GLES_CM
271 $EMUGL_HOST/host/libs/Translator/GLES_V2
273 The translator libraries also use a common routines defined under:
275 $EMUGL_HOST/host/libs/Translator/GLcommon
278 Wire Protocol Overiew:
279 ----------------------
281 The "wire protocol" is implemented as follows:
283 - EGL/GLES function calls are described through several "specification"
284 files, which describes the types, function signatures and various
285 attributes for each one of them.
287 - These files are read by a tool called "emugen" which generates C
288 source files and headers based on the specification. These correspond
289 to both encoding, decoding and "wrappers" (more on this later).
291 - System "encoder" static libraries are built using some of these generated
292 files. They contain code that can serialize EGL/GLES calls into simple
293 byte messages and send it through a generic "IOStream" object.
295 - Host "decoder" static libraries are also built using some of these
296 generated files. Their code retrieves byte messages from an "IOStream"
297 object, and translates them into function callbacks.
299 IOStream abstraction:
300 - - - - - - - - - - -
302 The "IOStream" is a very simple abstract class used to send byte messages
303 both in the guest and host. It is defined through a shared header under
304 $EMUGL/host/include/libOpenglRender/IOStream.h
306 Note that despite the path, this header is included by *both* host and guest
307 source code. The main idea around IOStream's design is that to send a message,
308 one does the following:
310 1/ call stream->allocBuffer(size), which returns the address of a
311 memory buffer of at least 'size' bytes.
313 2/ write the content of the serialized command (usually a header + some
314 payload) directly into the buffer
316 3/ call stream->commitBuffer() to send it.
318 Alternatively, one can also pack several commands into a single buffer with
319 stream->alloc() and stream->flush(), as in:
321 1/ buf1 = stream->alloc(size1)
322 2/ write first command bytes into buf1
323 3/ buf2 = stream->alloc(size2)
324 4/ write second command bytes into buf2
327 Finally, there are also explict read/write methods like stream->readFully()
328 or stream->writeFully() which can be used when you don't want an intermediate
329 buffer. This is used in certain cases by the implementation, e.g. to avoid
330 an intermediate memory copy when sending texture data from the guest to the
333 The host IOStream implementations are under $EMUGL/shared/OpenglCodecCommon/,
336 $EMUGL_HOST/shared/OpenglCodecCommon/TcpStream.cpp
337 -> using local TCP sockets
338 $EMUGL_HOST/shared/OpenglCodecCommon/UnixStream.cpp
339 -> using Unix sockets
340 $EMUGL_HOST/shared/OpenglCodecCommon/Win32PipeStream.cpp
341 -> using Win32 named pipes
343 The guest IOStream implementation uses the TcpStream.cpp above, as well as
344 an alternative QEMU-specific source:
346 $EMUGL_GUEST/system/OpenglSystemCommon/QemuPipeStream.cpp
347 -> uses QEMU pipe from the guest
349 The QEMU Pipe implementation is _significantly_ faster (about 20x) due to
352 - all succesful read() and write() operations through it are instantaneous
353 from the guest's point of view.
355 - all buffer/memory copies are performed directly by the emulator, and thus
356 much faster than performing the same thing inside the kernel with emulated
359 - it doesn't need to go through a kernel TCP/IP stack that will wrap the
360 data into TCP/IP/MAC packets, send them to an emulated ethernet device,
361 which is itself connected to an internal firewall implementation that
362 will unwrap the packets, re-assemble them, then send them through BSD
363 sockets to the host kernel.
365 However, would it be necessary, you could write a guest IOStream implementation
366 that uses a different transport. If you do, please look at
367 $EMUGL_GUEST/system/OpenglCodecCommon/HostConnection.cpp which contains the
368 code used to connect the guest to the host, on a per-thread basis.
371 Source code auto-generation:
372 - - - - - - - - - - - - - -
374 The 'emugen' tool is located under $EMUGL_HOST/host/tools/emugen. There is a
375 README file that explains how it works.
377 You can also look at the following specifications files:
380 $EMUGL_HOST/host/GLESv1_dec/gl.types
381 $EMUGL_HOST/host/GLESv1_dec/gl.in
382 $EMUGL_HOST/host/GLESv1_dec/gl.attrib
385 $EMUGL_HOST/host/GLESv2_dec/gl2.types
386 $EMUGL_HOST/host/GLESv2_dec/gl2.in
387 $EMUGL_HOST/host/GLESv2_dec/gl2.attrib
390 $EMUGL_HOST/host/renderControl_dec/renderControl.types
391 $EMUGL_HOST/host/renderControl_dec/renderControl.in
392 $EMUGL_HOST/host/renderControl_dec/renderControl.attrib
394 Note that the EGL specification files are under a directory named
395 "renderControl_dec" and have filenames that begin with "renderControl"
397 This is mainly for historic reasons now, but is also related to the fact that
398 this part of the wire protocol contains support functions/calls/specifications
399 that are not part of the EGL specification itself, but add a few features
400 required to make everything works. For example, they have calls related to
401 the "gralloc" system library module used to manage graphics surfaces at a
402 lower level than EGL.
404 Generally speaking, guest encoder sources are located under directories
405 named $EMUGL_GUEST/system/<name>_enc/, while the corresponding host decoder
406 sources will be under $EMUGL_HOST/host/libs/<name>_dec/
408 However, all these sources use the same spec files located under the
409 decoding directories.
411 The encoder files are built from emugen and spec files located in $EMUGL_HOST
412 and copied to the encoder directories in $EMUGL_GUEST by the gen-encoder.sh
413 script. They are checked in, so that a given version of Android supports a
414 specific version of the protocol, even if newer versions of the renderer (and
415 future Android versions) support a newer protocol version. This step needs to
416 be done manually when the protocol changes; these changes also need to be
417 accompanied by changes in the renderer to handle the old version of the
424 Meta EGL/GLES system libraries, and egl.cfg:
425 - - - - - - - - - - - - - - - - - - - - - -
427 It is important to understand that the emulation-specific EGL/GLES libraries
428 are not directly linked by applications at runtime. Instead, the system
429 provides a set of "meta" EGL/GLES libraries that will load the appropriate
430 hardware-specific libraries on first use.
432 More specifically, the system libEGL.so contains a "loader" which will try
435 - hardware-specific EGL/GLES libraries
436 - the software-based rendering libraries (called "libagl")
438 The system libEGL.so is also capable of merging the EGL configs of both the
439 hardware and software libraries transparently to the application. The system
440 libGLESv1_CM.so and libGLESv2.so, work with it to ensure that the thread's
441 current context will be linked to either the hardware or software libraries
442 depending on the config selected.
444 For the record, the loader's source code in under
445 frameworks/base/opengl/libs/EGL/Loader.cpp. It depends on a file named
446 /system/lib/egl/egl.cfg which must contain two lines that look like:
451 The first number in each line is a display number, and must be 0 since the
452 system's EGL/GLES libraries don't support anything else.
454 The second number must be 1 to indicate hardware libraries, and 0 to indicate
455 a software one. The line corresponding to the hardware library, if any, must
456 always appear before the one for the software library.
458 The third field is a name corresponding to a shared library suffix. It really
459 means that the corresponding libraries will be named libEGL_<name>.so,
460 libGLESv1_CM_<name>.so and libGLESv2_<name>.so. Moreover these libraries must
461 be placed under /system/lib/egl/
463 The name "android" is reserved for the system software renderer.
465 The egl.cfg that comes with this project uses the name "emulation" for the
466 hardware libraries. This means that it provides an egl.cfg file that contains
472 See $EMUGL_GUEST/system/egl/egl.cfg and more generally the following build
475 $EMUGL_GUEST/system/egl/Android.mk
476 $EMUGL_GUEST/system/GLESv1/Android.mk
477 $EMUGL_GUEST/system/GLESv2/Android.mk
479 to see how the libraries are named and placed under /system/lib/egl/ by the
484 - - - - - - - - - - -
486 The emulator-specific libraries are under the following:
488 $EMUGL_GUEST/system/egl/
489 $EMUGL_GUEST/system/GLESv1/
490 $EMUGL_GUEST/system/GLESv2/
492 The code for GLESv1 and GLESv2 is pretty small, since it mostly link against
493 the static encoding libraries.
495 The code for EGL is a bit more complex, because it needs to deal with
496 extensions dynamically. I.e. if an extension is not available on the host
497 it shouldn't be exposed by the library at runtime. So the EGL code queries
498 the host for the list of available extensions in order to return them to
499 clients. Similarly, it must query the list of valid EGLConfigs for the
503 "gralloc" module implementation:
504 - - - - - - - - - - - - - - - - -
506 In addition to EGL/GLES libraries, the Android system requires a
507 hardware-specific library to manage graphics surfaces at a level lower than
508 EGL. This library must be what is called in Android land as a "HAL module".
510 A "HAL module" must provide interfaces defined by Android's HAL
511 (Hardware Abstraction Library). These interface definitions can be found
512 under $ANDROID/hardware/libhardware/include/
514 Of all possible HAL modules, the "gralloc" one is used by the system's
515 SurfaceFlinger to allocate framebuffers and other graphics memory regions,
516 as well as eventually lock/unlock/swap them when needed.
518 The code under $EMUGL/system/gralloc/ implements the module required by the
519 GLES emulation project. It's not very long, but there are a few things to
522 - first, it will probe the guest system to determine if the emulator that
523 is running the virtual device really supports GPU emulation. In certain
524 circumstances this may not be possible.
526 If this is the case, then the module will redirect all calls to the
527 "default" gralloc module that is normally used by the system when
528 software-only rendering is enabled.
530 The probing happens in the function "fallback_init" which gets called
531 when the module is first opened. This initializes the 'sFallback' variable
532 to a pointer to the default gralloc module when required.
534 - second, this module is used by SurfaceFlinger to display "software surfaces",
535 i.e. those that are backed by system memory pixel buffers, and written to
536 directly through the Skia graphics library (i.e. the non-accelerated ones).
538 the default module simply copies the pixel data from the surface to the
539 virtual framebuffer i/o memory, but this project's gralloc module sends it
540 to the renderer through the QEMU Pipe instead.
542 It turns out that this results in _faster_ rendering/frame-rates overall,
543 because memory copies inside the guest are slow, while QEMU pipe transfers
544 are done directly in the emulator.
550 The host renderer library is located under
551 $EMUGL_HOST/host/libs/libOpenglRender, and it provides an interface described
552 by the headers under $EMUGL_HOST/host/libs/libOpenglRender/render_api.h
553 (e.g. for use by the emulator).
555 In a nutshell, the rendering library is responsible for the following:
557 - Providing a virtual off-screen video surface where everything will get
558 rendered at runtime. Its dimensions are fixed by the call to
559 initOpenglRender() that must happen just after the library is
562 - Provide a way to display the virtual video surface on a host application's
563 UI. This is done by calling createOpenGLSubWindow() which takes as argument
564 the window ID or handle of a parent window, some display dimensions and
565 a rotation angle. This allows the surface to be scaled/rotated when it is
566 displayed, even if the dimensions of the video surface do not change.
568 - Provide a way to listen to incoming EGL/GLES commands from the guest.
569 This is done by providing a so-called "port number" to initOpenglRender().
571 By default, the port number corresponds to a local TCP port number that the
572 renderer will bind to and listen. Every new connection to this port will
573 correspond to the creation of a new guest host connection, each such
574 connection corresponding to a distinct thread in the guest system.
576 For performance reasons, it is possible to listen to either Unix sockets
577 (on Linux and OS X), or to a Win32 named pipe (on Windows). To do so, one
578 had to call setStreamType() between library initialization
579 (i.e. initLibrary()) and construction (i.e. initOpenglRender()).
581 Note that in these modes, the port number is still used to differentiate
582 between several emulator instances. These details are normally handled by
583 the emulator code so you shouldn't care too much.
585 Note that an earlier version of the interface allowed a client of the renderer
586 library to provide its own IOStream implementation. However, this wasn't very
587 convenient for a number of reasons. This maybe something that could be done
588 again if it makes sense, but for now the performance numbers are pretty good.
594 The code under $QEMU/android/opengles.c is in charge of dynamically loading
595 the rendering library and initializing / constructing it properly.
597 QEMU pipe connections to the 'opengles' service are piped through the code
598 in $QEMU/android/hw-pipe-net.c. Look for the openglesPipe_init() function,
599 which is in charge of creating a connection to the renderer library
600 (either through a TCP socket, or a Unix pipe depending on configuration.
601 support for Win32 named pipes hasn't been implemented yet in the emulator)
602 whenever a guest process opens the "opengles" service through /dev/qemu_pipe.
604 There is also some support code for the display of the GLES framebuffer
605 (through the renderer library's subwindow) under $QEMU/skin/window.
607 Note that at the moment, scaling and rotation are supported. However,
608 brightness emulation (which used to modify the pixel values from the
609 hardware framebuffer before displaying them) doesn't work.
611 Another issue is that it is not possible to display anything on top of the
612 GL subwindow at the moment. E.g. this will obscure the emulated trackball
613 image (that is normally toggled with Ctrl-T during emulation, or enabled
614 by pressing the Delete key).