RenderingProject/CMArchitecture

= Architectures for a Compositing Manager =

There are various ways that we can implement an X compositing manager; this document tries to sketch out the tradeoffs involved.

1. Compositing manager on top of RENDER
Using RENDER to implement a compositing manager has the advantage that RENDER works directly with the objects that the compositing manager is manipulating; pixmaps and windows.

But it has disadvantages; first, the RENDER API does not expose the full power of modern graphics cards. If you want to say, use pixel shaders, in your compositing manager, you are out of luck.

Second, to get decent performance, we need hardware acceleration. The basic problem with hardware acceleration of RENDER is that it doesn't always match the hardware that well; in particular, for filtered scaling, hardware is mipmap based, a concept not exposed in RENDER.

Hardware accelerating RENDER to get good performance for the drawing we do within windows is not challenging, but getting good performance for full-window and full-desktop operations may be harder.

1.a. RENDER acceleration written directly to hardware
The first approach to accelerating RENDER is what we do currently; directly program the hardware. At some level, this allows optimum performance, but involves duplicating work that's being done in the 3D drivers; which have a much more active development community. RENDER is also an unknown quantity to hardware vendors so we're unlikely to get good support in closed-source or vendor-written drivers. Using a known API like OpenGL to define the hardware interaction would give us much more of a common language with hardware vendors.

1.b. RENDER acceleration written on top of GL in the server.
The other approach is to implement RENDER on top of GL. Since GL is a pretty close match to the hardware, we shouldn't lose a lot of efficiency doing this, and features such as pixel shaders should eventually allow for very high-powered implementations of RENDER compositing. A start of this work has been done by Dave Reveman for the 'glitz' library.

1.b.i. GL in the server done with a GL-based server
If we are accelerating RENDER in the server with GL, we clearly need GL in the server. We could do this by running the entire server as a GL client, on top of something like mesa-solo, or nested on top of the existing server like Xgl. Both have their disadvantages; mesa-solo is far from usable for hosting an X server, and nested X server still requires that the "backend" X server be maintained.

1.b.ii. GL in the server done incrementally
An alternative approach would to be to start by implementing indirect rendering; once we had that, we'd have DRI drivers running inside the server process. It would be conceivable to use those DRI drivers to implement parts of the 2D rendering stack while keeping all the video card initialization, Xvideo implementation, and so forth the same. This work has been started in the accel_indirect_glx branch of Xorg and is making good progress.

2.a. Compositing manager renders to separate server
The way that luminocity works is that it runs a headless "Xfake" server (all software), sucks the window contents off of that server then displays them on an output server. Input events are forward the other way from the output server to the headless server.

This model has a great deal of simplicity, because the CM can simply be a normal direct rendering client. And it doesn't perform that badly; rendering within windows simply isn't the limiting factor for normal desktop apps. Performance could be further optimized by running the CM in the same process as the headless server (imagine a libxserver).

The forwarding of input events is pretty difficult, and having to extend that for XKB, for Xinput, for Xrandr, and so forth would be painful, though doable. The real killer of this model is Xvideo and 3D applications. There's no way that we can get reasonable performance for such applications without having them talking to the output server.

2.b. Compositing manager renders to same server
The more reasonable final model for a GL-based compositing manager is that we use a single X server. Application rendering, whether classic X, GL, or Xvideo is redirected to offscreen pixmaps. Then a GL based compositing manager transfers the contents of those pixmaps to textures and renders to the screen.

2.b.i. Compositing manager uses indirect rendering
We normally think that direct rendering is always better than indirect rendering. However, this may not be the case for the compositing manager. The source data that we are using is largely on the X server; for a direct rendering client to copy a pixmap onto a texture basically requires an XGet Image. Right now the DRI relies heavily on having a application-memory copy of all textures, but it is conceivable that it could be extended to allow textures that are "locked" into video memory. With such a texture, you could imagine the server taking care of copying from pixmap to texture, even if rendering with the texture was being done by the client.

Still, there seems to be an inevitable high synchronization overhead for such a system; the client and server are doing a complex dance to get the rendering done.