Analysis of Ozone Wayland
In the past two months, I have been working with my colleague Antonio Gomes on a Chromium project supported by Renesas. The goal is to have Chrome running on Wayland with accelerated rendering happening in a separate GPU process. Intel has done a great job to develop a specific Wayland platform for Ozone but unfortunately the code only works for older releases of Chromium and is not fully upstreamed yet. In theory, easy out-of-tree platforms was one of the guiding principles of Ozone but in practice we had to fix many build and run time errors for Ozone, even though we were working upstream. More generally, it is very challenging to keep in sync with all the refactoring work happening upstream and some work is definitely required to align Intel’s code with Google’s goals.
To summarize the situation briefly, if we follow Intel’s approach then the
currently upstreamed Wayland code only
compiles for ChromeOS (i.e.
target_os="chromeos" in your build config)
not for standard Linux
build of Chrome. You can only have accelerated rendering at the price of
removing separation of UI/GPU processes (i.e. via the
switch). If you keep the default behavior of having the UI and GPU components
running in separate processes then accelerated rendering fails. A fallback
to software rendering is then attempted but it in turn fails
When Antonio joined Igalia, he did some experiments on Ozone/Wayland and wrote a quick workaround to make the software rendering fallback work when UI and GPU are in separate processes. He was also able to run standard Linux build of Chrome on Wayland by upstreaming more code from Intel. He also noticed that in Google’s code, only GL drawing is happening in the GPU component (i.e. Wayland objects are owned by the UI, contrary to Intel’s approach) which is the reason why accelerated rendering fails in separate-process mode. After discussion with Google developers it however became clear that they would prefer a different approach that is consistent with two features they are working on:
Mus-ash, a project to separate the window management and shell functionality of ash from the chrome browser process.
Mojo, a new API for inter-process communication.
During the project, we were able to get chrome running on Wayland using the Mus code path, either on ChromeOS or on standard Linux build. For that code path, GPU and UI components are currently running in the same process so as discussed above accelerated rendering works for Wayland. However, the plan for mus is to move these two components into separate processes and hence we need to adapt the Wayland code in order to allow communication between the GPU (doing GL drawings) and the UI (owning Wayland objects).
I’ll let Antonio describe more precisely on
his blog the work we have been
doing to get
chrome --mash running on Wayland.
In this blog post, I’m aligning with Google’s goal and hence I’m focusing on
this Mus code path. I’m going to give a quick overview of the structure of
Ozone and more specifically what is used by the Wayland platform to perform
OzonePlatform is the main Ozone interface used to instantiate and initialize an Ozone platform (X11, Wayland, DRM/GBM…). It provides factory getters for helper classes (SurfaceFactoryOzone, PlatformWindow…).
The goal is to have OzonePlatform::InitializeForUI and
OzonePlatform::InitializeForGPU called from different processes as well as two
different Mojo connectors for the UI and GPU services respectively.
However at the time of writing, the two components are only in
different threads in the same process. There is also only one Mojo connector
available: The one for the
service:ui service, passed to
Native Display Delegate
NativeDisplayDelegate is used to perform display configuration. The DRM/GBM platform has its own DrmNativeDisplayDelegate implementation. For the Wayland platform, we do not actually need real display devices, so we just do as for X11 and use the FakeDisplayDelegate class. See Issue 2389053003.
AcceleratedWidget is just a platform-specific object representing a surface on which compositors can paint pixels.
PlatformWindow represents a single window in the underlying platform windowing system with the usual property: It can be minimized, maximized, closed, put in fullscreen, etc
PlatformWindowDelegate. This is just a delegate to which the PlatformWindow sends events like e.g. OnBoundsChanged.
The implementations of PlatformWindow used for the Wayland and DRM/GBM platforms are respectively WaylandWindow and DrmWindow. At the moment, several features in WaylandWindow are not fully implemented but the minimal code to paint content into the window is present.
Surface and OpenGL
SurfaceFactoryOzone provides surface to be used to paint on a window. The implementations for the Wayland and DRM/GBM platforms are respectively WaylandSurfaceFactory and GbmSurfaceFactory. There are two options:
- Accelerated drawing (GL path). This is done via a GLOzone instance returned by SurfaceFactoryOzone::GetGLOzone
- Software Drawing (Skia). This is done via a SurfaceOzoneCanvas instance returned by SurfaceFactoryOzone::CreateCanvasForWidget.
In this project, we focus on accelerated rendering and on EGL, so we consider the GLOzoneEGL class. The following virtual pure functions must be implemented in derived class:
- GLOzoneEGL::LoadGLES2Bindings, performing the GL initalization. In general, the default works well.
- GLOzoneEGL::CreateOffscreenGLSurface returning an offscreen GLSurface of the specified dimension. It seems that it is mostly needed to create a dummy zero-dimensional offscreen surface during initialization. Hence the generic SurfacelessEGL should work fine. See Issue 2387063002.
- GLOzoneEGL::GetNativeDisplay returns the EGL display connection to use.
- GLOzoneEGL::CreateViewGLSurface returns a new GLSurface associated to a gfx::AcceleratedWidget.
SurfaceFactoryOzone also provides functions to create NativePixmap, which represents a buffer that can be directly imported via GL for rendering, or exported via dma-buf fds. The DRM/GBM platform implements it and uses GbmPixmap. For now, such pixmap objects are not needed by the Wayland platform so the SurfaceFactoryOzone function members are not implemented.
The instances of GLSurface returned by CreateViewGLSurface for the Wayland and DRM/GBM platforms are respectively GLSurfaceWayland and GbmSurface. GLSurfaceWayland is just a gl::NativeViewGLSurfaceEGL associated to a window created by
wl_egl_window_create. GbmSurface instead provides surface-like
semantics by deriving from GbmSurfaceless itself deriving from gl::SurfacelessEGL. Internally, a framebuffer is bound automatically for GL drawing in the GPU and the result are exported to the UI via pixmaps.
WaylandConnection is a class specific to the Wayland platform that helps to instantiate all the objects necessary to communicate with the Wayland display server.
It also manages a map from gfx::AcceleratedWidget instances to WaylandWindow instances that you can modify with the public GetWindow, AddWindow and RemoveWindow function members.
Wayland Platform Initialization
OzonePlatformWayland::InitializeUI is called from the UI thread. It creates instances of WaylandConnection and WaylandSurfaceFactory as members of OzonePlatformWayland. The Mojo connection of the
service:uiservice is discarded.
OzonePlatformWayland::InitializeGPU in the same process but a different thread. The WaylandConnection and WaylandSurfaceFactory members previously created are hence accessible from the GPU thread too. Nothing is done by this initialization function.
A WaylandWindow is tied to a WaylandConnection and takes care of registering and unregistering itself to that WaylandConnection. It is merely a wrapper to native Wayland surfaces (wl_surface and xdg_surface) with additional window bounds. It also maintains communication with the PlatformWindowDelegate.
Wayland Surface and OpenGL
Currently, the constructor of
WaylandSurfaceFactory receives the
WaylandConnection. Because we want the UI process to hold the WaylandConnection
and the GPU process to use WaylandSurfaceFactory for the GL rendering, the
current setup will not work after mus is split. Instead, we should pass it the
Mojo connector of the
service:gpu service in order to indirectly communicate with
service:ui service (for testing purpose, we can for now just use the Mojo
connector of the
service:ui service passed to OzonePlatformWayland::InitializeUI).
The WaylandSurfaceFactory::CreateCanvasForWidget function and the WaylandCanvasSurface instance created require the WaylandConnection. However, they are used for software rendering so we can ignore them for now.
GLOzoneEGL::LoadGLES2Bindings and GLOzoneEGL::CreateOffscreenGLSurface do not seem fundamental here and they do not need any Wayland-specific code. Hence we can probably just keep the current default implementations.
At the moment GLOzoneEGLWayland::GetNativeDisplay just returns a native display provided by the WaylandConnection. It seems that we will not be able to do so when the WaylandConnection is no longer available on the GPU side. Probably we should just do like GLOzoneEGLGbm and return EGL_DEFAULT_DISPLAY.
The remaining GLOzoneEGLWayland::CreateViewGLSurface uses WaylandConnection::GetWindow to retrieve the WaylandWindow associated to AcceleratedWidget to draw on. This window provides a wl_surface and sizes that can be passed to wl_egl_window_create to create an egl_window. Then as said in the previous paragraph, a GLSurfaceWayland instance is created which is merely a gl::NativeViewGLSurfaceEGL associated to the egl_window.
In this blog post, an overview of the Ozone Architecture was provided, with focus on the Wayland platform. It also contains an analysis of how we could get accelerated rendering in a separate GPU process aligned with Google’s goals (Mus+ash and Mojo) and hence have it well-integrated with upstream code.
The main problem is that the GLOzoneEGLWayland code is very tied to the Wayland native objects (connection, surface, window…). We should instead only provide a Mojo connector to the GLOzoneEGLWayland class and hence to the GLSurfaceWayland class it constructs. Instances of WaylandWindow and WaylandConnector will only live on the UI side while the GL classes will live on the GPU side.
The GLSurfaceWayland should be rewritten to derive from SurfacelessEGL instead of NativeViewGLSurfaceEGL. We would then follow what is done in GbmSurface to provide some surface-like semantics but without the need to have a real egl_window object. Under the hood, the GL drawings will be performed on a framebuffer.
We should then find a way to export those framebuffers to the UI component via the Mojo connector. Again, following the DRM/GBM code with this NativePixmap objects seems the right option. At the end, the UI would convert the buffers into wl_buffers that can finally be attach to the wl_surface of the WaylandWindow.
During the past two months we have been maintaining the upstream Ozone code and made the Wayland platform work again. Try bots now build and run tests for the Wayland platform and we expect that such continuous testing will make the whole thing more robust. We have also been working on making Linux desktop work with the Mus+ash code path and started encouraging experiments of Mojo communication between the UI and GPU components of the Wayland platform.
It is really great to work with Antonio on this project and we are looking forward to continuing the collaboration on this with Google and Ozone developers. Last but not least, I would like to thank Renesas for supporting Igalia in this work to add Wayland support to chromium.