How to debug Wayland problems

Wayland is a successor of Xorg server (which is the most widely used implementation of X11 - the X Window System). It changes the design of a Linux desktop architecture considerably. Unlike Xorg, there is no dedicated standalone server in Wayland. What was previously done between the app, its toolkit, the Xserver and the window manager is now shared between the app, its toolkit and the Wayland compositor which manages the compositing, input, windows management, etc. The apps and toolkits are now in charge of their own rendering and decorations (client side decorations), so any issues usually sit between the toolkit (e.g. GTK+) and the Wayland compositor (e.g. mutter).

Identifying Wayland problems

Are you running a Wayland session?

In GNOME, there's a gear button at the login screen which can be used to either log into standard GNOME desktop (Xorg session), or GNOME on Wayland session (if you have a password-less user account, you won't see the gear icon, it is displayed only when the password prompt appears). Use the gear button to determine type of session you're logging into.

FIXME: how to run wayland with KDE?

Other desktop environments are not capable of running a Wayland session right now.

If you want to figure out which type of session you're running right now, without logging out and in again, .... FIXME: how to do that?

If you're not running Xorg session, not a Wayland session, your problems are not related to Wayland. It's a bug either in that particular application, or Xorg itself, see How to debug Xorg problems

Does your application run on Wayland natively, or uses XWayland (Xorg compatibility layer)?

It is important to know whether the problematic application is a native Wayland application, or runs through XWayland, which allows legacy applications to still run on top of Xorg server, but display in a Wayland session.

There are several ways how to identify whether an application is using Wayland or XWayland:

• Select the window using xwininfo or xprop. Run:

  $ xwininfo

  Your mouse pointer should change to a cross under Xorg, it doesn't seem to change under Wayland. Now click anywhere inside the app window you want to test. If the xwininfo command finishes (it should print window properties into the terminal), the app under test is running under XWayland. If nothing happens (the xwininfo command is still waiting until you select a window), the app under test is running under Wayland (you can close the command with Ctrl+C).
  You can also use xprop command using the same instructions.

• XWayland applications are listed in xlsclients output. Run:

  $ xlsclients

  However, this list of not always entirely reliable, some apps might be missing.

• You can try to run the app while unsetting DISPLAY environment variable:

  $ DISPLAY='' command

  If the application runs OK, it should be using Wayland natively.

• You can run the app with WAYLAND_DEBUG=1 environment variable:

  $ WAYLAND_DEBUG=1 command

  If you see loads of output (when compared to a standard run), the app is using Wayland natively.

If you have identified the problem to be in a XWayland application, try to reproduce the issue in a standard Xorg session. If it happens as well, it is not related to Wayland, it's a bug either in that particular application, or Xorg itself, see How to debug Xorg problems. If the problem happens only under XWayland but not in an Xorg session... FIXME: what to do now? report against xwayland somewhere?

Identifying problem component

Wayland itself is a protocol and the problem is rarely in the protocol itself. Rather, the problem is likely to be in the app or its toolkit, or in the compositor.

The most notable Wayland-ready toolkits are:

• GTK+ - default apps in GNOME environment use almost exclusively this toolkit, also heavily used in XFCE, MATE, LXDE, Cinnamon
• QT - default apps in KDE environment use almost exclusively this toolkit

The most notable Wayland compositors are:

• Weston - the reference implementation of a Wayland compositor, maintained directly by the Wayland project
• Mutter - compositor in GNOME. If you run GNOME, it is using this compositor.
• Kwin - compositor in KDE. If you run KDE, it is using this compositor.

Testing under different compositors

If you experience a problem with a Wayland app, it is very useful to know whether the problem is present under just a single compositor (in that case it's likely a compositor bug) or under multiple compositors (in that case it's likely an app/toolkit bug).

Please run your session with the reference Weston compositor and try to reproduce the issue. FIXME: how to do that?

If you can reproduce the issue with Weston, file an issue against the app or its toolkit (gtk+, qt, etc). Otherwise file the issue against the compositor your environment uses (mutter, kwin, etc).

Reporting the issue

After you've identified against which component to (most probably) report the issue, there are several places where to report it:

• Red Hat Bugzilla - recommended for issues related to wayland itself, weston compositor, non-GNOME apps, KDE project, QT toolkit
• GNOME Bugzilla - recommended for issues related to mutter compositor, GTK+ toolkit, applications under the GNOME project (most of default apps in Fedora Workstation)

When reporting the issue, please make your report block our tracker, so that we have a good overall picture of what is broken across many different components. In your bug report, set Blocks: WaylandRelated (one of the fields available after you've reported the bug, you might need to toggle showing advanced fields). That will make it block one of these trackers, depending where you reported the bug:

• Wayland Tracker in Red Hat Bugzilla
• Wayland Tracker in GNOME Bugzilla

Information to include in your bug report

1. System journal. Since there is no unique server like the X11 server, most of the important information will come from the the Wayland compositor and the apps. All of that should be in system journal nowadays. You can save a full journal since last boot like this:

   $ journalctl -ab > journal.log

   You can also edit the file and according to the timestamps remove everything long prior to when the issue occurred, in order to make the log more succinct.

   • If your system crashed or became unresponsive so that you had to reboot it, you can see the journal from the previous boot using journalctl -a -b -1 instead.
2. Wayland debug output. If you can reproduce the issue, please run the problematic app like this:

   $ WAYLAND_DEBUG=1 command |& tee debug.out

   You should see loads of output being printed out. It will involve all communication between the app and the compositor.

3. Information whether the same problem occurs when you run the app using XWayland instead of Wayland. For GTK+ apps, you can force a native Wayland app to run using XWayland like this:

   $ GDK_BACKEND=x11 command

   Vice versa, you can also force a XWayland app to run using Wayland (in case it has just experimental support):

   $ GDK_BACKEND=wayland command

   All of this applies to just GTK+ apps. FIXME: what about other toolkits, like QT?

4. Hardware description is useful for some hardware-related bugs:

   $ lspci -nn > lspci.out

5. Package versions. You can collect the list and versions of all your packages installed using:

   $ rpm -qa | sort > packages.out

6. The usual information that every bug report should have.

Known issues, frequent complaints, fundamental changes

Here we will list high-profile issues which are known to be broken, not yet implemented, or intentionally behaving differently from regular Xorg apps.

You can see more of known issues in our Bugzilla trackers, see Fedora Wayland tracker and GNOME Wayland tracker.

There is no middle-click clipboard selection

Under Xorg, you have one clipboard accessible through well-known ctrl+c, ctrl+v shortcuts, and another clipboard (called primary selection) that copies everything you highlight using the cursor and pastes text by pressing the middle mouse button. The former clipboard is still accessible in Wayland programs, but the middle-click clipboard is no longer supported. That is a design choice, not a bug, though quite contentious.

Many well-known X11 utilities don't work

Power users are familiar with a large range of X11-related utilities, like xkill, xrandr, xdotool, xsel. These tools won't work under Wayland session, or will only work with XWayland applications but not Wayland applications. Some tools might have a replacement which allows to perform similar tasks.

FIXME: add some Wayland-ready replacements for popular X11 tools