Update #2: The need for Wayland

March 24, 2016

We left off previously in that I was about to write code to load xkbcommon keymaps into the X server, that soon proved to be not a very good idea because the keymap type on xkbcommon is an opaque structure, this means the authors want to be able to change its organization at their will without breaking someone else’s code, so the only way to manipulate it is through their API. We could technically bypass this but I think the maintainer burden added would be too much, and it would make difficult to update to newer versions of the library. Another way to accomplish this is as a patch to the tree to libxkbcommon, which sounds fine, but as I said before I don’t want to bother other people because this is sort of an experiment, also it would add code there that will be useless once we move to Wayland, and will also make people less willing to move out from X.

After putting this to the side, it occurred to me that I could use the .vapi file I already had for libxkbcommon to manually translate keypresses into their unicode characters and then send them to the applications, I just needed to know where to “tap” the event stream going from a keypress up to when it was sent to an application. This seamed a reasonable idea because I would have to do it anyway to implement ibus (or at least it seemed like the obvious way to do so). Turns out this stream can’t be tapped because it doesn’t even go through Gala, and it also not how ibus is implemented (I guess precisely due to impossibility of intercepting keypress events). So how does this actually work? I’ll try to explain briefly next.

The first warning sign that told me this wasn’t how things worked was how ibus is implemented in Gnome’s Shell, there is a comment in their code that explains this, it says events reach the X server, then Mutter intercepts all of them, lets Clutter handle them through the clutter_x11_handle_event() function which will send them to the corresponding application (Clutter Actor), then the application receives an event that hasn’t been translated yet, notices an input method is enabled and sends it to the d-bus daemon that will push the resulting translation into a gdk event stream that the shell is listening into (through Mutter). Mutter then, assumes every event here comes from ibus, so it sends it back to Clutter so it reaches the corresponding Clutter Actor again.

At this point I decided to verify all this by myself, because if we are filtering all X events through Mutter and sending them to the corresponding Clutter Actor, why can’t we translate them in Mutter and then send them just once without this strange round tripping to the application?. I found 3 points at which I could intercept this event stream to see what was happening.

I knew there was a way of intercepting Clutter events, but nothing guaranteed that the function clutter_x1_handle_event() handled X events by translating them into Clutter events. After some reading of Clutter’s code I verified this was actually true, X events are translated and pushed into Clutter’s event queue. So I thought adding a Clutter event filter would show all keypresses before them being delivered to the application, if this were true I would just need to translate the keycode as I wanted and let the event reach is normal destination. After doing this I found out no CLUTTER_KEYPRESS events where going through (except for the tab and alt key while alt tabbing), what was going on here? at some point we were loosing events, but they were reaching the application because everything worked normally, so was there another event queue somewhere handling this?.

Then I found out another filter on ClutterX11 this one received an X event and a Clutter event, my guess was that this function was called when translating a X event into a Clutter event, so if my function filled the Clutter event appropriately and returned TRANSLATED then I would be doing the translation and continue sending the Clutter event instead of the X event. So I tried listening for X keypress events. Sadly enough, there were still no keypress events to be seen. I could only see some events of type GenericEvent, this seemed like another dead end. After some research I found out that listening for X key events was very naive, now a days to support weird input devices like Wacom tablets we use XInput2 that uses GenericEvents to send bigger events than what X supports. So I needed to decode these GenericEvents (there seemed to be a lot of these, just what I would expect if every keypress and keyrelease was being sent) to know if there were some XI_keypress events going through there. Doing this wasn’t a trivial task because XInput2 does not have much documentation and there are no Vala bindings for this. After creating a .vapi file for XI2.h I still had some trouble because I needed to cast a struct pointer to another struct pointer which is not documented on the manual vapi file tutorial (turns out to be a simple pointer cast on vala). After all this I could finally check if my keypresses were going through here, and guess what… they weren’t, only the tab and alt key I had seen before were going through here.

The third point at which I could intercept events was overriding the method Meta.Plugin.xevent_filter(), I could see exactly where it was being called from Mutter and I am pretty confident all X events that reach Mutter go through this before anything happens, in fact handling the event here would make it never to go through Clutter at all. Doing things here had a problem though, I would have to find the correct Clutter Actor that should get the event by myself, but nevertheless if keypresses were going through here it would be something I could work with. So I copied my code from the ClutterX11 filter here, and if stuff wasn’t disappointing enough already… well, no interesting keypress events were to be seen here either. So again, what’s going on here?.

My conclusion of all this is, key press events don’t ever reach Mutter and consequently neither Gala. This means the comment on Gnome Shell’s code is quite misleading or maybe I misunderstood it from the beginning. Events reach the application (X client) directly then if an input method is enabled these are sent to the window manager to be displayed on the input method window (the bubble). Keycode translation does not happen in the window manager but in the application itself, mostly hidden into Gtk so that application developers don’t have to do this explicitly.

It seems to me like i18n is a hard problem that no one can claim to have solved completely without knowing every language that a unicode string may represent. This makes people unwilling to commit to an API that will be frozen forever. Instead what I feel is that projects just pass the ball around to other projects without no one trying to actually solve it directly. Wayland right now only has basic keycode translation with libxkbcommon just like X did with xkb, and leaves everything else to the applications, but then Gtk tries to make easy to create internationalized applications by hiding all this below some API. Let’s hope wayland-im will improve things.

Where does this leave us regarding Gala?, well I’ve came to the conclusion that what I wanted to do (load arbitrary xkb files) can’t be done in a non hacky way, because currently as an X window manager we don’t have full control of the events going through, this will only be true once we move to Wayland (and even then we may have to deal with the remaining abstractions (Mutter and Clutter). Still I think users can’t wait for Wayland right now so I’ve decided to move away from my original idea and fix this temporarily for the next elementary release just the way Gnome does it. I will then start looking into moving Gala to Wayland because I think this is what we really need to take the X server out of the middle and let us handle events by ourselves. I also think we need a better infrastructure that allows developers to test things quickly before actually committing and releasing to everyone else, without having to deal with code that is outside in abstract dependencies, so we’ll see how that turns out later.