You are not logged in.

#1 2023-03-08 08:05:48

dmho
Member
Registered: 2023-03-08
Posts: 6

I suspect that localectl keymap conversion is broken for everyone

Problem

I am running into an issue where localectl doesn't choose correct keymaps when converting virtual console keymaps to X11 and vice versa.

When I ask localectl to set a given keymap to use for keyboard input at the virtual console (set-keymap) or with X11 programs (set-x11-keymap), by default it should convert the given keymap to the other type and update the respective configuration files for each system.

Expected result:

[root@byte user]# localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: dvorak-programmer
   X11 Layout: us
    X11 Model: pc105
  X11 Variant: dvp

Current result:

[root@byte user]# localectl set-keymap ""
[root@byte user]# localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: (unset)
   X11 Layout: (unset)

[root@byte user]# localectl set-keymap dvorak-programmer
[root@byte user]# localectl status 
System Locale: LANG=en_AU.UTF-8
    VC Keymap: dvorak-programmer
   X11 Layout: (unset)

[root@byte user]# localectl set-x11-keymap us pc105 dvp ""
[root@byte user]# localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: us-acentos
   X11 Layout: us
    X11 Model: pc105
  X11 Variant: dvp

It looks like the X11 keymap is not getting updated at all when setting the keymap for the virtual console, and in the case of setting the X11 keymap the wrong conversion is being chosen for the virtual console (us-acentos). Clearly localectl is not functioning as expected, and the logs confirm this:

Mar 08 12:03:54 byte dbus-daemon[256]: [system] Activating via systemd: service name='org.freedesktop.locale1' unit='dbus-org.freedesktop.locale1.service' requested by ':1.952' (uid=0 pid=65237 comm="localectl set-keymap dvorak-programmer")
Mar 08 12:03:54 byte systemd[1]: Starting Locale Service...
Mar 08 12:03:54 byte dbus-daemon[256]: [system] Successfully activated service 'org.freedesktop.locale1'
Mar 08 12:03:54 byte systemd[1]: Started Locale Service.
Mar 08 12:03:54 byte systemd-localed[65238]: No conversion found for virtual console keymap "dvorak-programmer".
Mar 08 12:03:54 byte systemd-localed[65238]: Changed virtual console keymap to 'dvorak-programmer' toggle ''
Mar 08 12:03:54 byte systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.
Mar 08 12:03:54 byte systemd[1]: Stopped Setup Virtual Console.
Mar 08 12:03:54 byte systemd[1]: Stopping Setup Virtual Console...
Mar 08 12:03:54 byte systemd[1]: Starting Setup Virtual Console...
Mar 08 12:03:54 byte systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully.
Mar 08 12:03:54 byte systemd-vconsole-setup[65239]: Failed to import credentials, ignoring: No such file or directory
Mar 08 12:03:54 byte systemd[1]: Finished Setup Virtual Console.
Mar 08 12:04:24 byte systemd[1]: systemd-localed.service: Deactivated successfully.

Mar 08 12:04:42 byte dbus-daemon[256]: [system] Activating via systemd: service name='org.freedesktop.locale1' unit='dbus-org.freedesktop.locale1.service' requested by ':1.954' (uid=0 pid=65253 comm="localectl set-x11-keymap us pc105 dvp ")
Mar 08 12:04:42 byte systemd[1]: Starting Locale Service...
Mar 08 12:04:42 byte dbus-daemon[256]: [system] Successfully activated service 'org.freedesktop.locale1'
Mar 08 12:04:42 byte systemd[1]: Started Locale Service.
Mar 08 12:04:42 byte systemd-localed[65254]: The X11 keyboard layout 'us' is converted to virtual console keymap 'us-acentos'
Mar 08 12:04:42 byte systemd-localed[65254]: Changed X11 keyboard layout to 'us' model 'pc105' variant 'dvp' options ''
Mar 08 12:04:42 byte systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.
Mar 08 12:04:42 byte systemd[1]: Stopped Setup Virtual Console.
Mar 08 12:04:42 byte systemd[1]: Stopping Setup Virtual Console...
Mar 08 12:04:42 byte systemd[1]: Starting Setup Virtual Console...
Mar 08 12:04:42 byte systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully.
Mar 08 12:04:42 byte systemd-vconsole-setup[65255]: Failed to import credentials, ignoring: No such file or directory
Mar 08 12:04:42 byte systemd[1]: Finished Setup Virtual Console.
Mar 08 12:05:12 byte systemd[1]: systemd-localed.service: Deactivated successfully.
Discovery

While doing some digging I found a github issue where someone was having the same problem. Something to note is that ariasuni the user who opened the issue stated that they were also running Arch Linux.

I attempted to reproduce the problem they were having in 2017, and obtained the same erroneous result they were getting.

Expected result:

[root@byte user]# localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: fr-bepo
   X11 Layout: fr
  X11 Variant: bepo

Current result:

[root@byte user]# localectl set-keymap fr-bepo
[root@byte user]# localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: fr-bepo
   X11 Layout: (unset)

[root@byte user]# localectl set-x11-keymap fr "" bepo
[root@byte user]# localectl status
System Locale: LANG=en_AU.UTF-8
    VC Keymap: fr
   X11 Layout: fr
  X11 Variant: bepo

Unfortunately it seems that the issue was never resolved for ariasuni, but they did some investigation and had some relevent findings.

Localectl seems to depend on the existence of a particular filesystem directory for correct behavior. ariasuni found a reference to a hardcoded directory in localectl's source code, that is used when it attempts to find suitable keymap conversions, it can be seen here:

On Arch Linux the directory would be /usr/share/kbd/keymaps/xkb/

After perusing the source it looks like this directory is searched only when converting an X11 keymap to a virtual console keymap. Of note there is also a fallback mechanism that tries to find a 'legacy' keymap if a keymap cannot be found in /usr/share/kbd/keymaps/xkb/, but this process doesn't produce the desired effect as can be seen in my results above (dvp -> us-acentos).

Before sharing what I've found in regards to the xkb keymap directory I'll take a second to also mention that converting VC(virtual console) keymaps to X11 keymaps also fails. The reasons are beyond me. In this case I can't see anything easy to spot like a directory path, but it should be noted that someone by the name of yuwata responded to ariasuni's github issue, and despite localectl being able to successfully convert X11 keymaps to VC keymaps on their system, they could not get localectl to convert VC keymaps to X11 ones either. I want to tentatively suggest that this is purely an upstream issue with localectl but there does seem to be some tests included in the source code, which one would assume, would catch any failures (can we then assume that the tests were passing on the developer's systems?). That's all I can say about VC -> X11 keymap conversion.

If you missed it in the previous paragraph I stated that on yuwata's system localectl was able to convert X11 keymaps to VC keymaps. They were using Fedora. This fact may point to some distribution specific configuration, but at this point I didn't know what I was looking for so I did some more digging.

After a little time I worked out that a piece of software by the name of 'kbd' is responsible for installing keymaps in the /usr/share/kbd/keymaps/ directory, and this seems to be the case for at least a few popular linux distributions. Interestingly though, looking at the files installed by Arch Linux's kbd package build shows no mention of the directory or any files that may exist therein. Looking further upstream I did a search on github limited to the kbd repository sources to see if the directory was mentioned there, but to no avail. In fact this directory doesn't seem to be mentioned in any official project sources or documentation for any project in the general vicinity of keymap or keyboard handling systems.

Looking for hits on the major search engines did turn up some references to /usr/share/kbd/keymaps/xkb/. In some distro specific issue trackers there was scant mentions of a transition that occured involving kbd-legacy and the new xkb directory but nothing that went into much depth, with much of the content regarding the logistics of the change-over.

Then I found a file on an OpenSUSE package build repository called kbd.spec. It looks like the equivalent of an Arch Linux PKGBUILD and as part of it's script it creates an xkb directory and does some other conversion process with the keymaps provided alongside kbd, then puts the converted keymaps in the xkb directory. Fedora has one as well, and I'll share that one since that's the distro that yuwata above was running. There it is on line 144, and just above on line 140 there are all the keymaps provided by kbd being stuffed into a directory called legacy.

Looking at git blame for those lines shows that the change was introduced 10 years ago.

Also, here is a link to Fedora maintainers talking about the same problem that ariasuni was observing, but in 2013, four years earlier.

Now I don't know why Fedora chose to do this or if it even was the first to implement the change but searching for kbd.spec shows that at least a few distros implement the same changes on top of the base upstream kbd source build. I can only assume that the developers working on systemd were doing so on any one of these Linux distributions.

I knew very little about how these software components worked before I started digging, and I have to admit I still know little, to the point that I can't progress any further with my investigation, but I wonder, why did distros start modifying kbd builds in this way? What does it achieve? Has X11 -> VC keymap conversion been broken for the last 10 7 years on all distros that provide vanilla package builds? Has VC -> X11 keymap conversion been broken for everybody since it's inception?

I don't know if this is an upstream bug or a necessary user configuration that must be made, The Arch Linux and Gentoo wiki's don't mention anything in this vein either, so I'm putting what I've found out there in the hopes that a community effort can sort this out, perhaps someone knows something about this.

Last edited by dmho (2023-03-09 22:15:11)

Offline

#2 2023-03-09 21:49:54

dmho
Member
Registered: 2023-03-08
Posts: 6

Re: I suspect that localectl keymap conversion is broken for everyone

Ah, so there was some information in that post on the Fedora issue tracker, namely links to other issues where the intentions, and the conversion, are discussed. Also some of the Fedora maintainers are systemd developers too, which explains the dependency between localectl/localed and Fedora specific configurations.

There seems to have been issues with the original design when Fedora/systemd were attempting to smooth out the interaction between the two systems (VC keymaps provided by kbd, and the X11 keymaps). I think some thought that designing a correct solution may not be feasible/possible and so Fedora just made the call to convert X11 keymaps to kbd keymaps and provide them with kbd under the xkb/ directory, putting the keymaps originally provided with kbd into a legacy/ directory. The systemd developers chose to rely on these changes for localectl/localed to function appropriately

This means that if you have a vanilla build of kbd then setting keymaps with systemd provided utilities simply will not work. This includes Arch Linux, any distributions that aim to provide unmodified builds, and any source based distributions that do not implement these changes.

If localectl/localed is going to continue to rely on these keymaps then the only other solution will be to replicate the changes that Fedora made back in 2013. Should Arch Linux provide a kbd-xkb instead of a kbd? I don't know but it sort of goes against Arch Linux philosophy to have distribution specific patches/configurations over and above the choice of what packages should make up the base distribution. Since Arch chooses systemd then it should make sure that it's complete set of services are functional, but in this case there is no upstream to draw from. Just a weird version of kbd that exists on a Fedora package build server somewhere.

Last edited by dmho (2023-03-09 22:12:51)

Offline

#3 2023-03-10 10:58:03

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: I suspect that localectl keymap conversion is broken for everyone

VC keymap and X11 layout  are unset on the systems I use/help maintain so conversion of keymaps is never done on them.
Please  clarifiy in which usecases the broken conversion is  a problem.

Welcome to archlinux forums.


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#4 2023-03-10 11:40:37

dmho
Member
Registered: 2023-03-08
Posts: 6

Re: I suspect that localectl keymap conversion is broken for everyone

For both, the default is the ubiquitous US qwerty, if that is what you use then you won't have run into this issue because you won't have had a reason to change from the default.

For any one that wants to use localectl to change from the default it will be a problem.

Of course this doesn't stop someone from setting them via other means like directly editing /etc/vconsole.conf, etc.

- - - -

Editing to mention that in some cases it will work by accident, for instance if you are using a layout but not a specific variant (fr vs. fr-bepo for instance).

Last edited by dmho (2023-03-10 12:05:06)

Offline

#5 2023-03-10 11:46:48

dmho
Member
Registered: 2023-03-08
Posts: 6

Re: I suspect that localectl keymap conversion is broken for everyone

Lone_Wolf wrote:

Welcome to archlinux forums.

Thanks :)

Offline

#6 2023-03-10 13:39:23

seth
Member
Registered: 2012-09-03
Posts: 49,992

Re: I suspect that localectl keymap conversion is broken for everyone

For any one that wants to use localectl to change from the default it will be a problem.

Yes and no, you can set the maps independently ("--no-convert"), https://wiki.archlinux.org/title/Linux_ … figuration & https://wiki.archlinux.org/title/Xorg/K … _localectl

What does not work all that well is the "conversion" heuristics (ie. selecting a matching layout for the other system)
The wiki btw. references localectl "for convenience" what is a nice way of saying "if you want somebody else to screw your system for you" and does not suggest "proper control"

Offline

#7 2023-03-10 19:01:00

dmho
Member
Registered: 2023-03-08
Posts: 6

Re: I suspect that localectl keymap conversion is broken for everyone

seth wrote:

Yes, thanks. I was aware of that but it slipped my mind last night (long day at work).

seth wrote:

What does not work all that well is the "conversion" heuristics (ie. selecting a matching layout for the other system)
The wiki btw. references localectl "for convenience" what is a nice way of saying "if you want somebody else to screw your system for you" and does not suggest "proper control"

Sure, but the function it performs is well defined to the point that one should be able to predict what the outcome is, thereby achieving control, and the modifications to system configuration files is so limited in scope that it hardly qualifies as screwing around (imo of course) Hell, pacman does half a dozen things automatically after every system update and I wouldn't consider that screwing around. Pacman and systemd are both included in the base install so I would assume they are both equally important and trusted utilities. I'll admit what I just wrote is simply my thoughts and opinions, I'm not looking to start an argument.

I don't suggest that using localectl is a substitute for knowing how keymaps are managed though.

Last edited by dmho (2023-03-10 19:20:19)

Offline

#8 2023-03-10 19:53:53

seth
Member
Registered: 2012-09-03
Posts: 49,992

Re: I suspect that localectl keymap conversion is broken for everyone

that one should be able to predict what the outcome is

Yes "should" …

It's a heuristic by necessity, it does a lousy job at the fringes. That's all.

Pacman and systemd are both included in the base install so I would assume they are both equally important and trusted utilities

pacman is your package manager, "systemd" is a kitchen sink.
It provides init process, but there's a bunch of stuff under the systemd umbrella that I'd not touch with pliers - and localectl, as mostly useless it is, is by far not the worst of that pack.

Offline

#9 2023-03-10 20:35:03

dmho
Member
Registered: 2023-03-08
Posts: 6

Re: I suspect that localectl keymap conversion is broken for everyone

seth wrote:

It's a heuristic by necessity, it does a lousy job at the fringes. That's all.

Mostly in agreement here, although I don't believe wanting to use dvorak-programmer as a keymap is particularly fringe, I mean it has representation in both kbd and [insert package that provides x11 keymaps here], but yes it is most definitely a lousy heuristic that is used for finding supposed 'legacy'/VC keymaps.


Edited to remove off topic.

Last edited by dmho (2023-03-11 09:32:31)

Offline

Board footer

Powered by FluxBB