Locale encoding goes to lowercase.

kiljacken · 2017-05-29 16:49:44

Hello!

I'm having a weird issue with setting my locale, that cropped up when trying a new piece of software.

In /etc/locale.gen I have both en_DK.UTF-8 and en_US.UTF-8 enabled and built, and thusly I set LANG=en_DK.UTF-8 in /etc/locale.conf.

locale-ctl correctly returns en_DK.UTF-8, but locale returns en_DK.utf8 (notice the lowercase and lack of dash).

Has anybody seen something like this before, and can point me in the right direction?

Cheers.

seth · 2017-05-29 19:01:21

"locale" should print en_DK.UTF-8, but "locale -a" should print "en_DK.utf8" - that is normal expectable.

In case of worries, post the outputs of
locale
locale -a
localectl
cat /etc/locale.conf

kiljacken · 2017-05-29 19:15:46

Okay. Does the following look right then?

kiljacken@exheater ~> locale
LANG=en_DK.utf8
LC_CTYPE="en_DK.utf8"
LC_NUMERIC="en_DK.utf8"
LC_TIME="en_DK.utf8"
LC_COLLATE="en_DK.utf8"
LC_MONETARY="en_DK.utf8"
LC_MESSAGES="en_DK.utf8"
LC_PAPER="en_DK.utf8"
LC_NAME="en_DK.utf8"
LC_ADDRESS="en_DK.utf8"
LC_TELEPHONE="en_DK.utf8"
LC_MEASUREMENT="en_DK.utf8"
LC_IDENTIFICATION="en_DK.utf8"
LC_ALL=

kiljacken@exheater ~> locale -a
C
en_DK.utf8
en_US.utf8
POSIX

kiljacken@exheater ~> localectl 
   System Locale: LANG=en_DK.UTF-8
       VC Keymap: dk-latin1
      X11 Layout: dk
       X11 Model: pc105
     X11 Variant: nodeadkeys

kiljacken@exheater ~> cat /etc/locale.conf 
LANG=en_DK.UTF-8

What about the $LANG environment variable? I assume it should be the one set in locale.conf, but on my system:

kiljacken@exheater ~> echo $LANG
en_DK.utf8

seth · 2017-05-29 19:41:27

No - "printenv" output?
What kind of session is this and what's the rep. output when logging into a console (ctrl+alt+f2) and notably when logging in as root there?

Ropid · 2017-05-29 21:01:45

I see the same behavior here. I use lightdm and i3. In i3 I use gnome-terminal.

When I log into a console, this does not happen. At a bash prompt there, I see this:

$ echo $LANG
en_IE.UTF-8

And in X at a bash prompt run through gnome-terminal, I see this:

$ echo $LANG
en_IE.utf8

I also checked xterm and urxvt started by i3 through a launcher tool, and those also get "utf8".

When I inspect the environment of different processes in htop by selecting them and hitting the "e" key, I see that the parent lightdm process is "LANG=en_IE.UTF-8", but its child i3 process is "LANG=en_IE.utf8". For programs I start through i3 (Firefox etc.), I see "utf8".

The gnome-terminal program is weird in that it makes a process "gnome-terminal-server" show up as a child of a "systemd --user" process. The "systemd --user" process has "UTF-8" in its variable, but gnome-terminal-server has "utf8".

I tried to find out if there's a script that sets this "LANG=...utf8", but found nothing. I scrolled through the output of these commands here:

sudo ack LANG /etc
rg LANG= /usr   # rg = "ripgrep" here because ack was very slow for /usr
rg --hidden LANG= ~   # needs --hidden or it won't look at .bashrc or .config/

The only script that might write to $LANG at login for me is a file /etc/profile.d/locale.sh that sets LANG=C if it is not defined. It also sources a file ~/.config/locale.sh if it exists, but I don't have a file like that.

Last edited by Ropid (2017-05-29 21:03:35)

seth · 2017-05-29 21:06:42

I see that the parent lightdm process is "LANG=en_IE.UTF-8", but its child i3 process is "LANG=en_IE.utf8".

I bet on lightdm. Try startx or xdm.

kiljacken · 2017-05-29 21:31:12

I run lightdm too, that could definitely be the cause. Back to startx, who needs GUIs anyways.

Ropid · 2017-05-29 22:31:02

I just checked, and it really seems it's lightdm. It doesn't happen when using startx. Trying to search about the problem online, I found people complaining about this. For some reason it's translating the name to what it's like in the output of "locale -a".

I decided to keep the weirdo lightdm and work around it by changing the utf8 to UTF-8 in ~/.xprofile like this:

export LANG="${LANG/utf8/UTF-8}"

This should hopefully never break something in the future if I forget about the line.

loqs · 2017-05-29 23:29:44

Possibly this code in lightdm? http://bazaar.launchpad.net/~lightdm-te … ild.c#L458

Ropid · 2017-05-30 00:19:11

loqs wrote:

Possibly this code in lightdm? http://bazaar.launchpad.net/~lightdm-te … ild.c#L458

I'm guessing this isn't it because that "locale_var_names[]" array has entries for things like LC_COLLATE, LC_TIME, etc., and those are all being kept like I've set them in /etc/locale.conf. Only LANG is getting changed.

seth · 2017-05-30 07:04:57

http://bazaar.launchpad.net/~lightdm-te … age.c#L180 looks fishy - lightdm_get_language tries LANG but returns something that's fished out of a "locale -a" subprocess (it really actually runs "locale -a". several times. please use xdm, one can configure it to be not ugly; "pacman -S xdm-archlinux" has a nice pattern on this. suboptimal color choice but a start ;-)

justasug · 2017-05-30 07:20:46

I might be missing something, but is the only problem you are having the inconsistency regarding the case? As in: .utf8 instead of UTF-8?
Because I've been having the same behaviour for years now and never experienced any bugs that I could connect with this. It's not really giving me any problems.

kiljacken · 2017-05-30 07:50:43

justasug wrote:

I might be missing something, but is the only problem you are having the inconsistency regarding the case? As in: .utf8 instead of UTF-8?
Because I've been having the same behaviour for years now and never experienced any bugs that I could connect with this. It's not really giving me any problems.

Well, I'm using an in-development cross-platform code editor, and apparently querying X's for an input method using the lowercase version of the locale doesn't work.
So the key issue lies there. (Or well, perhaps in the creator loads am IM from X, perhaps another api call could fix it)

seth · 2017-05-30 11:05:33

The IANA code is "UTF-8", everything is else "wrong".
However, putting a hyphen/dash in there caused a lot of trouble (because of its special meaning in all sorts of code syntax) so "utf8" became common (notably also on OtherOS™)

Therfore every code is doing "smart" in expecting any case insensitive variant of
* UTF-8
* utf8
* Utf 8
but the "legal" code is "UTF-8" and if a DM just changes that export on the fly (away btw. from an explicit user export ...) that DM simply has a bug.

---
Henceforce (and henceforth as well), 8-bit unicode shall be known as "UTF-8".
Wrongspelling shall be punishable by decapitation or the eternal usage of DOS.
So let it be written. So let it be done.
-- Seth, Dictator of the world

See how easy that is?

Arch Linux

#1 2017-05-29 16:49:44

Locale encoding goes to lowercase.

#2 2017-05-29 19:01:21

Re: Locale encoding goes to lowercase.

#3 2017-05-29 19:15:46

Re: Locale encoding goes to lowercase.

#4 2017-05-29 19:41:27

Re: Locale encoding goes to lowercase.

#5 2017-05-29 21:01:45

Re: Locale encoding goes to lowercase.

#6 2017-05-29 21:06:42

Re: Locale encoding goes to lowercase.

#7 2017-05-29 21:31:12

Re: Locale encoding goes to lowercase.

#8 2017-05-29 22:31:02

Re: Locale encoding goes to lowercase.

#9 2017-05-29 23:29:44

Re: Locale encoding goes to lowercase.

#10 2017-05-30 00:19:11

Re: Locale encoding goes to lowercase.

#11 2017-05-30 07:04:57

Re: Locale encoding goes to lowercase.

#12 2017-05-30 07:20:46

Re: Locale encoding goes to lowercase.

#13 2017-05-30 07:50:43

Re: Locale encoding goes to lowercase.

#14 2017-05-30 11:05:33

Re: Locale encoding goes to lowercase.

Board footer