[SOLVED]Locales: Random russian strings popping out

L1ghtmareI · 2017-05-08 14:39:42

At first only a few pygtk apps would display their interface in Russian, but now I created /etc/locale.conf to fix that exact problem and suddenly Russian pops out even with sudo and journalctl.

For some reason LANG reverted to en_US.US-ASCII even though it was specified in /etc/locale.conf, localectl set-locale doesn't work either.

locale -a
C
en_US
en_US.iso88591
en_US.utf8
ja_JP
ja_JP.eucjp
ja_JP.ujis
ja_JP.utf8
japanese
japanese.euc
POSIX
ru_RU
ru_RU.iso88595
ru_RU.koi8r
ru_RU.utf8
russian
zh_CN
zh_CN.gb18030
zh_CN.gb2312
zh_CN.gbk
zh_CN.utf8

locale
LANG=en_US.US-ASCII
LC_CTYPE=en_US.UTF-8
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE=en_US.UTF-8
LC_MONETARY=en_US.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_PAPER=en_US.UTF-8
LC_NAME=en_US.UTF-8
LC_ADDRESS=en_US.UTF-8
LC_TELEPHONE=en_US.UTF-8
LC_MEASUREMENT=en_US.UTF-8
LC_IDENTIFICATION=en_US.UTF-8
LC_ALL=

cat /etc/locale.conf                  
LANG="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"

Any advice appreciated.

Edit: and japanese too!

  -> Сжатие документации (man и info)...
==> パッケージの問題をチェック...
==> Создание пакета

Last edited by L1ghtmareI (2017-05-11 14:36:54)

Dheart · 2017-05-08 19:11:51

Looks like you have a lot of locales generated.
Look at your /etc/locale.gen, remove any locales you are not using and then run locale-gen.

L1ghtmareI · 2017-05-08 20:20:07

Is it bad to have a lot of locales generated? What does it mean to 'be using' a locale? I uncommented those for a reason.

I checked on the environment files described in the wiki, but none of them contain locale variables.

My KDE settings have languages prioritized in the order en>ru>ja.

Dheart · 2017-05-08 20:37:57

Nowadays you should mostly be using UTF-8. The purpose of the non-UTF8 locales is so that you have a default decoding of text files that you read, browser web pages (now obsolete unless using some obscure web 95 web sites) and save the files that you have with specific locales.

Locale-gen behaves kind'a weird with regards to what takes priority over what, I am not sure myself what the order should be.
My suggestion is for you to keep a one English, Russian and Japanese locale (the UTF-8 one, unless you REALLY need another one).

I am not sure what is causing your weird behaviour, but I'd bet it is one of the legacy non UTF-8 locales.

Hope that helps

Ropid · 2017-05-08 22:13:32

Where do those locales "japanese" and "russian" in the output of 'locale -a' come from?

EDIT:

I found out how that works: these names appear in the output of locale -a when the 1980s charmaps for those locales are available instead of just the UTF-8 ones. For example, when there's only ru_RU.UTF-8 available, there's no "russian". It shows up when ru_RU.ISO-8859-5 gets added.

Last edited by Ropid (2017-05-08 22:29:50)

L1ghtmareI · 2017-05-09 12:18:13

I regenerated the locales, leaving only the 4 UTF-8 ones (and C and POSIX), the issue still persists. Even after i removed every locale but english.

Last edited by L1ghtmareI (2017-05-09 12:20:56)

seth · 2017-05-09 20:08:44

locale
LANG=en_US.US-ASCII

cat /etc/locale.conf                  
LANG="en_US.UTF-8"

??

At first only a few pygtk apps would display their interface in Russian, but now I created /etc/locale.conf to fix that exact problem and suddenly Russian pops out even with sudo and journalctl.

So the problem is limited to a specific set of processes?
What VTE do you use? Does it also happen w/ xterm?
What desktop session is this (if any)?

L1ghtmareI · 2017-05-09 22:58:36

It was limited at first, after I created the /etc/locale.conf it spread to zsh too.

I use Konsole but the issue persists in urxvt as well as when using bash.

Not sure if i got the question right, but I use Plasma 5.9.5.

seth · 2017-05-10 07:05:29

zsh means "ls" (or other built in functions) or "when I run a python script from zsh"?

strace pacman 2>&1 | grep -i locale
stat /usr/lib/locale/locale-archive
stat /usr/share/locale/locale.alias
pacman -Qkk glibc # let's hope somebody can read the output ;-)

L1ghtmareI · 2017-05-10 15:39:43

Most of the commands i've used so far (e.g. sudo, pacman, stat) respond in Russian at this point, with an occasional Japanese string. Oddly, I can only control this behavior by setting LC_ALL - setting LANG has no effect. Qt programs seem to be unaffected.

strace pacman 2>&1 | grep -i locale
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
read(3, "# Locale name alias data base.\n#"..., 4096) = 2997
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/ru/LC_MESSAGES/libc.mo", O_RDONLY) = 3
open("/usr/share/locale/en_US/LC_MESSAGES/libgpg-error.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libgpg-error.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/ru/LC_MESSAGES/libgpg-error.mo", O_RDONLY) = 5
open("/usr/share/locale/en_US/LC_MESSAGES/pacman.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/pacman.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/ru/LC_MESSAGES/pacman.mo", O_RDONLY) = 3

LANG="ja_JP.UTF-8" pacman -Qkk glibc
резервная копия: glibc: /etc/locale.gen (Не совпадает время изменения)
резервная копия: glibc: /etc/locale.gen (Не совпадает размер)
glibc: всего 1553 файла, 0 измененных файлов

LC_ALL=C pacman -Qkk glibc
backup file: glibc: /etc/locale.gen (Modification time mismatch)
backup file: glibc: /etc/locale.gen (Size mismatch)
glibc: 1553 total files, 0 altered files

LC_ALL=C stat /usr/lib/locale/locale-archive
  File: /usr/lib/locale/locale-archive
  Size: 1669168         Blocks: 3112       IO Block: 4096   regular file
Device: 801h/2049d      Inode: 33555121    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2017-05-10 18:29:07.953325459 +0300
Modify: 2017-05-09 15:19:34.812046933 +0300
Change: 2017-05-09 15:19:34.812046933 +0300
 Birth: -

LC_ALL=C stat /usr/lib/locale/locale.alias  
stat: cannot stat '/usr/lib/locale/locale.alias': No such file or directory

Last edited by L1ghtmareI (2017-05-10 15:40:39)

seth · 2017-05-10 18:34:21

stat: cannot stat '/usr/lib/locale/locale.alias': No such file or directory

Please post your entire /var/log/pacman.log (either use code tags or a pastebin service, it's a hell lot of text)

L1ghtmareI · 2017-05-10 19:42:57

full pacman.log

there's an awful lot of these strings there during the last upgrade:

[2017-05-08 17:57] [ALPM] running 'update-appstream-index.hook'...
[2017-05-08 17:57] [ALPM-SCRIPTLET] 
[2017-05-08 17:57] [ALPM-SCRIPTLET] (appstreamcli:6331): GLib-CRITICAL **: g_variant_get_type: assertion 'value != NULL' failed
[2017-05-08 17:57] [ALPM-SCRIPTLET] 
[2017-05-08 17:57] [ALPM-SCRIPTLET] (appstreamcli:6331): GLib-CRITICAL **: g_variant_type_is_subtype_of: assertion 'g_variant_type_check (type)' failed

and so on thousands of times

The first and only russian string appears after these.

I had to force upgrade glibc before the upgrade for a reason.

Last edited by L1ghtmareI (2017-05-10 19:43:19)

seth · 2017-05-10 19:57:45

I'm more concerned about the absence of that file that is provided by the glibc package, yet "pacman -Qkk" doesn't complain about its absence, so the package itself should be corrupted and the forceful installation smells suspicious enough.

=> "for a reason" - that is which exactly?

Also and since you're not using the stock mirrorlist (no problem, everyone uses reflector ;-) please post your /etc/pacman.d/mirrorlist, resp.

$ md5sum glibc-2.25-1-x86_64.pkg.tar.xz
46f39da55b90f814defad24d920d37e1  glibc-2.25-1-x86_64.pkg.tar.xz

loqs · 2017-05-10 20:25:49

@seth another possible explanation for the pacman -Qkk output is a pacman.conf NoExtract directive covering the file.
Edit:
grammar missing "the"

Last edited by loqs (2017-05-10 20:26:41)

seth · 2017-05-10 20:30:11

thenwhyhaveanintegritycheckwhenyoucanoverrideintegritygnagnagna... ;-)

@L1ghtmareI
well, check your /etc/pacman.conf on whether you explicitly broke it ... :-P

L1ghtmareI · 2017-05-10 21:26:02

 # With:       reflector --sort rate -a 1 -f 20 -l 20 --save /etc/pacman.d/mirrorlist
# When:       2017-05-08 14:29:21 UTC
# From:       https://www.archlinux.org/mirrors/status/json/
# Retrieved:  2017-05-08 14:29:18 UTC
# Last Check: 2017-05-08 14:16:14 UTC

Server = http://mirror.f4st.host/archlinux/$repo/os/$arch
Server = http://ftp.sh.cvut.cz/arch/$repo/os/$arch
Server = https://ftp.sh.cvut.cz/arch/$repo/os/$arch
Server = http://mirror.vfn-nrw.de/archlinux/$repo/os/$arch
Server = https://mirror.f4st.host/archlinux/$repo/os/$arch
Server = http://archlinux.dynamict.se/$repo/os/$arch
Server = https://archlinux.dynamict.se/$repo/os/$arch
Server = http://archlinux.prometeolibero.eu/archlinux/$repo/os/$arch
Server = https://mirror.vfn-nrw.de/archlinux/$repo/os/$arch
Server = http://mirror.csclub.uwaterloo.ca/archlinux/$repo/os/$arch
Server = rsync://ftp.sh.cvut.cz/arch/$repo/os/$arch
Server = rsync://mirror.vfn-nrw.de/archlinux/$repo/os/$arch
Server = rsync://mirror.f4st.host/archlinux/$repo/os/$arch
Server = rsync://rsync.osbeck.com/archlinux/$repo/os/$arch
Server = https://mirror.csclub.uwaterloo.ca/archlinux/$repo/os/$arch
Server = rsync://archlinux.dynamict.se/archlinux/$repo/os/$arch
Server = rsync://mirror.csclub.uwaterloo.ca/archlinux/$repo/os/$arch
Server = https://mirror.osbeck.com/archlinux/$repo/os/$arch
Server = http://archmirror.tomforb.es/$repo/os/$arch
Server = https://archmirror.tomforb.es/$repo/os/$arch

md5sum glibc-2.25-1-x86_64.pkg.tar.xz 
46f39da55b90f814defad24d920d37e1  glibc-2.25-1-x86_64.pkg.tar.xz

Again, for some reason the only change to my pacman.conf besides adding repos is adding glibc to HoldPkg, although this should't be relevant AFAIU.

I must've tinkered with glibc while trying to solve some other issue, i don't remember which though, it's not in the news.

L1ghtmareI · 2017-05-10 21:32:31

I reinstalled glibc and now it's locale-archive that's missing.

LC_ALL=C  stat /usr/share/locale/locale.alias
  File: /usr/share/locale/locale.alias
  Size: 2997            Blocks: 8          IO Block: 4096   regular file
Device: 801h/2049d      Inode: 34647295    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2017-05-11 00:27:06.245024956 +0300
Modify: 2017-03-06 17:06:05.000000000 +0300
Change: 2017-05-11 00:27:06.128358292 +0300
 Birth: -
LC_ALL=C  stat /usr/share/locale/locale-archive
stat: cannot stat '/usr/share/locale/locale-archive': No such file or directory

seth · 2017-05-11 06:23:47

Because it's more like "stat /usr/lib/locale/locale-archive" ;-) and the file is generated locally - no package owns it.

More relevant is whether the problem is gone with existing /usr/share/locale/locale.alias and the still interesting question is why it was undetectedly absent itfp...

L1ghtmareI · 2017-05-11 12:27:20

The problem is still present.

seth · 2017-05-11 13:04:47

Ok, red herring - you initially stat'ed /usr/lib/locale/locale.alias instead of /usr/share/locale/locale.alias - no wonder it didn't work and the package is fine - so that's not the cause of the issue.
Nevertheles, the present languages are tested until the first generated translation is found (what will usually be russian or japanese and likely just depends on whether a translation of the lib is available in either language) - the question is "why" - "en_US.UTF-8" should test en_US.UTF-8, en_US.utf8, en_US, en.utf8, en and then resort to C - not "ru". I cannot remotely imagine how that matching list could occur...

Does exporting "LC_ALL=C LANG=C LC_MESSAGES=C" change anything (strace something) and also try running

export LC_ALL=C
export LANG=C
export LC_MESSAGES=C
xterm &
tr '\0' '\n' < /proc/${!}/environ

and check the language related variables in that process.

L1ghtmareI · 2017-05-11 13:50:36

ps aux | grep bleach                     
    3553  0.1  0.2 400376 48756 pts/1    S+   16:40   0:00 python2 /usr/bin/bleachbit
    3597  0.0  0.0   9024   852 pts/3    S+   16:42   0:00 grep bleach
export LANG=C                            
export LC_ALL=C                          
export LC_MESSAGES=C                     
xterm & tr '\0' '\n' < /proc/3553/environ
[1] 3599
LC_CTYPE=en_US.UTF-8
LC_TIME=en_US.UTF-8
SHELL=/usr/bin/zsh
SHLVL=2
LC_ADDRESS=en_US.UTF-8
LC_COLLATE=en_US.UTF-8
LANG=C
LANGUAGE=en_US:ru:ja
LC_MONETARY=en_US.UTF-8
LC_MESSAGES=C
LC_NUMERIC=en_US.UTF-8
LC_NAME=en_US.UTF-8
LC_IDENTIFICATION=en_US.UTF-8
LC_TELEPHONE=en_US.UTF-8
LC_PAPER=en_US.UTF-8
LC_MEASUREMENT=en_US.UTF-8
LC_ALL=C

The app in question has its interface in russian despite being launched from the same shell (with exported vars). Sudo has changed its output to english though.

Locale variables as seen from the app:

locale_dir = /usr/share/locale/

locale.getdefaultlocale = (None, None)

Last edited by L1ghtmareI (2017-05-11 13:52:55)

seth · 2017-05-11 13:57:38

There you go: LANGUAGE=en_US:ru:ja - get rid of that, wherever it comes from.
In doubt unset it in some session environment script eg. ~/.config/plasma-workspace/env/zz_finalfixes.sh

And get rid of bleachbit.
https://blog.martin-graesslin.com/blog/ … bleachbit/
https://lwn.net/Articles/313679/

L1ghtmareI · 2017-05-11 14:22:24

Ok, that was an easy one: I had those 3 languages selected in that order in Plasma System Settings. That fixes everything.

Isn't it meant to be this way, though? Or do i have to only leave en_US if i want to use the system in English?

Anyways, AFAIU, some files related to the en locale were missing. They still are:

strace pacman 2>&1 | grep -i locale
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 3
read(3, "# Locale name alias data base.\n#"..., 4096) = 2997
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libgpg-error.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libgpg-error.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/pacman.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/pacman.mo", O_RDONLY) = -1 ENOENT (No such file or directory)

I'm gonna regenerate other locales now and report back.

Confirmed only setting LANGUAGE that way breaks it.

Last edited by L1ghtmareI (2017-05-11 14:28:46)

seth · 2017-05-11 14:32:51

This creates a preference table that says "use en_US, if unavailable use russian, if unavailable as well, use japanese (and ultimately use C)", what
a) doesn't work because english translations are usually not generated
b) doesn't make sense wrt. messages, since ("usually") en_US euqals C and and thus *always* available.

A typical setup would eg. look like "nl:de:en" (prefer netherlandish, then german, ultimately english) or "en_AU:en_GB:en_US" - for a US english system, you can just keep LANGUAGE unset

L1ghtmareI · 2017-05-11 14:35:39

Ok got it, so the files missing are perfectly normal? Also doesn't explain why it only broke now when it's always been set this way.

Thank you so much for your help.

Arch Linux

#1 2017-05-08 14:39:42

[SOLVED]Locales: Random russian strings popping out

#2 2017-05-08 19:11:51

Re: [SOLVED]Locales: Random russian strings popping out

#3 2017-05-08 20:20:07

Re: [SOLVED]Locales: Random russian strings popping out

#4 2017-05-08 20:37:57

Re: [SOLVED]Locales: Random russian strings popping out

#5 2017-05-08 22:13:32

Re: [SOLVED]Locales: Random russian strings popping out

#6 2017-05-09 12:18:13

Re: [SOLVED]Locales: Random russian strings popping out

#7 2017-05-09 20:08:44

Re: [SOLVED]Locales: Random russian strings popping out

#8 2017-05-09 22:58:36

Re: [SOLVED]Locales: Random russian strings popping out

#9 2017-05-10 07:05:29

Re: [SOLVED]Locales: Random russian strings popping out

#10 2017-05-10 15:39:43

Re: [SOLVED]Locales: Random russian strings popping out

#11 2017-05-10 18:34:21

Re: [SOLVED]Locales: Random russian strings popping out

#12 2017-05-10 19:42:57

Re: [SOLVED]Locales: Random russian strings popping out

#13 2017-05-10 19:57:45

Re: [SOLVED]Locales: Random russian strings popping out

#14 2017-05-10 20:25:49

Re: [SOLVED]Locales: Random russian strings popping out

#15 2017-05-10 20:30:11

Re: [SOLVED]Locales: Random russian strings popping out

#16 2017-05-10 21:26:02

Re: [SOLVED]Locales: Random russian strings popping out

#17 2017-05-10 21:32:31

Re: [SOLVED]Locales: Random russian strings popping out

#18 2017-05-11 06:23:47

Re: [SOLVED]Locales: Random russian strings popping out

#19 2017-05-11 12:27:20

Re: [SOLVED]Locales: Random russian strings popping out

#20 2017-05-11 13:04:47

Re: [SOLVED]Locales: Random russian strings popping out

#21 2017-05-11 13:50:36

Re: [SOLVED]Locales: Random russian strings popping out

#22 2017-05-11 13:57:38

Re: [SOLVED]Locales: Random russian strings popping out

#23 2017-05-11 14:22:24

Re: [SOLVED]Locales: Random russian strings popping out

#24 2017-05-11 14:32:51

Re: [SOLVED]Locales: Random russian strings popping out

#25 2017-05-11 14:35:39

Re: [SOLVED]Locales: Random russian strings popping out

Board footer