[CLOSED] Broken Unicode

seth · 2017-02-23 07:41:02

Ok, let's rule out urxvt and the shell.
Run.

xterm -e /bin/bash
urxvt -e /bin/bash
xterm -e /bin/zsh

Does any of those behave as expected?

Biboozz · 2017-02-24 15:44:07

$ export LANG=en_US.UTF-8
$ locale
LANG=en_US.UTF-8
LC_CTYPE=
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
$ echo $LANG
en_US.UTF-8

But my UTF8 probleme stay the same.

$ cat "$HOME/.config/locale.conf" 
cat: /home/thor/.config/locale.conf: No such file or directory
$ source /etc/profile.d/locale.sh
$ locale
LANG=en_US.UTF-8
LC_CTYPE=
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Biboozz · 2017-02-24 15:45:29

@seth YES! my zsh print the right utf8 chars ! but how do bring toses changes to my usual shell ?

EDIT2: But it look like it only work if it's launched from a shell if i run xterm from dmenue my utf8 is still proken but if i run

xterm

it work.

Last edited by Biboozz (2017-02-24 15:47:36)

seth · 2017-02-24 16:45:03

if i run xterm from dmenu

Shall we rule out dmenu? ;-)
Get your self a clean urxvt & xterm by adding

urxvt & xterm &

to your ~/.xinitrc (given dmenu, I assume you'll use some sort of session that invokes xinit ;-)

Biboozz · 2017-02-24 17:26:32

i do, i use i3 environement. But you solution look like a trash solution is there a cleaner way to achieve it ?

loqs · 2017-02-24 18:03:37

Biboozz wrote:

But you solution look like a trash solution is there a cleaner way to achieve it ?

seth wrote:

Shall we rule out dmenu? ;-)

I think you have misread seth proposing a test to see if the issue is caused by dmenu or not as a proposed solution.

seth · 2017-02-24 19:43:48

Indeed. And if those terminals have shells with the proper locale, let's see your dmenu_runfoobarstuff script as well as the way you invoke it.

(My suspicion is that you encountered "no locale support" segfaults , see eg. https://bbs.archlinux.org/viewtopic.php?id=219327 and exported something like LC_ALL=C to work around that - which then accidentally does not only apply to dmenu but the executing shell)

Biboozz · 2017-02-25 14:19:55

$ cat /usr/bin/dmenu_run 
#!/bin/sh
dmenu_path | dmenu "$@" | ${SHELL:-"/bin/sh"} &

EDIT: As you said i added

urxvt & xterm &

my unicode is fixed, evend if i run urxvt from dmenu.

EDIT2: It even work after colsing X11 session, removing the line and running startx.

EDIT3: I think i'm running startx before initializing my utf8 because if i restart my X11 session (CTRL+E and startx) my utf8 is fixed

Here is my .zprofile in witch i run startx :

cat .zprofile 
if [ -z "$SSH_AUTH_SOCK" ]; then
  eval `ssh-agent -s`
  ssh-add
fi

startx

Last edited by Biboozz (2017-02-25 15:31:53)

seth · 2017-02-25 15:29:30

.zprofile is sourced before any zshrc but /etc/zsh/zprofile should have sourced /etc/profile first which should have setup your locales from /etc/locale.conf

I suggest to strace a login shell and what what gets opened when ...

Biboozz · 2017-02-25 15:32:39

seth wrote:

.I suggest to strace a login shell and what what gets opened when ...

How to do that ?

seth · 2017-02-25 15:50:13

man zsh
man strace
man grep

you'll have to redirect stdout into stderr for strace (google how to do that)

If you intend to run a custom zsh/xinit/autostart/whatever setup, you *really* should be able to do such things anyway.

Biboozz · 2017-02-25 16:10:22

I know strate and redicrection but my question is which file shoud i strace.

seth · 2017-02-25 16:26:24

You don't strace files, but processes ... now "zsh --help | grep login".
Does that answer your question?

Biboozz · 2017-02-25 17:56:16

Here is the output of the following command :

sudo strace zsh -l &> strace.log

http://filebin.ca/3DdOCj4i63yS/strace.log

Sorry i'm not good enought with linux.

Last edited by Biboozz (2017-02-25 17:56:38)

seth · 2017-02-25 19:53:33

Try

strace zsh -l 2>&1 | grep -E '(open|stat)' | grep -E '(zprofile|zshrc|locale)'

And notably don't sudo strace, that doesn't work.

Biboozz · 2017-02-25 21:31:35

$ strace zsh -l 2>&1 | grep -E '(open|stat)' | grep -E '(zprofile|zshrc|locale)'
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
stat("/etc/zsh/zprofile.zwc", 0x7ffc07758430) = -1 ENOENT (No such file or directory)
stat("/etc/zsh/zprofile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
open("/etc/zsh/zprofile", O_RDONLY|O_NOCTTY) = 3
^C
stat("/home/thor/.zprofile.zwc", 0x7ffc07758430) = -1 ENOENT (No such file or directory)
stat("/home/thor/.zprofile", {st_mode=S_IFREG|0644, st_size=76, ...}) = 0
open("/home/thor/.zprofile", O_RDONLY|O_NOCTTY) = 3
stat("/etc/zsh/zshrc.zwc", 0x7ffc07758430) = -1 ENOENT (No such file or directory)
stat("/etc/zsh/zshrc", 0x7ffc077584c0)  = -1 ENOENT (No such file or directory)
open("/etc/zsh/zshrc", O_RDONLY|O_NOCTTY) = -1 ENOENT (No such file or directory)
stat("/home/thor/.zshrc.zwc", 0x7ffc07758430) = -1 ENOENT (No such file or directory)
stat("/home/thor/.zshrc", {st_mode=S_IFREG|0644, st_size=2536, ...}) = 0
open("/home/thor/.zshrc", O_RDONLY|O_NOCTTY) = 3
lstat("/usr/share/zsh/site-functions/_localectl", {st_mode=S_IFREG|0644, st_size=3548, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Base/_comp_locale", {st_mode=S_IFREG|0644, st_size=572, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Unix/_locale", {st_mode=S_IFREG|0644, st_size=2870, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Unix/_locales", {st_mode=S_IFREG|0644, st_size=349, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Unix/_localedef", {st_mode=S_IFREG|0644, st_size=3091, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/X/_x_locale", {st_mode=S_IFREG|0644, st_size=120, ...}) = 0

seth · 2017-02-25 21:58:49

Well, /etc/profile.d/locale.sh isn't run.
Either because it doesn't exist, or /etc/profile isn't sourced or /etc/profile does not source /etc/profile.d/*

stat /etc/profile.d/locale.sh
pacman -Qkk zsh filesystem
strace zsh -l 2>&1 | grep -E '(open|stat)' | grep -E '(profile|locale)'

Biboozz · 2017-02-27 08:16:36

& stat /etc/profile.d/locale.sh
  File: /etc/profile.d/locale.sh
  Size: 940       	Blocks: 8          IO Block: 4096   regular file
Device: 804h/2052d	Inode: 656612      Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2017-01-31 16:07:05.000000000 +0000
Modify: 2016-12-05 23:43:55.000000000 +0000
Change: 2017-01-31 16:07:05.923646676 +0000
 Birth: -
$ sudo pacman -Qkk zsh filesystem
warning: zsh: /etc (Permissions mismatch)
zsh: 1077 total files, 1 altered file
warning: filesystem: /etc (Permissions mismatch)
backup file: filesystem: /etc/fstab (Modification time mismatch)
backup file: filesystem: /etc/fstab (Size mismatch)
backup file: filesystem: /etc/group (Modification time mismatch)
backup file: filesystem: /etc/group (Size mismatch)
backup file: filesystem: /etc/gshadow (Modification time mismatch)
backup file: filesystem: /etc/gshadow (Size mismatch)
warning: filesystem: /etc/mtab (Symlink path mismatch)
warning: filesystem: /etc/mtab (Modification time mismatch)
backup file: filesystem: /etc/passwd (Modification time mismatch)
backup file: filesystem: /etc/passwd (Size mismatch)
backup file: filesystem: /etc/profile (Modification time mismatch)
backup file: filesystem: /etc/profile (Size mismatch)
backup file: filesystem: /etc/resolv.conf (Modification time mismatch)
backup file: filesystem: /etc/resolv.conf (Size mismatch)
backup file: filesystem: /etc/shadow (Modification time mismatch)
backup file: filesystem: /etc/shadow (Size mismatch)
backup file: filesystem: /etc/shells (Modification time mismatch)
backup file: filesystem: /etc/shells (Size mismatch)
filesystem: 88 total files, 2 altered files
$ strace zsh -l 2>&1 | grep -E '(open|stat)' | grep -E '(profile|locale)'
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
stat("/etc/zsh/zprofile.zwc", 0x7ffd2394e890) = -1 ENOENT (No such file or directory)
stat("/etc/zsh/zprofile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
open("/etc/zsh/zprofile", O_RDONLY|O_NOCTTY) = 3
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
stat("/etc/profile.zwc", 0x7ffd2394b200) = -1 ENOENT (No such file or directory)
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
open("/etc/profile", O_RDONLY|O_NOCTTY) = 3
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
stat("/etc/profile.zwc", 0x7ffd23947b70) = -1 ENOENT (No such file or directory)
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
open("/etc/profile", O_RDONLY|O_NOCTTY) = 3

...

open("/etc/profile", O_RDONLY|O_NOCTTY) = 3
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
stat("/etc/profile.zwc", 0x7ffece5382b0) = -1 ENOENT (No such file or directory)
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
open("/etc/profile", O_RDONLY|O_NOCTTY) = 3
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
stat("/etc/profile.zwc", 0x7ffece534c20) = -1 ENOENT (No such file or directory)
stat("/etc/profile", {st_mode=S_IFREG|0644, st_size=36, ...}) = 0
open("/etc/profile", O_RDONLY|O_NOCTTY) = 3
^C
stat("/home/thor/.zprofile.zwc", 0x7ffecebda6d0) = -1 ENOENT (No such file or directory)
stat("/home/thor/.zprofile", {st_mode=S_IFREG|0644, st_size=76, ...}) = 0
open("/home/thor/.zprofile", O_RDONLY|O_NOCTTY) = 3
lstat("/usr/share/zsh/site-functions/_localectl", {st_mode=S_IFREG|0644, st_size=3548, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Base/_comp_locale", {st_mode=S_IFREG|0644, st_size=572, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Unix/_locale", {st_mode=S_IFREG|0644, st_size=2870, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Unix/_locales", {st_mode=S_IFREG|0644, st_size=349, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/Unix/_localedef", {st_mode=S_IFREG|0644, st_size=3091, ...}) = 0
lstat("/usr/share/zsh/functions/Completion/X/_x_locale", {st_mode=S_IFREG|0644, st_size=120, ...}) = 0

seth · 2017-02-27 08:25:37

/etc (Permissions mismatch)

likely unrelated, but "stat /etc"?

backup file: filesystem: /etc/profile (Modification time mismatch)
backup file: filesystem: /etc/profile (Size mismatch)

You're not supposed to edit /etc/profile since about a decade or so.

diff -pru /etc/profile /etc/profile.pacnew

If that yells an error (missing pacnew) simply paste /etc/profile (in code tags)

Biboozz · 2017-02-27 08:50:21

$ stat /etc
  File: /etc
  Size: 4096      	Blocks: 8          IO Block: 4096   directory
Device: 804h/2052d	Inode: 655361      Links: 64
Access: (0757/drwxr-xrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-12-11 03:07:34.000000000 +0000
Modify: 2017-02-26 04:49:05.293323618 +0000
Change: 2017-02-26 04:49:05.293323618 +0000
 Birth: -
$ diff -pru /etc/profile /etc/profile.pacnew
diff: /etc/profile.pacnew: No such file or directory
$ cat /etc/profile
emulate sh -c 'source /etc/profile'

EDIT: I once added Write right to my /etc because of a permision problem, was it a bad idea ?

Last edited by Biboozz (2017-02-27 08:54:59)

seth · 2017-02-27 09:19:07

Yes, very bad idea.

"emulate sh -c 'source /etc/profile'" is the typical content of /etc/zsh/zprofile and in /etc/profile causes a recursion.
Get rid of that and replace it with the version of the filesystem packake (you can simply download the package, extract it and copy the file there; root:root, 644)

Biboozz · 2017-03-01 08:23:07

So i should comment the line "emulate sh -s 'source /etc/profile'" in my /etc/profile ? And what mean "root:root, 644" ? And how to know my filesystem package ?

seth · 2017-03-01 08:40:10

No, you should replace that file with the version from https://www.archlinux.org/packages/core … /download/
"root:root, 644" refers to ownership and permissions of the file... welcome to a posix system.

BUT, before you do that: Are you *really* using archlinux or are you trying to get some better help for an manjaro/crunchbang/whatever installation?
You got to be honest on this, because if you're using some other distribution (even some arch deviation like crunchbang etc.) you might make things worse this way.
If you are, just replace the file or simply delete it and run "pacman -S filesystem" to update the filesystem package and get the proper file this way.

And "sudo chmod 755 /etc". Having /etc world-writable is quite a security breach.

Biboozz · 2017-03-02 08:42:35

Yes i really use Archlinux, installed it with the archboot intaller.

$ sudo pacman -S filesystem 
[sudo] password for thor: 
warning: filesystem-2016.12-2 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) filesystem-2016.12-2

Total Installed Size:  0.01 MiB
Net Upgrade Size:      0.00 MiB

:: Proceed with installation? [Y/n] 
(1/1) checking keys in keyring                                                                                     [#####################################################################] 100%
(1/1) checking package integrity                                                                                   [#####################################################################] 100%
(1/1) loading package files                                                                                        [#####################################################################] 100%
(1/1) checking for file conflicts                                                                                  [#####################################################################] 100%
(1/1) checking available disk space                                                                                [#####################################################################] 100%
:: Processing package changes...
(1/1) reinstalling filesystem                                                                                      [#####################################################################] 100%
warning: directory permissions differ on /boot/
filesystem: 700  package: 755
warning: directory permissions differ on /etc/
filesystem: 757  package: 755
:: Running post-transaction hooks...
(1/1) Arming ConditionNeedsUpdate...

it deosn't look like something was updated.

otherwise i used the following commands :

$ cp /etc/profile ~/profile.backup
$  sudo mv etc/profile /etc/profile
$ sudo chmod 755 /etc #Why do we give execution right on /etc ?

EDIT: AAAAAAAAAAAND nothing changed...

Last edited by Biboozz (2017-03-02 08:44:08)

seth · 2017-03-02 08:52:22

a) what does /etc/profile look like now
b) "Why do we give execution right on /etc" - how long exactly are you using a POSIX system? You cannot enter a directory w/o the x bit set ...

Arch Linux

#26 2017-02-23 07:41:02

Re: [CLOSED] Broken Unicode

#27 2017-02-24 15:44:07

Re: [CLOSED] Broken Unicode

#28 2017-02-24 15:45:29

Re: [CLOSED] Broken Unicode

#29 2017-02-24 16:45:03

Re: [CLOSED] Broken Unicode

#30 2017-02-24 17:26:32

Re: [CLOSED] Broken Unicode

#31 2017-02-24 18:03:37

Re: [CLOSED] Broken Unicode

#32 2017-02-24 19:43:48

Re: [CLOSED] Broken Unicode

#33 2017-02-25 14:19:55

Re: [CLOSED] Broken Unicode

#34 2017-02-25 15:29:30

Re: [CLOSED] Broken Unicode

#35 2017-02-25 15:32:39

Re: [CLOSED] Broken Unicode

#36 2017-02-25 15:50:13

Re: [CLOSED] Broken Unicode

#37 2017-02-25 16:10:22

Re: [CLOSED] Broken Unicode

#38 2017-02-25 16:26:24

Re: [CLOSED] Broken Unicode

#39 2017-02-25 17:56:16

Re: [CLOSED] Broken Unicode

#40 2017-02-25 19:53:33

Re: [CLOSED] Broken Unicode

#41 2017-02-25 21:31:35

Re: [CLOSED] Broken Unicode

#42 2017-02-25 21:58:49

Re: [CLOSED] Broken Unicode

#43 2017-02-27 08:16:36

Re: [CLOSED] Broken Unicode

#44 2017-02-27 08:25:37

Re: [CLOSED] Broken Unicode

#45 2017-02-27 08:50:21

Re: [CLOSED] Broken Unicode

#46 2017-02-27 09:19:07

Re: [CLOSED] Broken Unicode

#47 2017-03-01 08:23:07

Re: [CLOSED] Broken Unicode

#48 2017-03-01 08:40:10

Re: [CLOSED] Broken Unicode

#49 2017-03-02 08:42:35

Re: [CLOSED] Broken Unicode

#50 2017-03-02 08:52:22

Re: [CLOSED] Broken Unicode

Board footer