[SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

tenplaza9 · 2017-09-28 15:59:45

Hi I have been using rxvt-unicode for a while now and am quite happy. However some characters can't be seen (they appear as question marks) when I paste these from webpages or when I use "mpc status" to print current playback information from mpd, for example: محمد منير or ሙስሊን (these are arabic and amharic scripts respectively). In text programs such as ranger and ncmpcpp these appear blank. However when I append the output to a file and open it in gedit they appear as they are. Also i3bar is able to display the characters (I'm using the same font in i3bar with the following): in .config/i3/config.

bar {
	status_command i3blocks # -c ~/.config/i3/i3blocks.conf
	strip_workspace_numbers yes
	font pango:Noto Sans Mono, FontAwesome 9
	colors {
		background $color0
		statusline $color12
#class				border		background	text
focused_workspace               $color0		$color0		$color15
active_workspace		$color0		$color0		$color12
inactive_workspace              $color0		$color0		$color12
urgent_workspace		$color0		$color0		$color1
}
}

Also, Characters that I have seen being printed in the terminal such as ö (when using "cat" on arbitrary files such as .mobi format) produce the same question mark symbol when pasting that character from a web page.

Coming to locales
I uncommented the "en_US.UTF-8 UTF-8". during install and ran locale-gen. I got the following message:

Generating locales...
  en_US.UTF-8... done
Generation complete.

However when I run "locale", all fields are empty!

LANG=
LC_CTYPE=
LC_NUMERIC=
LC_TIME=
LC_COLLATE=
LC_MONETARY=
LC_MESSAGES=
LC_PAPER=
LC_NAME=
LC_ADDRESS=
LC_TELEPHONE=
LC_MEASUREMENT=
LC_IDENTIFICATION=
LC_ALL=

This is the relevant bit of my ~/.Xresources.

URxvt.font: xft:Inconsolata:hinting=true:size=12,\
            xft:Noto Sans Mono,\
            xft:IPAGothic,\
            xft:Symbola,\
            xft:FontAwesome:pixelsize=12,\
            -*-unifont-*-*-*-*-*-*-*-*-*-*-*-*

I read the rxvt-unicode doucumentation on fonts and searched the forums for a while, and messed around with the font for hours but I am still highly confused as to what is going on. Any help would be appreciated.

Edit: I am quite certain the problem is to do with my locales: Reason being, this problem with unicode characters extends to other programs, for example adding torrent file with unicode characters as fileneames produces errors that say The file ***.torrent could not be added (where unicode characters are replaced with U+FFFD). Also the same problem persists in Konsole, which I installed just to test, unless I specifically set a unicode font under the option View>Set Encoding>Unicode>UTF-8 (though even then glyphs don't show in text programs like ncmpcpp and ranger).

My current /etc/locale.conf for reference:

LANG=en_US.UTF-8

I hadn't set this during install, but did this quite recently when I was first trying to troubleshoot this issue.
I don't know if this is relevant, but for some reason I see a lowercase string "en_US.utf8" (for utf8) when I run "locale -a", or "localectl list-locales". My output of "locale-a" for reference:

C
POSIX
en_US.utf8

I have tried putting that exact string (en_US.utf8) in /etc/locales.conf (and then rebooting) but to no avail; my output for locale still remains empty. Thanks a lot for the help so far!

Last edited by tenplaza9 (2017-09-30 18:31:47)

Trilby · 2017-09-28 16:27:21

Have you tried urxvt with just the same two fonts as i3bar? If a character (even an unsuitable one) is found in Inconsolata, urxvt wouldn't check the other fonts. I really don't know if this is at all likely, but it's easy to test.

EDIT: but yes, something is wrong with your locale ... What is in your locale.conf?

Last edited by Trilby (2017-09-28 16:28:30)

escondida · 2017-09-28 16:31:16

tenplaza9 wrote:

Coming to locales
I uncommented the "en_US.UTF-8 UTF-8". during install and ran locale-gen. I got the following message:
Generating locales...
  en_US.UTF-8... done
Generation complete.
However when I run "locale", all fields are empty!
LANG=
[snip]

Let's handle this first. /etc/locale.gen doesn't actually *set* your locale, it just sets things up so that it's possible to use a given locale. You could have multiple options uncommented in /etc/locale.gen then run locale-gen, which would allow you to switch between locales.

To actually set your locale, you need to edit or create /etc/locale.conf. See the locale page on the Arch wiki or locale.conf(5) for the sorts of things you might put in there.

Hi I have been using rxvt-unicode [....] However some characters can't be seen (they appear as question marks) when I paste these from webpages or when I use "mpc status" to print current playback information from mpd, for example: محمد منير or ሙስሊን (these are arabic and amharic scripts respectively). In text programs such as ranger and ncmpcpp these appear blank. However when I append the output to a file and open it in gedit they appear as they are.
Also, Characters that I have seen being printed in the terminal such as ö (when using "cat" on arbitrary files such as .mobi format) produce the same question mark symbol when pasting that character from a web page.

I'll be honest, it's been a while since I looked at urxvt, so take this with a grain of salt. I seem to recall its font handling not being nearly as excellent as its documentation claims. Fortunately, in the days since it was invented, other terms have picked up the slack.

I'd suggest trying [community]/termite, which is pretty excellent and calls on fontconfig, so fallbacks work like they should.

Good luck!

Trilby · 2017-09-28 16:35:48

escondida wrote:

I'd suggest trying [community]/termite, which is pretty excellent and calls on fontconfig, so fallbacks work like they should.

$ ldd /usr/bin/urxvt | grep fontconfig
        libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007f890e417000)

escondida · 2017-09-28 16:37:24

Trilby wrote:

escondida wrote:
I'd suggest trying [community]/termite, which is pretty excellent and calls on fontconfig, so fallbacks work like they should.
$ ldd /usr/bin/urxvt | grep fontconfig
        libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x00007f890e417000)

o,0 Whoops!

seth · 2017-09-28 18:44:04

Trilby wrote:

but yes, something is wrong with your locale ... What is in your locale.conf?

Probably nothing, the rest is/was a systemd bug:
https://bbs.archlinux.org/viewtopic.php … 8#p1728478

@tenplaza9, try to (install and) add "DejaVu Sans Mono" to the list of urxvt fonts.

tenplaza9 · 2017-09-29 09:30:31

Trilby wrote:

Have you tried urxvt with just the same two fonts as i3bar? If a character (even an unsuitable one) is found in Inconsolata, urxvt wouldn't check the other fonts. I really don't know if this is at all likely, but it's easy to test.

Yeah, I did... Spent a long time messing around with the fonts in urxvt.

escondida wrote:

Let's handle this first. /etc/locale.gen doesn't actually *set* your locale, it just sets things up so that it's possible to use a given locale. You could have multiple options uncommented in /etc/locale.gen then run locale-gen, which would allow you to switch between locales.
To actually set your locale, you need to edit or create /etc/locale.conf. See the locale page on the Arch wiki or locale.conf(5) for the sorts of things you might put in there.

Please check the edit to the original post.

seth wrote:

@tenplaza9, try to (install and) add "DejaVu Sans Mono" to the list of urxvt fonts.

I tried this and many other fonts, didn't help

Thanks for all the help so far!

tenplaza9 · 2017-09-29 09:35:54

escondida wrote:

I'd suggest trying [community]/termite, which is pretty excellent and calls on fontconfig, so fallbacks work like they should.
Good luck!

Yeahh, I probably should, but spent lotta time configuring urxvt over the past month

tenplaza9 · 2017-09-29 10:32:27

seth wrote:

Probably nothing, the rest is/was a systemd bug:
https://bbs.archlinux.org/viewtopic.php … 8#p1728478

Checked this out and tried to manually edit /etc/profile.d/locale.sh with the following line (commenting out the current {LANG:-C}).

LANG=${LANG:en_US.UTF-8}

and

LANG=${LANG:en_US.utf8}

Didn't really help :-(

seth · 2017-09-29 12:51:14

What do you think that does? :-P

fix your /etc/locale.conf, eg.

LANG="en_US.UTF-8"
LC_COLLATE="C"

add /etc/profile.d/zz_unsetlocales.sh (ensure it's executable

#!/bin/sh
for LCV in `printenv | grep -E '^LC_'`; do
	[ -z ${LCV#*=} ] && unset ${LCV%=*}
done

and revert the local changes to /etc/profile.d/locale.sh

tenplaza9 · 2017-09-30 18:30:52

Thanks for all the support guys, turns out I was doing it wrong all along. Somehow I downloaded a template .zshenv with the following line:

unsetopt GLOBAL_RCS

and forgot to uncomment it throughout. Then I was messing with my environment variables today and that fixed it. So sorry for the trouble, have a nice day. And thanks a lot @seth for the effort.

Arch Linux

#1 2017-09-28 15:59:45

[SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#2 2017-09-28 16:27:21

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#3 2017-09-28 16:31:16

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#4 2017-09-28 16:35:48

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#5 2017-09-28 16:37:24

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#6 2017-09-28 18:44:04

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#7 2017-09-29 09:30:31

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#8 2017-09-29 09:35:54

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#9 2017-09-29 10:32:27

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#10 2017-09-29 12:51:14

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

#11 2017-09-30 18:30:52

Re: [SOLVED] Problem with unicode fonts in rxvt-unicode (and locales).

Board footer