Arch Linux

Bipolarbearz

Member

Registered: 2014-12-22

Posts: 6

[SOLVED] Xterm Japanese/Unicode Characters not displaying (??) instead

Hey,

So I'm new to linux and arch and I've made a point of reading every relevant thread posted when I searched for 'Japanese terminal' and I haven't been able to get my terminal to display Japanese characters. All I see are ?????.

This is my /etc/locale.config

LANG=en_CA.UTF-8
LC_COLLATE=C

This is the output from running:

$ locale

LANG=en_CA.UTF-8
LC_CTYPE="en_CA.UTF-8"
LC_NUMERIC=\"en_CA.UTF-8\"
LC_TIME=en_CA.UTF-8
LC_COLLATE=C
LC_MONETARY=\"en_CA.UTF-8\"
LC_MESSAGES=C
LC_PAPER=\"en_CA.UTF-8\"
LC_NAME=\"en_CA.UTF-8\"
LC_ADDRESS=\"en_CA.UTF-8\"
LC_TELEPHONE=\"en_CA.UTF-8\"
LC_MEASUREMENT=\"en_CA.UTF-8\"
LC_IDENTIFICATION=\"en_CA.UTF-8\"
LC_ALL=

This is the output from running:

$ locale -a

C
POSIX
en_CA.utf8
en_US.utf8
ja_JP.utf8

This is my ~/.Xresources file:

! Name
Xterm*termName: xterm-256color

! Font style
xterm*faceName: xft:monospace:size=11
xterm*faceNameDoublesize:   xft:Droid Sans Japanese:style=Regular

xterm*utf8: 2

! Window size
xterm*VT100.geometry: 100x40

! No more bell
xterm*vbell: True

! Scrollback
xterm*savelines: 4096

! double-click to select whole URLs :D
xterm*charClass: 33:48,36-47:48,58-59:48,61:48,63-64:48,95:48,126:48

! Scrollbar
xterm*rightScrollBar: false
xterm*ScrollBar: false

! Foreground & background
*background:#293b3b
*foreground:#bf6e63

!terminal colours----------------------------------------------------------------------
!http://www.xcolors.net/dl/fishbone

!black
*color0:#293b3b
*color8:#344040

!red
*color5:#bf5357
*color13:#bf6367

!green
*color2:#41a551
*color10:#51a55f

!yellow
*color3:#aea64f
*color11:#aea75d

!blue
*color4:#416ea5
*color12:#5177a5

!magenta
*color1:#bf6053 
*color09:#bf6e63

!cyan
*color6:#714da5
*color14:#7859a5

!white
*color7:#876868
*color15:#876f6f

!----------------------------------------------------------------------------

Also, when I trying inputting text into xterm using ibus's anthy, nothing gets entered, and I'm assuming that is related to the problem of not being able to view the Japanese characters properly
I'm really not sure where I'm going wrong, but it's driving me nuts and any help would be greatly appreciated.

Last edited by Bipolarbearz (2014-12-24 03:24:40)

thisoldman

Member

From: Pittsburgh

Registered: 2009-04-25

Posts: 1,172

Re: [SOLVED] Xterm Japanese/Unicode Characters not displaying (??) instead

I can't help with anthy.  But maybe I can help if the problem is elsewhere.

I did try your xterm settings and they worked for me, testing with a line of script copied from a Wikipedia page. I tried this in an empty directory, from xterm. The Japanese script was pasted into the first command.

$ touch 'ラドクリフマラソン五輪代表に 1万m出場にも含み'
$ ls
ラドクリフマラソン五輪代表に 1万m出場にも含み

Is 'Droid Sans Japanese' properly installed? Also, 'monospace' is an alias for some font: what font is it?

$ fc-match 'Droid Sans Japanese'
DroidSansJapanese.ttf: "Droid Sans Japanese" "Regular"

$ fc-match monospace
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

Do the fonts work with an xterm started from the command line?

$ xterm -fa 'xft:monospace:size=11' -fd 'xft:Droid Sans Japanese:style=Regular'

Did you post your entire ~/.Xresources file?

Bipolarbearz

Member

Registered: 2014-12-22

Posts: 6

Re: [SOLVED] Xterm Japanese/Unicode Characters not displaying (??) instead

Yes, that is my entire ~/.Xresources file. Running the commands you suggested gave me the following:

$ fc-match 'Droid Sans Japanese'
DroidSansJapanese.ttf: "Droid Sans Japanese" "Regular"

$ fc-match monospace
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

Running:

$ xterm -fa 'xft:monospace:size=11' -fd 'xft:Droid Sans Japanese:style=Regular'

Did open up a terminal without any issues, however, I'm not entirely sure if I can tell the difference between whatever the terminal default is and Droid Sans/DejaVu.
Also, when I tried to run $ xterm -fa ... I got the following warning:

Warning: locale not supported by C library, locale unchanged

I get a similar warning when trying to open up other programs, such as gedit:

(gedit:954): Gtk-WARNING **: Locale not supported by C library. 
                      Using the fallback 'C' locale.

My assumption is these problems are some how related?

Last edited by Bipolarbearz (2014-12-22 16:33:40)

thisoldman

Member

From: Pittsburgh

Registered: 2009-04-25

Posts: 1,172

Re: [SOLVED] Xterm Japanese/Unicode Characters not displaying (??) instead

Your default xterm font is DejaVu Sans Mono. I don't know how scim and anthy may complicate setting up a locale but you do appear to have something oddly configured.

Your output from the bare locale command shows backslashed double quotes.  My output shows no backslashes:

$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=C
LC_MONETARY="en_US.UTF-8"
    -- snip --
LC_ALL=

The only line I have uncommented in /etc/locale.gen is:

en_US.UTF-8 UTF-8

The only possibly significant difference between your xterm settings and mine is the way we set xterm to use UTF-8. I have:
⁣    XTerm*locale: true
You have:
⁣    xterm*utf8: 2
I think they may just be different methods that reach the same end when you only have UTF-8 locales installed.  I can get the Droid glyphs to display using either resource setting.

If I were in your shoes, I would work through the Archwiki's Locale page, step by step, to see if I could silence the warnings.

If the warnings still appeared, I would step back, and do things one step at a time. If you uninstall anthy, or scim-anthy, and regenerate your locales, do the warnings disappear?  Does xterm display the CJK glyphs? If not, does regenerating the locales using only en_CA.UTF-8 work?

I wish I could offer more help but I don't have the knowledge to offer more.

If you want to see which installed fonts say they have Japanese language support, use this command:

fc-list ':lang=ja'

Do note that what a font advertises and what you and I would call language support may be different.

Bipolarbearz

Member

Registered: 2014-12-22

Posts: 6

Re: [SOLVED] Xterm Japanese/Unicode Characters not displaying (??) instead

Hey, thanks for the advice. I've actually tried going through the locale set-up with no luck, but I'll take your advice and uninstall anthy and check a few other things. Thanks for the help : )

EDIT: IT WORKS! Wow, feels good to get to the root of a problem and fix it yourself : ). I regenerated my locale using en_US.UTF-8, rebooted, and ran $ locale ... only to get the same output I had before I regenerated the locale! The solution was found here: https://wiki.archlinux.org/index.php/Lo … g_language  - I didn't know GNOME kept it's own data about my locale, and that's why I was getting errors and improperly displayed CJK glyphs!

Solution (At least for those using GNOME):
1) Delete the contents of  ~/.pam_environment
2) Setup locale according to the first few steps here: https://wiki.archlinux.org/index.php/Lo … g_language

Thanks for the help!

Last edited by Bipolarbearz (2014-12-24 03:22:11)