Unicode support in xterm & TTY

hughitt1 · 2012-12-12 12:27:13

Hey all,

I've been trying out and reading up on different terminals lately, and have been gradually tweaking my .Xresources along the way.

I spent some time working in urxvt, and getting unicode and font support working there was pretty straight-forward. Now I'd like to learn how to do the same thing with xterm, and if possibly also in in the virtual console.

Currently, when I try to display unicode characters (e.g. 中文) in either, I just see rectangular blocks.

Here is what I have in .Xresources now:

xterm*faceName: Droid Sans Mono:pixelsize=24
xterm*dynamicColors: true
xterm*utf8: 2

My goal is to achieve something similar to my urxvt configuration:

!! xft
Xft.dpi: 96
Xft.antialias: true.
Xft.rgba: rgb.
Xft.hinting: true.
Xft.hintstyle: hintslight

!! URxvt
URxvt*font: xft:droid sans mono:pixelsize=20:antialias=true:hinting=true
URxvt*boldFont: xft:droid sans mono:bold:pixelsize=20:antialias=true:hinting=true

I tried a few other things, e.g. launching uxterm, and "xterm -u8", but neither matter much. Also, it seems like there are a couple different ways to specify a font for xterm ("faceName" and "font"), but I haven't found a good explanation of the differences between the two yet.

I may not use anti-aliasing/hinting in the either if it seems to have a significant impact on performance, but I would like to first try it out at least.

Any help would be greatly appreciated.

Thanks!
Keith

thisoldman · 2012-12-13 04:55:10

The linux console (tty) won't support many non-Western languages. The fonts are limited to 256 glyphs or if you're willing to give up the bright colors, 512 glyphs. That number certainly won't support the CJK character sets. I've seen others recommend a framebuffer terminal such as fbterm. I'd try Googling "framebuffer terminal".

Edited after re-reading the origiinal question.

Last edited by thisoldman (2012-12-13 10:37:12)

hughitt1 · 2012-12-14 10:21:34

Thanks thisoldman,

I'll check that out.

Anyone have any suggestions about xterm?

Keith

thisoldman · 2012-12-14 11:42:48

The different ways of specifying fonts for xterm are used to keep backward compatibility. The old style is called X Logical Font Description, or XLFD, https://en.wikipedia.org/wiki/X_logical … escription. The Xft or X Free Type style is newer and provides compatibility with fontconfig, http://freedesktop.org/wiki/Software/Xft.

I think specifying a wide font for xterm is needed. I did try an experiment. I temporarily changed xterm fonts, by editing my .Xdefaults, from DejaVu Sans Mono to a font that would support Chinese and Japanese characters:

!XTerm*faceName:             xft:DejaVu Sans Mono:style=Book:size=8
xterm*font:     -misc-fixed-medium-r-normal--18-120-100-100-c-90-iso10646-1
xterm*wideFont: -misc-fixed-medium-r-normal-ja-18-120-100-100-c-180-iso10646-1

Then I updated my X resources database and started an xterm in utf-8 mode:

$ xrdb -merge .Xdefaults
$ xterm -u8 &

Before: After:

I use a helper app, xfontsel, for previewing and selecting fonts using the XLFD method, from extra/xorg-xfontsel.

hughitt1 · 2012-12-14 12:15:09

Thanks for the information regarding XLFD vs. XFT.

Just to be clear, "xterm*font" and "xterm*wideFont" relate to using an XLFD font and "xterm*faceName" is for specifying an XFT font to use?

I tried out your suggestion and it indeed helps to display CJK and other unicode characters. What is strange, however, is that in other terminals (e.g. terminator and urxvt) I could display unicode characters, including CJK, when droid sans mono was specified as my main font. Maybe they are just better at choosing a fallback font?

Another app I've been using lately for trying out different kinds of fonts is Speciman (aur/gnome-specimen). It is GTK-based though so a little heavier.

hughitt1 · 2012-12-14 12:32:28

Okay, a little more experimenting and I found one pretty simple setting that works for xterm*faceName:

xterm*faceName:monospace:pixelsize=20

I tried a few other (Ubuntu Mono, Bitstream Vera Sans Mono, etc), and none of them displayed unicode characters properly -- only monospace has worked so far.

thisoldman · 2012-12-14 23:28:05

For xterm and Droid fonts, try these in your configuration:

XTerm*faceName:             xft:Droid Sans Mono:style=Regular:size=9
XTerm*faceNameDoublesize:   xft:Droid Sans Japanese:style=Regular
! Or...
! XTerm*faceNameDoublesize:   xft:Droid Sans Fallback:style=Regular

When I try 'xterm -u8', it works. I don't have many CJ files to test these 2 Droid variants for differences. So far I can see no differences between 'Japanese' and 'Fallback'.

Edit:
The '-u8' option does not have to be given with the default Arch xterm, UTF-8 support is compiled in. All one has to do to enable CJ glyphs is specify a proper font for 'faceNameDoublesize'.

I haven't explored the effect enabling a Chinese locale would have on xterm.

Last edited by thisoldman (2012-12-15 02:31:40)

hughitt1 · 2012-12-18 12:22:52

Cool. Got it working now.

I had to switch away from Droid Sans Mono to another font in order for non-CJK unicode characters (e.g. ╰) to display correctly. The below configuration worked well for me in both cases:

xterm*faceName: xft:monospace:size=16
xterm*faceNameDoublesize:   xft:Droid Sans Japanese:style=Regular

I also had to use "xterm*" instead of "XTerm*" for it to work properly.

What is interesting is that if I use gnome-terminal with 'droid sans mono' everything just works. Perhaps they are automatically choosing fallback font(s) when for unicode characters?

Thanks, thisoldman!
Keith

thisoldman · 2012-12-19 04:41:12

Out of curiosity, what does 'fc-match monospace' return?

GTK apps will sift fonts to find the 'closest font' that includes the missing character. I assume that's what's happening with gnome-terminal.

Your experience with capitalization in the config settings, XTerm' versus 'xterm', has me scratching my head. 'XTerm' is the class name, 'xterm' is an instance name. Class should take precedence over instance, so all my .Xdefaults settings use the class names. Are you using an .Xdefaults or an .Xresources file? I ask because I can't get my urxvt settings to take effect when using .Xresources, and I've always used class names.

After reading the xterm manpage and looking at the xterm default configuration file, /usr/share/X11/app-defaults/xterm, I think using the XLFD method for font selection allows one to create a ladder of up to seven fonts plus a wide font for character display. I plan to experiment when I have time and perhaps I'll have a paragraph or two to add to the wiki.

hughitt1 · 2012-12-19 12:09:45

$ fc-match monospace
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

I believe I made a mistake regarding the capitalization. When I was initially testing out different settings I would use "xrdb -merge" to apply the changes and then open up a new terminal. What I found today, however, is that the old changes can persist in a current X session, for example if you switch from using class to instance names or vice versa, when doing an 'xrdb -query -all' both settings appear.

To be safe, today I tried completely killing the X session in between each change and this time using "XTerm*" worked fine.

Here are some screenshots for settings I tested:

1. Defaults
xterm defaults
No font was specified. The result was same with utf/colors settings removed.

2. Droid Sans with Fallback
xterm droid sans

3. Monospace + MS Yahei
xterm monospace yahei
Result is similar when using Monospace + Droid Sans Japanese or another CJK font.

Note: the last two images were scaled to 50%.

Keep me posted if you do end up experimenting more with XLFD settings.

Best,
Keith

Arch Linux

#1 2012-12-12 12:27:13

Unicode support in xterm & TTY

#2 2012-12-13 04:55:10

Re: Unicode support in xterm & TTY

#3 2012-12-14 10:21:34

Re: Unicode support in xterm & TTY

#4 2012-12-14 11:42:48

Re: Unicode support in xterm & TTY

#5 2012-12-14 12:15:09

Re: Unicode support in xterm & TTY

#6 2012-12-14 12:32:28

Re: Unicode support in xterm & TTY

#7 2012-12-14 23:28:05

Re: Unicode support in xterm & TTY

#8 2012-12-18 12:22:52

Re: Unicode support in xterm & TTY

#9 2012-12-19 04:41:12

Re: Unicode support in xterm & TTY

#10 2012-12-19 12:09:45

Re: Unicode support in xterm & TTY

Board footer