Arch Linux

emacsomancer

Member

Registered: 2014-09-20

Posts: 211

Missing characters when running Dwarf Fortress from Arch in TEXT mode

I'm trying to run Dwarf Fortress on my Arch desktop in TEXT PRINT mode and connect to it from a less powerful laptop. However, a number of the symbolic characters are missing/black.

I tried running `localectl set-locale LANG=en_US.UTF-8` on my Arch machine as per http://www.bay12forums.com/smf/index.php?topic=162274.0 . I've tried various terminal emulators from the other machine, including kitty, xterm, st, konsole. I checked in konsole, to make sure that it was set to use 'UTF-8' encoded, and it was.

Any other things I should try checking to see why there are missing characters?

mpan

Member

Registered: 2012-08-01

Posts: 1,334

Website

Re: Missing characters when running Dwarf Fortress from Arch in TEXT mode

Not a DF user here, but can you post an image that shows the problem? It’s hard to guess what characters are missing.

Can you tell which characters are the problem? Do you have typefaces installed for them?

Last edited by mpan (2020-07-03 17:21:40)

---

Sometimes I seem a bit harsh — don’t get offended too easily!

tenfoxes

Member

From: Pittsburgh, PA., USA

Registered: 2014-01-06

Posts: 11

Website

Re: Missing characters when running Dwarf Fortress from Arch in TEXT mode

Any other things I should try checking to see why there are missing characters?

If you haven't already, try installing the gnu-free-fonts package, which has very wide Unicode support. If that doesn't get you there, you could also try unifont—I'd recommend the TrueType version in the AUR over the BDF one in extra because Pango doesn't support bitmap fonts anymore and some terminal emulators use it for text rendering. It's likely that you don't have full coverage of all of the glyphs DF uses with the fonts you currently have installed.

I don't think setting your locale will make a difference in this case (although you should do it anyway if you didn't when you first set up your system). That forum thread is about running DF at the system console (i.e. outside of a display server); your locale can affect what glyphs are loaded from your console font.

EDIT: Oh, I realize the laptop in question might not be running Arch. Well, from the terms you listed, I assume it's some flavor of Linux at least, and those packages should probably be available there whatever it is. If you're logging into your desktop via ssh, your desktop won't have anything to do with the glyphs you'll actually see in your term on the laptop—it's just sending you text (provided you're not doing X11 forwarding, which it doesn't sound like you are). It's up to the environment on your laptop to actually render it.

Last edited by tenfoxes (2020-07-04 09:54:19)

emacsomancer

Member

Registered: 2014-09-20

Posts: 211

Re: Missing characters when running Dwarf Fortress from Arch in TEXT mode

I don't think it's a font issue. I've tried DejaVu Sans Mono, Fira Mono, Free Mono, Unifont and others that support a large character set. I think it must be either a colour issue (i.e. something are being shown in black that should be a different colour) or a terminal encoding issue (i.e. some characters are not being shown despite being supported by the font).

tenfoxes

Member

From: Pittsburgh, PA., USA

Registered: 2014-01-06

Posts: 11

Website

Re: Missing characters when running Dwarf Fortress from Arch in TEXT mode

I don't think it's a font issue. I've tried DejaVu Sans Mono, Fira Mono, Free Mono, Unifont and others that support a large character set.

Interesting. (I assume you mean on the laptop, just to be sure.)

I think it must be either a colour issue (i.e. something are being shown in black that should be a different colour)…

Couldn't you check for this by just changing your term background color?

…or a terminal encoding issue (i.e. some characters are not being shown despite being supported by the font).

It's sort of hard for me to see how that could be the case. The desktop is just sending a stream of characters from DF to the laptop over the wire; as long as it's identified the term you're using on the laptop correctly, I don't think it matters if the desktop even knows what the characters are or not. On the laptop, everything should be fine as long as your term really does have full Unicode support and the fonts you have include the necessary glyphs.

Let's try this. Here is a text document containing every character DF uses (I made it based on this page). There's one per line. With the exception of the nullchar on the first line, the non-breaking space on the last line, and the regular space on line 33, does every character in that file render properly in your term on the laptop?

Also, just to be sure, have you tried running DF with PRINT_MODE:TEXT on your desktop and making sure it works properly there? Just to make sure there's not some more serious problem afoot. In my environment, for example, DF 0.47.04 segfaults a bit into world creation if I run it with PRINT_MODE:TEXT but works fine with PRINT_MODE:2D, so I wouldn't exactly call TEXT mode a paragon of stability.

emacsomancer

Member

Registered: 2014-09-20

Posts: 211

Re: Missing characters when running Dwarf Fortress from Arch in TEXT mode

I don't think it's a font issue. I've tried DejaVu Sans Mono, Fira Mono, Free Mono, Unifont and others that support a large character set.

Interesting. (I assume you mean on the laptop, just to be sure.)

I think it must be either a colour issue (i.e. something are being shown in black that should be a different colour)…

Couldn't you check for this by just changing your term background color?

I have tried changing the background colour and it's the same behaviour, so I suppose it's not that. I was thinking that somehow it isn't using as many colours as it should and that's why some are not appearing?

Let's try this. Here is a text document containing every character DF uses (I made it based on this page). There's one per line. With the exception of the nullchar on the first line, the non-breaking space on the last line, and the regular space on line 33, does every character in that file render properly in your term on the laptop?

Yes, they do.

Also, just to be sure, have you tried running DF with PRINT_MODE:TEXT on your desktop and making sure it works properly there? Just to make sure there's not some more serious problem afoot. In my environment, for example, DF 0.47.04 segfaults a bit into world creation if I run it with PRINT_MODE:TEXT but works fine with PRINT_MODE:2D, so I wouldn't exactly call TEXT mode a paragon of stability.

Well, I haven't had any issues with it crashing so far, but you're right: when I run PRINT_MODE:TEXT on my Arch desktop (that it, run it locally), I have the exactly the same issues. So I'm not sure what might fix that.

Last edited by emacsomancer (2020-07-05 14:35:54)

tenfoxes

Member

From: Pittsburgh, PA., USA

Registered: 2014-01-06

Posts: 11

Website

Re: Missing characters when running Dwarf Fortress from Arch in TEXT mode

I have tried changing the background colour and it's the same behaviour, so I suppose it's not that. I was thinking that somehow it isn't using as many colours as it should and that's why some are not appearing?

Right off the bat, let's give this a shot on the laptop:

echo -e "\
\e[30m DBLK \e[31m DRED \e[32m DGRN \e[33m DYEL \e[0m
\e[34m DBLU \e[35m DMAG \e[36m DCYN \e[37m DWHI \e[0m
\e[30;1m BBLK \e[31;1m BRED \e[32;1m BGRN \e[33;1m BYEL \e[0m
\e[34;1m BBLU \e[35;1m BMAG \e[36;1m BCYN \e[37;1m BWHI \e[0m"

DF with PRINT_MODE:TEXT expects you to be able to see every label there except DBLK. (It uses only 16 colors, the "basic" 4-bit ANSI colors.) If you can't, I would (a) make your terminal background color jet black and (b) modify your color scheme until all 15 other labels are clearly visible.

If you like to use kitty, by the way, you have no hope aside from forking it or maybe running DF within some kind of wrapper that can modify its color codes as they're output, because the maintainer refuses to add a setting enabling bold text to be shown with bright colors (and was awfully rude to the people in that thread, I might add). Luckily I think most other terms will behave as DF expects.

As an alternative to wrangling with graphical terms, you could switch to another tty and play the game at the system console. This sidesteps a lot of the complexity involved in getting an application like this to cooperate with a modern graphical term, and it looks really good as well if you have a nice console font—reminds me of playing Return to Kroz in the early '90s. One major disadvantage of this approach, though, is that I haven't been able to find a way to get DF to respect (tile-based) resolution settings; it just stays at the 80x25 minimum whatever I try.

As a side note, if you want some fonts to use in X11 that resemble the game's default font, the oldschool-pc-fonts package in the AUR is a good choice. I think "Px437 IBM EGA8" is the exact same font, in fact, but any of IBM's fonts in that package work well, and I also thought "Px437 TandyNew TV" was a nice choice for a square font if you like to play that way. The "PxPlus" variants include additional characters, but not ones DF uses.

Also, just to be sure, have you tried running DF with PRINT_MODE:TEXT on your desktop and making sure it works properly there? Just to make sure there's not some more serious problem afoot. In my environment, for example, DF 0.47.04 segfaults a bit into world creation if I run it with PRINT_MODE:TEXT but works fine with PRINT_MODE:2D, so I wouldn't exactly call TEXT mode a paragon of stability.

Well, I haven't had any issues with it crashing so far, but you're right: when I run PRINT_MODE:TEXT on my Arch desktop (that it, run it locally), I have the exactly the same issues. So I'm not sure what might fix that.

On this note, when I tried running DF with PRINT_MODE:TEXT at the system console, it worked fine there; I was able to complete world creation and embark without issue. After that I could play with the same fortress in X11 with no problems either, so the crash seems to be limited to a specific moment during history generation (as far as I can tell). I've opened a bug report for it since we both hit it.

In any case, it shouldn't really matter if you're playing over SSH, because you're not running the game in a graphical environment that way on the host anyhow (unless you're using X11 forwarding). To be sure, I was a bit naughty and tried installing DF on my web server and playing it there a little bit to see if I encountered any issues similar to what you're describing. Although it was a real bear getting it to run at all in Debian Stable, it was perfectly fine once I did. One thing that is worth keeping in mind is that you'll probably need to disable sound, although I imagine you've done this already.

Let's try this. Here is a text document containing every character DF uses (I made it based on this page). There's one per line. With the exception of the nullchar on the first line, the non-breaking space on the last line, and the regular space on line 33, does every character in that file render properly in your term on the laptop?

Yes, they do.

Well, all right then, we can rule that out. It's theoretically possible that DF itself is not sending the right characters, but since we know your term can display them properly, that would be the game's problem (and seems unlikely to me in any case).

As a side note, this has been one of the many times I wish DF was free software. PRINT_MODE:TEXT clearly needs some fixing up and I'd be happy to do it if I could. It would make an issue like you're encountering so much easier to troubleshoot, too.

Last edited by tenfoxes (2020-07-07 22:58:48)

emacsomancer

Member

Registered: 2014-09-20

Posts: 211

Re: Missing characters when running Dwarf Fortress from Arch in TEXT mode

So there seems to be something off either with some aspect of how my Arch system is configured (or how Dwarf Fortress is running there) as I can ssh from my laptop to a different machine running dwarf fortress on a different distro and it works as expected. I've tried copying over the same df configuration to the Arch machine and so on, but that doesn't seem to make a difference. I tried running (on the hosting Arch system) from bash, zsh, and dash and that also didn't seem to make a difference. Is there something else I should look at on the Arch machine? terminal configuration? xdefaults?