Arch Linux

JohnnyBB

Member

Registered: 2021-06-13

Posts: 2

[SOLVED] Rendering non-English characters in newsboat

Hello,

Newsboat does not render non-English characters such as special Norwegian (æ, ø, å) and French (é, è, ê etc) characters on my system and replaces them with question marks '?' instead.

I haven't been able to figure out why. Does anyone have a solution?

Thanks

$ env | grep 'LANG\|LC'
LANGUAGE=en_US.utf8
LC_ADDRESS=nb_NO.utf8
LC_NAME=nb_NO.utf8
LC_MONETARY=nb_NO.utf8
LC_PAPER=nb_NO.utf8
LANG=en_US.utf8
LC_IDENTIFICATION=nb_NO.utf8
LC_TELEPHONE=nb_NO.utf8
LC_MESSAGES=en_US.utf8
LC_MEASUREMENT=nb_NO.utf8
LC_CTYPE=C
LC_TIME=nb_NO.utf8
LC_COLLATE=C
LC_NUMERIC=nb_NO.utf8

$ newsboat --version
Newsboat 2.26.0 - https://newsboat.org/
Copyright (C) 2006-2015 Andreas Krennmair
Copyright (C) 2015-2021 Alexander Batischev
Copyright (C) 2006-2017 Newsbeuter contributors
Copyright (C) 2017-2021 Newsboat contributors

Newsboat is free software licensed under the MIT License. (Type `newsboat -vv' to see the full text.)
It bundles:
- JSON for Modern C++ library, licensed under the MIT License: https://github.com/nlohmann/json
- optional-lite library, licensed under the Boost Software License: https://github.com/martinmoene/optional-lite
- expected-lite library, licensed under the Boost Software License: https://github.com/martinmoene/expected-lite

Newsboat 2.26.0
System: Linux 5.17.0-rc7-1-mainline (x86_64)
Compiler: g++ 11.1.0
ncurses: ncurses 6.3.20211021 (compiled with 6.3)
libcurl: libcurl/7.82.0 OpenSSL/1.1.1m zlib/1.2.11 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.2 libpsl/0.21.1 (+libidn2/2.3.0) libssh2/1.10.0 nghttp2/1.47.0 (compiled with 7.80.0)
SQLite: 3.38.0 (compiled with 3.37.0)
libxml2: compiled with 2.9.12

$ uname -a
Linux JohnnyBB 5.17.0-rc7-1-mainline #1 SMP PREEMPT Tue, 08 Mar 2022 18:13:47 +0000 x86_64 GNU/Linux

Last edited by JohnnyBB (2022-03-09 14:35:05)

V1del

Forum Moderator

Registered: 2012-10-16

Posts: 25,302

Re: [SOLVED] Rendering non-English characters in newsboat

What's your terminal/monospace font? Are both the locales from that grep actually generated? Output of

locale
locale -a

?

FWIW You probably don't want any of these environments to use the C locale, so find out why ctype and collate are on C instead of one of the actual languages.

From a quick googler

LC_CTYPE
    Specifies the locale to use for LC_CTYPE category information. The LC_CTYPE category determines character handling rules governing the interpretation of sequences of bytes of text data characters (that is, single-byte versus multibyte characters), the classification of characters (for example, alpha, digit, and so on), and the behavior of character classes.

you probably definitely don't want that to be C

Last edited by V1del (2022-03-09 10:25:43)

JohnnyBB

Member

Registered: 2021-06-13

Posts: 2

Re: [SOLVED] Rendering non-English characters in newsboat

Hi V1del,

My terminal emulator is st. It is configured to use the 'JetBrains Mono Medium' and 'JoyPixels' fonts. It prints foreign characters just fine.
The locales are generated.

I just tried to change LC_CTYPE to en_US.utf8 and it solved my issue with the foreign characters in newsboat.
Thanks for your help, the issue is solved.

I keep LC_COLLATE = C so the output order of commands such as ls stay truly alphabetical (dots appear first), but if there's another way to obtain the same sorting order, I'm curious to hear about it.

Last edited by JohnnyBB (2022-03-09 18:37:02)