Special characters displaying as boxes on system and terminal

locuaz · 2018-08-23 04:45:38

Recently I fully upgraded my system (previously I uninstalled a package on perl because incompatibility and shutter), then I rebooted a couple of times and my system was working OK, after I installed FBReader and finally I had to switch to Windows (dual boot), when I returned to Linux, my whole system (terminal included) was displaying special characters as boxes. I uninstalled FBReader, and re-installed Google fonts and rebooted several times, but nothing changed. I don't know what happened, someone may help me? Thanks in advance.

PS: I followed the instructions on this article but my problem persists.

seth · 2018-08-23 05:32:36

Outputs of

localectl
locale
locale -a

and please detail on "special charaters" (assuming "beyond ASCII", but the question is whether it's a certain set only - eg. what do ø, ö, € and ¥ look like?)

locuaz · 2018-08-23 10:39:21

localectl

   System Locale: LANG=en_US.utf-8
       VC Keymap: es
      X11 Layout: es

locale

LANG=es_PE.UTF-8
LC_CTYPE="es_PE.UTF-8"
LC_NUMERIC="es_PE.UTF-8"
LC_TIME="es_PE.UTF-8"
LC_COLLATE="es_PE.UTF-8"
LC_MONETARY="es_PE.UTF-8"
LC_MESSAGES="es_PE.UTF-8"
LC_PAPER="es_PE.UTF-8"
LC_NAME="es_PE.UTF-8"
LC_ADDRESS="es_PE.UTF-8"
LC_TELEPHONE="es_PE.UTF-8"
LC_MEASUREMENT="es_PE.UTF-8"
LC_IDENTIFICATION="es_PE.UTF-8"
LC_ALL=

locale -a

C
en_US.utf8
es_PE.utf8
POSIX

Special chars displaying as boxes:

??▎???▎?

And their representations in UTF encoding I use in my bash file are:

	# ICONS
	local CHECK=$'\u2714'  			  #"✔"
	local CROSS=$'\u2718'			  #"✘"
	local PC="\360\237\222\273"       #?
	local BAR1="\360\237\231\275"     #?
	local BAR2="\360\237\231\274"     #?
	local BAR3="\342\226\216"		  #▎
	local FOLDER="\360\237\227\201"   #?
	local CLOCK="\360\237\225\224"    #?
	local SIGN="\342\214\230"		  #⌘
	local HOUSE="\360\237\217\240"	  #?
	local EARTH="\360\237\214\216"	  #?
	local WW="\360\235\222\262"

These chars are showed without any problem: ø, ö, €, ¥

seth · 2018-08-23 11:43:01

Run

FC_DEBUG=4 pango-view -t ✘ | grep family: > font_debug.txt 2>&1

and paste the resulting font_debug.txt (also check the glyph printed in the pango-view window)

What "google fonts" did you install? Droid, Noto, Cantarell, …?

locuaz · 2018-08-23 12:23:08

The glyph showed is "✘", and the content of "font_debug.txt" is:

	[01;31m[Kfamily:[m[K "serif"(s)
	[01;31m[Kfamily:[m[K "serif"(s)
	[01;31m[Kfamily:[m[K "Bitstream Vera Serif"(w) "DejaVu Serif"(w) "Times New Roman"(w) "Thorndale AMT"(w) "Luxi Serif"(w) "Nimbus Roman No9 L"(w) "Nimbus Roman"(w) "Times"(w) "serif"(s)
	[01;31m[Kfamily:[m[K "Bitstream Vera Serif"(w) "DejaVu Serif"(w) "Times New Roman"(w) "Thorndale AMT"(w) "Luxi Serif"(w) "Nimbus Roman No9 L"(w) "Nimbus Roman"(w) "Times"(w) "serif"(s) "Nazli"(w) "Lotoos"(w) "Mitra"(w) "Ferdosi"(w) "Badr"(w) "Zar"(w)
	[01;31m[Kfamily:[m[K "Bitstream Vera Serif"(w) "DejaVu Serif"(w) "Times New Roman"(w) "Thorndale AMT"(w) "Luxi Serif"(w) "Nimbus Roman No9 L"(w) "Nimbus Roman"(w) "Times"(w) "Artsounk"(w) "BPG UTF8 M"(w) "Kinnari"(w) "Norasi"(w) "Frank Ruehl"(w) "Dror"(w) "JG LaoTimes"(w) "Saysettha Unicode"(w) "Pigiarniq"(w) "B Davat"(w) "B Compset"(w) "Kacst-Qr"(w) "Urdu Nastaliq Unicode"(w) "Raghindi"(w) "Mukti Narrow"(w) "malayalam"(w) "Sampige"(w) "padmaa"(w) "Hapax Berbère"(w) "MS Mincho"(w) "SimSun"(w) "PMingLiu"(w) "WenQuanYi Zen Hei"(w) "WenQuanYi Bitmap Song"(w) "AR PL ShanHeiSun Uni"(w) "AR PL New Sung"(w) "ZYSong18030"(w) "HanyiSong"(w) "MgOpen Canonica"(w) "Sazanami Mincho"(w) "IPAMonaMincho"(w) "IPAMincho"(w) "Kochi Mincho"(w) "AR PL SungtiL GB"(w) "AR PL Mingti2L Big5"(w) "AR PL Zenkai Uni"(w) "ＭＳ 明朝"(w) "ZYSong18030"(w) "NanumMyeongjo"(w) "UnBatang"(w) "Baekmuk Batang"(w) "KacstQura"(w) "Frank Ruehl CLM"(w) "Lohit Bengali"(w) "Lohit Gujarati"(w) "Lohit Hindi"(w) "Lohit Marathi"(w) "Lohit Maithili"(w) "Lohit Kashmiri"(w) "Lohit Konkani"(w) "Lohit Nepali"(w) "Lohit Sindhi"(w) "Lohit Punjabi"(w) "Lohit Tamil"(w) "Meera"(w) "Lohit Malayalam"(w) "Lohit Kannada"(w) "Lohit Telugu"(w) "Lohit Oriya"(w) "LKLUG"(w) "serif"(s) "Nazli"(w) "Lotoos"(w) "Mitra"(w) "Ferdosi"(w) "Badr"(w) "Zar"(w)
	[01;31m[Kfamily:[m[K "Bitstream Vera Serif"(w) "DejaVu Serif"(w) "Times New Roman"(w) "Thorndale AMT"(w) "Luxi Serif"(w) "Nimbus Roman No9 L"(w) "Nimbus Roman"(w) "Times"(w) "Artsounk"(w) "BPG UTF8 M"(w) "Kinnari"(w) "Norasi"(w) "Frank Ruehl"(w) "Dror"(w) "JG LaoTimes"(w) "Saysettha Unicode"(w) "Pigiarniq"(w) "B Davat"(w) "B Compset"(w) "Kacst-Qr"(w) "Urdu Nastaliq Unicode"(w) "Raghindi"(w) "Mukti Narrow"(w) "malayalam"(w) "Sampige"(w) "padmaa"(w) "Hapax Berbère"(w) "MS Mincho"(w) "SimSun"(w) "PMingLiu"(w) "WenQuanYi Zen Hei"(w) "WenQuanYi Bitmap Song"(w) "AR PL ShanHeiSun Uni"(w) "AR PL New Sung"(w) "ZYSong18030"(w) "HanyiSong"(w) "MgOpen Canonica"(w) "Sazanami Mincho"(w) "IPAMonaMincho"(w) "IPAMincho"(w) "Kochi Mincho"(w) "AR PL SungtiL GB"(w) "AR PL Mingti2L Big5"(w) "AR PL Zenkai Uni"(w) "ＭＳ 明朝"(w) "ZYSong18030"(w) "NanumMyeongjo"(w) "UnBatang"(w) "Baekmuk Batang"(w) "KacstQura"(w) "Frank Ruehl CLM"(w) "Lohit Bengali"(w) "Lohit Gujarati"(w) "Lohit Hindi"(w) "Lohit Marathi"(w) "Lohit Maithili"(w) "Lohit Kashmiri"(w) "Lohit Konkani"(w) "Lohit Nepali"(w) "Lohit Sindhi"(w) "Lohit Punjabi"(w) "Lohit Tamil"(w) "Meera"(w) "Lohit Malayalam"(w) "Lohit Kannada"(w) "Lohit Telugu"(w) "Lohit Oriya"(w) "LKLUG"(w) "FreeSerif"(w) "Code2000"(w) "Code2001"(w) "serif"(s) "Nazli"(w) "Lotoos"(w) "Mitra"(w) "Ferdosi"(w) "Badr"(w) "Zar"(w)
	[01;31m[Kfamily:[m[K "Bitstream Vera Serif"(w) "DejaVu Serif"(w) "Times New Roman"(w) "Thorndale AMT"(w) "Luxi Serif"(w) "Nimbus Roman No9 L"(w) "Nimbus Roman"(w) "Times"(w) "Artsounk"(w) "BPG UTF8 M"(w) "Kinnari"(w) "Norasi"(w) "Frank Ruehl"(w) "Dror"(w) "JG LaoTimes"(w) "Saysettha Unicode"(w) "Pigiarniq"(w) "B Davat"(w) "B Compset"(w) "Kacst-Qr"(w) "Urdu Nastaliq Unicode"(w) "Raghindi"(w) "Mukti Narrow"(w) "malayalam"(w) "Sampige"(w) "padmaa"(w) "Hapax Berbère"(w) "MS Mincho"(w) "SimSun"(w) "PMingLiu"(w) "WenQuanYi Zen Hei"(w) "WenQuanYi Bitmap Song"(w) "AR PL ShanHeiSun Uni"(w) "AR PL New Sung"(w) "ZYSong18030"(w) "HanyiSong"(w) "MgOpen Canonica"(w) "Sazanami Mincho"(w) "IPAMonaMincho"(w) "IPAMincho"(w) "Kochi Mincho"(w) "AR PL SungtiL GB"(w) "AR PL Mingti2L Big5"(w) "AR PL Zenkai Uni"(w) "ＭＳ 明朝"(w) "ZYSong18030"(w) "NanumMyeongjo"(w) "UnBatang"(w) "Baekmuk Batang"(w) "KacstQura"(w) "Frank Ruehl CLM"(w) "Lohit Bengali"(w) "Lohit Gujarati"(w) "Lohit Hindi"(w) "Lohit Marathi"(w) "Lohit Maithili"(w) "Lohit Kashmiri"(w) "Lohit Konkani"(w) "Lohit Nepali"(w) "Lohit Sindhi"(w) "Lohit Punjabi"(w) "Lohit Tamil"(w) "Meera"(w) "Lohit Malayalam"(w) "Lohit Kannada"(w) "Lohit Telugu"(w) "Lohit Oriya"(w) "LKLUG"(w) "FreeSerif"(w) "Code2000"(w) "Code2001"(w) "serif"(s) "Nazli"(w) "Lotoos"(w) "Mitra"(w) "Ferdosi"(w) "Badr"(w) "Zar"(w)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "DejaVu Serif"(s)
	[01;31m[Kfamily:[m[K "NanumMyeongjo"(s) "나눔명조"(s)
	[01;31m[Kfamily:[m[K "NanumMyeongjo"(s) "나눔명조"(s)
	[01;31m[Kfamily:[m[K "NanumMyeongjoExtraBold"(s) "나눔명조"(s) "NanumMyeongjo"(s)
	[01;31m[Kfamily:[m[K "NanumMyeongjoExtraBold"(s) "나눔명조"(s) "NanumMyeongjo"(s)
	[01;31m[Kfamily:[m[K "Lohit Bengali"(s)
	[01;31m[Kfamily:[m[K "Lohit Bengali"(s)
	[01;31m[Kfamily:[m[K "Lohit Tamil"(s)
	[01;31m[Kfamily:[m[K "Lohit Tamil"(s)
	[01;31m[Kfamily:[m[K "FreeSerif"(s)
	[01;31m[Kfamily:[m[K "FreeSerif"(s)

Fonts installed in my system:

local/adobe-source-han-sans-cn-fonts 1.004-2
    Adobe Source Han Sans Subset OTF - Simplified Chinese OpenType/CFF fonts
local/adobe-source-han-sans-jp-fonts 1.004-2
    Adobe Source Han Sans Subset OTF - Japanese OpenType/CFF fonts
local/adobe-source-han-sans-kr-fonts 1.004-2
    Adobe Source Han Sans Subset OTF - Korean OpenType/CFF fonts
local/cantarell-fonts 1:0.101-1
    Cantarell font family
local/fontconfig 2.13.0+10+g58f5285-1
    A library for configuring and customizing font access
local/freetype2 2.9.1-1
    Font rasterization library
local/lib32-fontconfig 2.13.0+10+g58f5285-1.1
    A library for configuring and customizing font access (32-bit)
local/lib32-freetype2 2.9.1-1
    TrueType font rendering library (32-bit)
local/libfontenc 1.1.3-2
    X11 font encoding library
local/libxfont2 2.0.3-1
    X11 font rasterisation library
local/libxft 2.3.2-2
    FreeType-based font drawing library for X
local/noto-fonts 20180322-1
    Google Noto TTF fonts
local/noto-fonts-extra 20180322-1
    Google Noto TTF fonts - additional variants
local/ttf-croscore 20180322-1
    Chrome OS core fonts
local/ttf-dejavu 2.37-2
    Font family based on the Bitstream Vera Fonts with a wider range of characters
local/ttf-freefont 20120503-5
    Set of free outline fonts covering the Unicode character set
local/ttf-google-fonts-git 1:r1265.08b5f478-1
    TrueType fonts from the Google Fonts project (git version)
local/ttf-inconsolata 1:2.0.0.1-3
    Monospace font for pretty code listings and for the terminal
local/ttf-liberation 2.00.1-7
    Red Hats Liberation fonts.
local/ttf-monofur 1.0-6
    A monospaced font derived from the eurofurence typeface family
local/ttf-roboto 2.138-1
    Google's signature family of fonts
local/ttf-ubuntu-font-family 0.83-3
    Ubuntu font family
local/ttf-wps-fonts 1.0-3
    Symbol fonts required by wps-office.
local/woff2 1.0.2-1
    Web Open Font Format 2 reference implementation
local/xorg-bdftopcf 1.1-1 (xorg xorg-apps)
    Convert X font from Bitmap Distribution Format to Portable Compiled Format
local/xorg-font-util 1.3.1-2 (xorg-fonts xorg)
    X.Org font utilities
local/xorg-font-utils 7.6-5
    Transitional package depending on xorg font utilities
local/xorg-fonts-100dpi 1.0.3-3 (xorg)
    X.org 100dpi fonts
local/xorg-fonts-75dpi 1.0.3-3 (xorg)
    X.org 75dpi fonts
local/xorg-fonts-alias 1.0.3-2
    X.org font alias files
local/xorg-fonts-encodings 1.0.4-5 (xorg-fonts xorg)
    X.org font encoding files
local/xorg-mkfontdir 1.0.7-9 (xorg xorg-apps)
    Create an index of X font files in a directory
local/xorg-mkfontscale 1.1.3-1 (xorg-apps xorg)
    Create an index of scalable font files for X
local/xorgproto 2018.4-1
    combined X.Org X11 Protocol headers

seth · 2018-08-23 12:26:48

So, if the glyph is showing up for the default font chain, how does that correspond with "whole system (terminal included)"?
What font is your terminal emulator configured to?

locuaz · 2018-08-23 16:52:07

Dear Seth, first off thanks for your patience.

This picture shows Konsole, Kate and Vivaldi:
https://drive.google.com/open?id=1QPDPR … Ct2AW3i_6O

Font used in terminal: DejaVu Sans Mono
Font used in Kate: Hack

I've tried to change font for terminal, but same result.
KWrite & Libre Office Writer display same problem but Visual Studio Code displays OK.

Last edited by locuaz (2018-08-23 16:58:53)

seth · 2018-08-23 18:34:39

Ah, the secret is that "✘" isn't among the problematic icons at all.

FC_DEBUG=4 pango-view --font="DejaVu Sans Mono" -t ? | grep family:
FC_DEBUG=4 pango-view --font="Hack" -t ? | grep family:
fc-match "DejaVu Sans Mono"
fc-match "Hack"

The particular glyph is only resolved by "Noto Sans Symbols2" on my system, found in the noto-fonts package.
Notice that this isn't a monospace font.
Did you use some powerline patched fonts before?

At post #7, you don't have to thank for patience - we'll talk about taht as post #70 ;-)

locuaz · 2018-08-23 21:42:24

Nop, only those fonts listed at post #5.
You are right, those glyphs are displayed "horribly" with "Noto Sans Symbols2"
I've tried recently this command:

$ sudo fc-cache -f -v

and nothing. This is upsetting, what happened? one moment the OS is working fine and next time, ploop! I don't want to re-install from scratch everything, it's not funny

loqs · 2018-08-23 23:09:54

https://bugs.archlinux.org/task/59779 ?

seth · 2018-08-24 05:39:16

For what happened: pacman log? (If you actually did update fontconfig - unannounced - you've the problematic package from comment #10)

If not:
What do you mean by "horribly"?
You actually should have noto-fonts installed, so the question is why (not even that font is resolved for the symbol, thus please post the outputs to the commands in comment #8

Arch Linux

#1 2018-08-23 04:45:38

Special characters displaying as boxes on system and terminal

#2 2018-08-23 05:32:36

Re: Special characters displaying as boxes on system and terminal

#3 2018-08-23 10:39:21

Re: Special characters displaying as boxes on system and terminal

#4 2018-08-23 11:43:01

Re: Special characters displaying as boxes on system and terminal

#5 2018-08-23 12:23:08

Re: Special characters displaying as boxes on system and terminal

#6 2018-08-23 12:26:48

Re: Special characters displaying as boxes on system and terminal

#7 2018-08-23 16:52:07

Re: Special characters displaying as boxes on system and terminal

#8 2018-08-23 18:34:39

Re: Special characters displaying as boxes on system and terminal

#9 2018-08-23 21:42:24

Re: Special characters displaying as boxes on system and terminal

#10 2018-08-23 23:09:54

Re: Special characters displaying as boxes on system and terminal

#11 2018-08-24 05:39:16

Re: Special characters displaying as boxes on system and terminal

Board footer