You are not logged in.

#1 2013-10-05 13:02:33

wiggly
Member
Registered: 2012-02-02
Posts: 8

ASCII Characters Display As ????

hi,

i seem to be having an issue displaying ASCII characters, all i seem to get is black diamonds with white triangles inside, this happens on gnome-terminal, on xterm i get nothing at all. also some fonts do not appear to render in my web browser (chromium or firefox) i seem to get hollow squares instead of text.

$ locale - a

	C
	en_GB
	en_GB.iso88591
	en_GB.utf8
	POSIX

$ locale

	LANG=en_GB.UTF-8
	LC_CTYPE="en_GB.UTF-8"
	LC_NUMERIC=en_GB-UTF-8
	LC_TIME=en_GB-UTF-8
	LC_COLLATE="en_GB.UTF-8"
	LC_MONETARY=en_GB-UTF-8
	LC_MESSAGES="en_GB.UTF-8"
	LC_PAPER=en_GB-UTF-8
	LC_NAME="en_GB.UTF-8"
	LC_ADDRESS="en_GB.UTF-8"
	LC_TELEPHONE="en_GB.UTF-8"
	LC_MEASUREMENT=en_GB-UTF-8
	LC_IDENTIFICATION="en_GB.UTF-8"
	LC_ALL=

# print ascii chars (program prints number followed by ascii conversion)

	180		?
	181		?
	182		?
	183		?
	184		?
	185		?
	186		?

thank you for looking, i would appreciate any help possiable with this.
if you need more info please let me know.

Offline

#2 2013-10-05 13:15:52

msthev
Member
Registered: 2012-04-05
Posts: 177

Re: ASCII Characters Display As ????

1. Don't say "program prints number followed by ascii conversion", post the source.

2. It's not a bug, it's a feature:
http://en.wikipedia.org/wiki/UTF-8
http://en.wikipedia.org/wiki/ASCII
ASCII characters have 7-bit values (from 0 to 127).

Last edited by msthev (2013-10-05 13:16:39)

Offline

#3 2013-10-05 13:25:39

wiggly
Member
Registered: 2012-02-02
Posts: 8

Re: ASCII Characters Display As ????

its not the program that has the problem, i have missed something when configuring my system. i know this because it is not limited to my program.

as for ASCII values going to 127 this is true unless you include the extended characters that run from 128 - 255.

if you want to see the C Source code i can show you, alough it was not really relevant except for the fact that it demenstrates my issue.

#include <stdio.h>

int main(void)
{
	
	
	int i;
	
	for(i = 0; i < 255; i++)
	{
		printf("%d\t%c\n", i, i);
	}
	
	return 0;
	
}

Offline

#4 2013-10-05 13:44:00

msthev
Member
Registered: 2012-04-05
Posts: 177

Re: ASCII Characters Display As ????

wiggly wrote:

if you want to see the C Source code i can show you, alough it was not really relevant except for the fact that it demenstrates my issue.

I know, it's just so that people could 1. test it themselves easily, 2. be sure that everyone is talking about the same thing. (In fact I have written the exact same program after reading your first post.)

wiggly wrote:

unless you include the extended characters that run from 128 - 255.

But those are not "real" ASCII — those are various encodings, and the interpretation of those characters depends on which encoding you want to use to read them. They are incompatible with UTF-8.

On my machine (running with i from 32 to 254):

$ ./a.out
180	´
181	µ
182	¶
183	·
184	¸
185	¹
186	º

But:

$ ./a.out | less
180     <B4>
181     <B5>
182     <B6>
183     <B7>
184     <B8>
185     <B9>
186     <BA>
$ ./a.out | iconv -t utf-8
128	iconv: illegal input sequence at position 662
$ ./a.out >a; file a
a: data

The question: why do you care about / want to use non-UTF-8 encodings?

Offline

#5 2013-10-05 13:50:01

wiggly
Member
Registered: 2012-02-02
Posts: 8

Re: ASCII Characters Display As ????

i would rather not see the question marks is all, also im sure somewhere it is linked to the squares i get for some text (i think some kind of arabic) that i see on web pages

Offline

#6 2013-10-05 13:59:22

msthev
Member
Registered: 2012-04-05
Posts: 177

Re: ASCII Characters Display As ????

Squares in web pages are typically caused by missing fonts (and some browser bugs, like Chromium not displaying CJK characters with DejaVu fonts while Firefox doing it without problems). There are articles in the Wiki about this. I'd suggest starting with installing ttf-droid package — these fonts fixed a lot of web browser issues for me.

Question marks: no idea why it happens and how to fix it. But you really shouldn't care — those characters are not valid UTF-8, and they should never be present in text files (unless you explicitly use an Extended ASCII encoding).
Anyway, maybe someone else will come with a direct solution.


Edit:
Or maybe you just don't have any TTF fonts installed?

$ pacman -Qsq ttf-font

Last edited by msthev (2013-10-05 14:02:39)

Offline

Board footer

Powered by FluxBB