Arch Linux

helq

Member

Registered: 2012-09-09

Posts: 10

Firefox can't save files when using non-ASCII characters [solved]

Hi guys, I have a situation with firefox that I can't resolve, maybe you could help me.

Firefox seems to have some trouble when downloading any file which contains non-ASCII characters.

How to reproduce:

• Hit Ctrl+s to save this page in your disk, and then put any name with any non-ASCII character: For example: 'nombre_en_español.htm', or 'eine_Übung.htm', or '我不说中文.htm' (actually, this last one doesn't work at all, the download process fail)

• After the download, I go to the download file's location and I find two files, one with the right name (and empty) and other with the non-ASCII characters swapped for  (non-empty, containing the page). For example:

  > ls
  ...
  -rw-r--r-- 1 helq users  16341 Nov  1 18:03 eine_?bung.htm
  -rw-r--r-- 1 helq users      0 Nov  1 18:03 eine_Übung.htm
  ...

Examining the names carefully (I save the first: ls > names.txt, and then opened them with an hex editor), I could identify that the characters '?' are the characters original characters 'ñ' or 'ß' encoded in ISO8859-1 (in hex: DC and DF, which are non-displayable utf-8 characters, and though the ? simbol).

So, why is firefox saving files with weird names?, why is it using utf-8 and ISO8859-1 at the same time?, Isn't possible to files with utf-8 names?

PS: I'm using kde and my locale configuration is:

> cat /etc/locale.gen
en_GB.UTF-8 UTF-8  
en_GB ISO-8859-1  
es_CO.UTF-8 UTF-8  
es_CO ISO-8859-1

> cat /etc/locale.conf
LANG=en_GB.UTF-8

Last edited by helq (2015-11-02 00:57:53)

Ropid

Member

Registered: 2015-03-09

Posts: 1,069

Re: Firefox can't save files when using non-ASCII characters [solved]

It works for me. I used your "我不说中文.htm" test name. Firefox does not create empty files for me. I'm using Firefox v43 right now.

$ ls -1 --size --human ~/Downloads/test
total 16K
  0 我不说中文_files
16K 我不说中文.htm

$ locale
LANG=en_IE.utf8
...

EDIT: I now tried it in Firefox v41.0.2 and I used Ctrl-S like you described. I tried the file name with the ü and the one with .jp characters. Things still work correctly for me:

$ ls
eine_Übung.htm  我不说中文.htm

Last edited by Ropid (2015-11-02 00:24:03)

helq

Member

Registered: 2012-09-09

Posts: 10

Re: Firefox can't save files when using non-ASCII characters [solved]

Oh, I get it now. I launched firefox with an script, and the script was setting the variable LC_ALL to C:

export LC_ALL=C

And, I suppose the C encoding is equivalent to ISO8859-1, and though the weird error.

So, the solution was to change that line in the script for the respecting one in /etc/locale.conf

Thanks, @Ropoid, without the locale "function", I wouldn't be able to find the problem