Arch Linux

チルノ

Guest

Programs experiencing errors due to filenames

Hello there.

I purchased a digital album and got a ZIP download from the site. Trying to extract it turned out to be tricky, though. I'm using KDE and its host of programs, and Ark was giving me trouble. It would extract folders, but they'd be empty. It could not extract any files, nor could I open them directly.

After finally managing to get them out with unzip, I had additional problems playing the files with VLC, see video below. It refused to play the files, until I changed the filename, at which point it worked no problem.

Here is a video showing everything related to the problem:
https://cdn.fosstodon.org/media_attachm … 2e3ee5.mp4
(Using my Mastodon instance to host the video because I don't want to make a Google account just to post this on YouTube)

As you can see, I am able to get my files out and working, but whatever the default filenames are, they seem to create problems for both Ark and VLC. So I wonder, just what is it about them that's causing the issue? If I can know more, I'd like to file relevant bug reports if necessary.

EDIT: I just noticed that, if I delete the files without renaming them, I am unable to empty the trash, as Dolpin throws errors akin to "file does not exist".

Last edited by チルノ (2021-08-29 17:48:43)

mpan

Member

Registered: 2012-08-01

Posts: 1,639

Website

Re: Programs experiencing errors due to filenames

It seems that whoever has created that ZIP was using poorly written software, that can’t properly store file names.

ZIP is an archaic file format, that was never designed for portability between different platforms. Initially it stored memory dumps of the file names. While some efforts were made to modernize it, many pieces of software still struggle to support even the basic things. Judging by the file names, the author of the ZIP file was using UTF-16, UCS-2, CP-932, Shift-JIS or EUC-JP on their system and the program simply dumped that into the ZIP file. Which means you are receiving complete garbage.

Possible solutions I see, though half of that are literally blind guesses:

• Urge the author to notice 1990s are over and switch from ZIP to something like 7-zip.

• Urge the author to update their software to something that is not 15 years after the current format specification.

• Use command-line unzip, setting LANG=C in an attempt to force unzip to still produce garbage (probably a bunch of replacement characters), but at least garbage that can be used.

• See if 7zip is able to extract that into something useful. Unlikely, but worth trying.

• Uncomment the “ja_JP.EUC-JP” locale in “/etc/locale.gen” and regenerate locales. Run unzip with that locale — some chances are it will extract the file names properly.

• Write your own program that directly deals with ZIP and replaces file names before extracting them.

• Switch to the operating system used by the author.

File names are one thing. But you will probably have ID3 tags broken in a similar way.

--- edit
As for writing your own program, you may see Python’s zipfile, the ZipFile.infolist method and ZipInfo objects produced by that method with ZipInfo.filename containing the name. Unfortunately I can’t test that idea myself, as I do not have a broken file like that. There is a StackOverflow thread that deals with renaming files extracted using zipfile: that is basicaly what I mean.

--- edit
See Hideaki Takahashi’s code unzipping a zip file with non-utf8 encoding.

Last edited by mpan (2021-09-02 06:29:44)

---

Paperclips in avatars?