Arch Linux

kokoko3k

Member

Registered: 2008-11-14

Posts: 2,394

[solved] ntfs-3g: Invalid or incomplete multibyte or wide character

Hi all, i've a problem with an ntfs partition and (i think) locales.

#nickinicki-laptop koko # touch /mnt/vista/ò
touch: cannot touch `/mnt/vista/ò': Invalid or incomplete multibyte or wide character

nickinicki-laptop koko # grep -v \# /etc/locale.gen
en_US.UTF-8 UTF-8
en_US ISO-8859-1
es_ES.UTF-8 UTF-8
es_ES ISO-8859-1
es_ES@euro ISO-8859-15
it_IT.UTF-8 UTF-8
it_IT ISO-8859-1
it_IT@euro ISO-8859-15
pt_BR.UTF-8 UTF-8
pt_BR ISO-8859-1



nickinicki-laptop koko # locale -a
C
en_US
en_US.iso88591
en_US.utf8
es_ES
es_ES@euro
es_ES.iso88591
es_ES.iso885915@euro
es_ES.utf8
italian
it_IT
it_IT@euro
it_IT.iso88591
it_IT.iso885915@euro
it_IT.utf8
POSIX
pt_BR
pt_BR.iso88591
pt_BR.utf8
spanish


nickinicki-laptop koko # locale
LANG=it_IT.iso885915@euro
LC_CTYPE="it_IT.iso885915@euro"
LC_NUMERIC="it_IT.iso885915@euro"
LC_TIME="it_IT.iso885915@euro"
LC_COLLATE=C
LC_MONETARY="it_IT.iso885915@euro"
LC_MESSAGES="it_IT.iso885915@euro"
LC_PAPER="it_IT.iso885915@euro"
LC_NAME="it_IT.iso885915@euro"
LC_ADDRESS="it_IT.iso885915@euro"
LC_TELEPHONE="it_IT.iso885915@euro"
LC_MEASUREMENT="it_IT.iso885915@euro"
LC_IDENTIFICATION="it_IT.iso885915@euro"
LC_ALL=

nickinicki-laptop koko # grep -i locale /etc/rc.conf
# LOCALE: available languages can be listed with the 'locale -a' command
LOCALE="it_IT.iso885915@euro"

nickinicki-laptop koko # mount.ntfs-3g  -o locale=$LANG /dev/sda1 /mnt/vista

nickinicki-laptop koko # touch /mnt/vista/ò
touch: cannot touch `/mnt/vista/ò': Invalid or incomplete multibyte or wide character

nickinicki-laptop koko # tail /var/log/messages.log
Aug 27 18:45:57 nickinicki-laptop ntfs-3g[3660]: Version 2009.4.4 integrated FUSE 27
Aug 27 18:45:57 nickinicki-laptop ntfs-3g[3660]: Mounted /dev/sda1 (Read-Write, label "VISTA", NTFS 3.1)
Aug 27 18:45:57 nickinicki-laptop ntfs-3g[3660]: Cmdline options: locale=it_IT.iso885915@euro
Aug 27 18:45:57 nickinicki-laptop ntfs-3g[3660]: Mount options: silent,allow_other,nonempty,relatime,fsname=/dev/sda1,blkdev,blksize=4096
Aug 27 18:46:19 nickinicki-laptop ntfs-3g[3660]: Could not convert filename to Unicode: 'ò': Invalid or incomplete multibyte or wide character
Aug 27 18:46:19 nickinicki-laptop ntfs-3g[3660]: Could not convert filename to Unicode: 'ò': Invalid or incomplete multibyte or wide character

Does anybody has a clue on what's happening?

Thanks in advance!

Last edited by kokoko3k (2009-08-28 16:25:21)

---

Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

kokoko3k

Member

Registered: 2008-11-14

Posts: 2,394

Re: [solved] ntfs-3g: Invalid or incomplete multibyte or wide character

Oh well, i workarounded the problem by rolling back my ntfs-3g version.
The last one which correctly works for me (with "strange" characters) is ntfs-3g 1.5130-1:

nickinicki-laptop pkg # LC_ALL=en_US pacman -U ./ntfs-3g-1.5130-1-i686.pkg.tar.gz
loading package data...
checking dependencies...
(1/1) checking for file conflicts                   [################################################################] 100%
(1/1) upgrading ntfs-3g                             [################################################################] 100%
nickinicki-laptop koko # mount.ntfs-3g  -o locale=$LANG /dev/sda1 /mnt/vista
nickinicki-laptop koko # touch /mnt/vista/ò
nickinicki-laptop koko # ls -la /mnt/vista/ò
-rwxrwxrwx 1 root root 0 27 ago 20:18 /mnt/vista/ò

..Which is what i expected to see from the latest version too (now ntfs-3g 2009.4.4-1).

It is true that (due to bad new intel video drivers and some kernel problems) i have some ignorepkgs in my pacman.conf:

IgnorePkg   = amarok
IgnorePkg   = kdemod3-ksplash-engine-flexible
IgnorePkg   = kernel26
IgnorePkg   = kernel-headers
IgnorePkg   = kernel26-firmware
IgnorePkg   = klibc
IgnorePkg   = klibc-extras
IgnorePkg   = klibc-kbd
IgnorePkg   = klibc-module-init-tools
IgnorePkg   = klibc-udev
IgnorePkg   = intel-dri
IgnorePkg   = libdrm
IgnorePkg   = libgl
IgnorePkg   = libxfont
IgnorePkg   = mesa
IgnorePkg   = pixman
IgnorePkg   = synaptics
IgnorePkg   = xf86-input-evdev
IgnorePkg   = xf86-input-keyboard
IgnorePkg   = xf86-input-mouse
IgnorePkg   = xf86-video-intel
IgnorePkg   = xkeyboard-config
IgnorePkg   = xorg-server
IgnorePkg   = glibc
IgnorePkg   = suspend
IgnorePkg   = binutils
IgnorePkg   = coreutils
IgnorePkg   = gcc
IgnorePkg   = gcc-libs
IgnorePkg   = mkinitcpio
IgnorePkg   = cryptsetup
IgnorePkg   = xf86-video-vesa
IgnorePkg   = xorg-utils
IgnorePkg   = xorg-font-utils
IgnorePkg   = xorg-fonts-encodings
IgnorePkg   = xorg-apps
IgnorePkg   = xorg-fonts-alias
IgnorePkg   = xorg-fonts-100dpi
IgnorePkg   = xorg-fonts-misc
IgnorePkg   = xorg-res-utils
IgnorePkg   = xorg-server-utils
IgnorePkg   = xorg-twm
IgnorePkg   = xorg-xinit
IgnorePkg   = xorg-xkb-utils
IgnorePkg   = xf86dgaproto
IgnorePkg   = libxxf86dga
IgnorePkg   = xf86miscproto
IgnorePkg   = libxxf86misc
IgnorePkg   = xf86vidmodeproto
IgnorePkg   = xproto
IgnorePkg   = recordproto
IgnorePkg   = fixesproto
IgnorePkg   = renderproto
IgnorePkg   = xineramaproto
IgnorePkg   = damageproto
IgnorePkg   = compositeproto
IgnorePkg   = dmxproto
IgnorePkg   = fontcacheproto
IgnorePkg   = fontsproto
IgnorePkg   = glproto
IgnorePkg   = kbproto
IgnorePkg   = trapproto
IgnorePkg   = videoproto
IgnorePkg   = libice
IgnorePkg   = xtrans
IgnorePkg   = libxt
IgnorePkg   = libxmu
IgnorePkg   = libxtst
IgnorePkg   = libxfixes
IgnorePkg   = libxrender
IgnorePkg   = libxcursor
IgnorePkg   = libxinerama
IgnorePkg   = libxvmc
IgnorePkg   = libxslt
IgnorePkg   = libxdamage
IgnorePkg   = libxres
IgnorePkg   = libxcomposite
IgnorePkg   = libxaw
IgnorePkg   = libxdmcp
IgnorePkg   = libxfontcache
IgnorePkg   = libxkbui
IgnorePkg   = plotutils
IgnorePkg   = xterm
IgnorePkg   = mplayer
IgnorePkg   = directfb #(oppure ricompilare mplayer)
IgnorePkg   = gsynaptics

But if no one faced my same problem, the only thing i could think about is about some package version dependancy issue, even if pacman never complained about ntfs-3g upgrades.

Time for a IgnorePkg = ntfs-3g ?
...any hint?

Last edited by kokoko3k (2009-08-27 18:21:46)

---

Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

kokoko3k

Member

Registered: 2008-11-14

Posts: 2,394

Re: [solved] ntfs-3g: Invalid or incomplete multibyte or wide character

I know i'm writing this thread alone, but i'm experimenring more and more.
I was able to reproduce the issue even on an totally up-to-date system (every package upgraded to the latest version) using an ntfs filesystem on a file image:

Gozer ~ # cd /var/cache/pacman/pkg

########### Install latest ntfs-3g:

Gozer pkg # pacman -U ./ntfs-3g-2009.4.4-1-i686.pkg.tar.gz
loading package data...
checking dependencies...
(1/1) checking for file conflicts                   [########################################################] 100%
(1/1) upgrading ntfs-3g                             [########################################################] 100%

########### Make a 100MB file:

Gozer pkg # dd if=/dev/zero of=/tmp/image.ntfs bs=1024 count=100000
100000+0 records in
100000+0 records out
102400000 bytes (102 MB) copied, 0.899782 s, 114 MB/s

########### force mkfs.ntfs to make an ntfs filesystem there:

Gozer pkg # mkfs.ntfs /tmp/image.ntfs -F
/tmp/image.ntfs is not a block device.
mkntfs forced anyway.
The sector size was not specified for /tmp/image.ntfs and it could not be obtained automatically.  It has been set to 512 bytes.
The partition start sector was not specified for /tmp/image.ntfs and it could not be obtained automatically.  It has been set to 0.
The number of sectors per track was not specified for /tmp/image.ntfs and it could not be obtained automatically.  It has been set to 0.
The number of heads was not specified for /tmp/image.ntfs and it could not be obtained automatically.  It has been set to 0.
Cluster size has been automatically set to 4096 bytes.
To boot from a device, Windows needs the 'partition start sector', the 'sectors per track' and the 'number of heads' to be set.
Windows will not be able to boot from this device.
Initializing device with zeroes: 100% - Done.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.

########### My locales:

Gozer pkg # locale
LANG=en_US.iso88591
LC_CTYPE="en_US.iso88591"
LC_NUMERIC="en_US.iso88591"
LC_TIME="en_US.iso88591"
LC_COLLATE="en_US.iso88591"
LC_MONETARY="en_US.iso88591"
LC_MESSAGES="en_US.iso88591"
LC_PAPER="en_US.iso88591"
LC_NAME="en_US.iso88591"
LC_ADDRESS="en_US.iso88591"
LC_TELEPHONE="en_US.iso88591"
LC_MEASUREMENT="en_US.iso88591"
LC_IDENTIFICATION="en_US.iso88591"
LC_ALL=en_US.iso88591

########### Make the mountpoint and load the loop module 

Gozer pkg # mkdir /mnt/ntfs.test
Gozer pkg # modprobe loop

########### mount the filesystem through ntfs-3g-2009.4.4-1

Gozer pkg # ntfs-3g -o loop,locale=$LANG /tmp/image.ntfs /mnt/ntfs.test/

########### Try to create a file with accented letters and get the error:

Gozer pkg # touch /mnt/ntfs.test/àèìòu
touch: cannot touch `/mnt/ntfs.test/àèìòu': Invalid or incomplete multibyte or wide character
Gozer pkg # ls -la /mnt/ntfs.test/àèìòu
ls: cannot access /mnt/ntfs.test/àèìòu: Invalid or incomplete multibyte or wide character

########### umount the filesystem 

Gozer pkg # umount /mnt/ntfs.test

########### Get the outdated ntfs-3g package from arch rollback machine repo and install it

Gozer pkg # wget http://arm.kh.nu/extra::2009-1-1/os/i686/ntfs-3g-1.5130-1-i686.pkg.tar.gz
--2009-08-28 09:39:35--  http://arm.kh.nu/extra::2009-1-1/os/i686/ntfs-3g-1.5130-1-i686.pkg.tar.gz
Resolving arm.kh.nu... 75.119.207.164
Connecting to arm.kh.nu|75.119.207.164|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 360413 (352K) [application/x-tar]
Saving to: `ntfs-3g-1.5130-1-i686.pkg.tar.gz'

100%[=========================================================================>] 360,413      277K/s   in 1.3s

2009-08-28 09:39:36 (277 KB/s) - `ntfs-3g-1.5130-1-i686.pkg.tar.gz' saved [360413/360413]

Gozer pkg # pacman -U ./ntfs-3g-1.5130-1-i686.pkg.tar.gz
loading package data...
checking dependencies...
(1/1) checking for file conflicts                   [########################################################] 100%
(1/1) upgrading ntfs-3g                             [########################################################] 100%

########### Mount the filesystem again with ntfs-3g-1.5130-1

Gozer pkg # ntfs-3g -o loop,locale=$LANG /tmp/image.ntfs /mnt/ntfs.test/

########### Try to create a file with accented letters:

Gozer pkg # touch /mnt/ntfs.test/àèìòu
Gozer pkg # ls -la /mnt/ntfs.test/àèìòu
-rwxrwxrwx 1 root root 0 2009-08-28 09:41 /mnt/ntfs.test/àèìòu

...This time everything worked as expected!

Can someone try all this and reports his results here please?
Thanks.

Last edited by kokoko3k (2009-08-28 07:49:38)

---

Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

thisoldman

Member

From: Pittsburgh

Registered: 2009-04-25

Posts: 1,172

Re: [solved] ntfs-3g: Invalid or incomplete multibyte or wide character

A similar situation is being discussed in this thread: Nonlatin characters in URLs;NTFS Invalid or incomplete multibyte error. Please join us there. Your experience with ntfs-3g is helpful to solving this problem.

lolilolicon

Member

Registered: 2009-03-05

Posts: 1,722

Re: [solved] ntfs-3g: Invalid or incomplete multibyte or wide character

Hi, kokoko3k.
I use hal and pcmanfm to mount ntfs drives. Here are some info that might be helpful.

$ touch àèìòu ; ls -l  àèìòu 
-rw-rw-r-- 1 canti users 0 2009-08-28 18:29 àèìòu

$ pacman -Q ntfs-3g
ntfs-3g 2009.4.4-1
$ mount
...
/dev/sdc5  on  /media/20G   type  fuseblk  (rw,nosuid,nodev,noatime,allow_other,default_permissions,blksize=4096)
...

#/usr/share/pcmanfm/mount.rules
...
[ntfs-3g]
mount_options=locale=zh_CN.UTF-8;uid=1000;gid=100;fmask=0113;dmask=0002;noatime;exec
...

#/usr/share/hal/fdi/policy/10osvendor/20-ntfs-config-write-policy.fdi
<?xml version="1.0" encoding="UTF-8"?> 
<deviceinfo version="0.2">
   <device>
       <match key="volume.fstype" string="ntfs">
           <merge key="volume.fstype" type="string">ntfs-3g</merge>
           <merge key="volume.policy.mount_filesystem" type="string">ntfs-3g</merge>
           <append key="volume.mount.valid_options" type="strlist">locale=</append>
           <append key="volume.mount.valid_options" type="strlist">dmask=</append>
           <append key="volume.mount.valid_options" type="strlist">fmask=</append>
           <append key="volume.mount.valid_options" type="strlist">uid=</append>
           <append key="volume.mount.valid_options" type="strlist">gid=</append>
       </match>
   </device>
</deviceinfo>

Last edited by lolilolicon (2009-08-28 10:38:08)

---

This silver ladybug at line 28...

kokoko3k

Member

Registered: 2008-11-14

Posts: 2,394

Re: [solved] ntfs-3g: Invalid or incomplete multibyte or wide character

It seems that utf8 is the answer (meantime i readed something about on ntfs-3g forums)
thisoldman: thank you for the link and sorry for having missed it, i will continue to post there.
--edit--

Right, setting LOCALE="it_IT.utf8" in /etc/rc.conf, rebooting and upgrading again ntfs-3g solved the issue.

Last edited by kokoko3k (2009-08-28 16:27:46)

---

Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !