You are not logged in.

#1 2018-08-25 11:42:59

leonixyz
Member
Registered: 2014-07-12
Posts: 38

Is my usb key corrupted?

I have a 30GB usb stick which was connected to a Raspberry Pi for around one year, serving as its primary storage device. The system was configured to boot directly from the usb key, having its / directory there. (I had also a 3TB disk attached and mounted to some subdirectory for torrenting, but that is another story).

Already from the very beginning, while installing the system (Arch Linux ARM) on the usb stick, I noticed that it randomly hanged for several seconds while doing simple operations, like for example executing some "new" command (i.e. a command which i did not executed before during that ssh session), or while using bash completion inside of directories that i did not browsed during the current ssh session.

I thought that the device, a relatively cheap one, might have some kind of problem, I did not had time to investigate, and except from these "freezes" everything worked fine: it did its job for around one year, hosting the / directory of my headless tiny-server (samba, rtorrent, etc).

After some time the freezes started becoming more frequent and lasting longer (at least this is what it seemed to me), and therefore I moved everything to another usb stick, which is working perfectly since that day.

Today I found the former usb stick in a drawer, and decided to try testing it to check whether it is really broken or not. According to dmesg it seem to have corrupted blocks, but badblocks tells everything is ok.

Could you please help me? The following command outputs have been abbreviated to only show relevant information.

Thanks in advance


# Note: the stick has a single partition formatted ext4

# Write a lot of files to device

$ sudo rsync -avviu web-client-angular/node_modules /mnt/test
sent 289,278,701 bytes  received 582,537 bytes  15,668,175.03 bytes/sec
total size is 287,208,882  speedup is 0.99

# Read a lot of files from device

$ sudo rsync -avviu /mnt/test /tmp/test
sent 289,277,255 bytes  received 582,397 bytes  115,943,860.80 bytes/sec
total size is 287,208,882  speedup is 0.99

# Write a single big .iso file

$ sudo rsync -avviu Downloads /mnt/test2 
sent 607,114,120 bytes  received 139 bytes  173,461,216.86 bytes/sec
total size is 606,965,760  speedup is 1.00

# Read a single big .iso file

$ sudo rsync -avviu /mnt/test2 /tmp/test2
sent 607,114,163 bytes  received 147 bytes  242,845,724.00 bytes/sec
total size is 606,965,760  speedup is 1.00

# Removing a lot of files

$ time sudo rm -fr /mnt/test
sudo rm -fr /mnt/test  0.04s user 0.55s system 0% cpu 1:22.12 total

# Removing a single big .iso file

$ time sudo rm -fr /mnt/test2 
sudo rm -fr /mnt/test2  0.01s user 0.02s system 0% cpu 37.707 total

# Check for bad blocks

$ sudo badblocks -v /dev/sdb1
[sudo] password for leonixyz: 
Checking blocks 0 to 31505407
Checking for bad blocks (read-only test): done                                                 
Pass completed, 0 bad blocks found. (0/0/0 errors)

# Check dmesg

$ dmesg | grep sdb | grep error | tail -n 50 
[  867.153733] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 10 writing to inode 784899 (offset 29360128 size 4194304 starting block 57088)
[  867.156309] EXT4-fs (sdb1): I/O error while writing superblock
[  867.156322] EXT4-fs error (device sdb1) in ext4_free_inode:355: IO failure
[  867.156385] EXT4-fs (sdb1): I/O error while writing superblock
[  867.176978] print_req_error: critical target error, dev sdb, sector 496688
[  867.179906] print_req_error: critical target error, dev sdb, sector 496928
[  867.182849] print_req_error: critical target error, dev sdb, sector 497168
[  867.260508] print_req_error: critical target error, dev sdb, sector 497408
[  867.261185] print_req_error: critical target error, dev sdb, sector 497648
[  867.264977] print_req_error: critical target error, dev sdb, sector 497664
[  867.267922] print_req_error: critical target error, dev sdb, sector 497904
[  867.270939] print_req_error: critical target error, dev sdb, sector 498144
[  872.149471] print_req_error: critical target error, dev sdb, sector 863168
[  872.152442] print_req_error: critical target error, dev sdb, sector 863408
[  872.155600] print_req_error: critical target error, dev sdb, sector 863648
[  872.158608] print_req_error: critical target error, dev sdb, sector 863888
[  872.161568] print_req_error: critical target error, dev sdb, sector 864128
[  872.164663] print_req_error: critical target error, dev sdb, sector 864368
[  872.167659] print_req_error: critical target error, dev sdb, sector 864608
[  872.170734] print_req_error: critical target error, dev sdb, sector 864848
[  872.173634] print_req_error: critical target error, dev sdb, sector 865088
[  872.176541] print_req_error: critical target error, dev sdb, sector 865328
[  872.187627] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 230686720 size 4194304 starting block 108288)
[  872.187635] Buffer I/O error on device sdb1, logical block 107520
[  872.187655] Buffer I/O error on device sdb1, logical block 107521
[  872.187663] Buffer I/O error on device sdb1, logical block 107522
[  872.187670] Buffer I/O error on device sdb1, logical block 107523
[  872.187678] Buffer I/O error on device sdb1, logical block 107524
[  872.187689] Buffer I/O error on device sdb1, logical block 107525
[  872.187697] Buffer I/O error on device sdb1, logical block 107526
[  872.187704] Buffer I/O error on device sdb1, logical block 107527
[  872.187711] Buffer I/O error on device sdb1, logical block 107528
[  872.187718] Buffer I/O error on device sdb1, logical block 107529
[  872.239645] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 230686720 size 4194304 starting block 108800)
[  872.299243] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 234881024 size 8388608 starting block 109312)
[  872.353563] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 234881024 size 8388608 starting block 109824)
[  872.408056] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 234881024 size 8388608 starting block 110336)
[  872.463578] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 234881024 size 8388608 starting block 110848)
[  872.521029] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 243269632 size 8388608 starting block 111360)
[  872.572316] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 243269632 size 8388608 starting block 111872)
[  872.629040] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 243269632 size 8388608 starting block 112384)
[  872.686775] EXT4-fs warning (device sdb1): ext4_end_bio:323: I/O error 5 writing to inode 784899 (offset 243269632 size 8388608 starting block 112896)
[  876.752576] JBD2: Detected IO errors while flushing file data on sdb1-8
[  876.753383] Buffer I/O error on dev sdb1, logical block 3702784, lost sync page write
[  876.757238] JBD2: Detected IO errors while flushing file data on sdb1-8
[  881.333774] print_req_error: critical target error, dev sdb, sector 2048
[  881.333795] EXT4-fs (sdb1): I/O error while writing superblock
[  881.333806] EXT4-fs error (device sdb1): ext4_journal_check_start:61: Detected aborted journal
[  881.334230] print_req_error: critical target error, dev sdb, sector 2048
[  881.334245] EXT4-fs (sdb1): I/O error while writing superblock

$ lsusb -vvv
Bus 001 Device 008: ID 18a5:0245 Verbatim, Ltd 
Couldn't open device, some information will be missing
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            0 
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0        64
  idVendor           0x18a5 Verbatim, Ltd
  idProduct          0x0245 
  bcdDevice           11.00
  iManufacturer           1 
  iProduct                2 
  iSerial                 3 
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0020
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              300mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval             255
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval             255



Edit: After all this testing I cannot mount the filesystem anymore, I cannot fdisk the device, and seems it is got definitely broken...

fdisk: cannot open /dev/sdb: Read-only file system

Last edited by leonixyz (2018-08-25 11:51:23)

Offline

#2 2018-08-25 12:05:02

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,177

Re: Is my usb key corrupted?

Do you have a proper power supply for your pi?

Offline

#3 2018-08-25 16:10:57

seth
Member
Registered: 2012-09-03
Posts: 9,423

Re: Is my usb key corrupted?

USB keys (like SD cards) are no SSDs - they've like no wear leveling.
This means that some blocks tend to get overused (in particular the superblocks, as you figured)
There're multiple ways to mitigate it, but they boil down to one: avoid writing onto the device.

You only performed a read test and reading is not where NAND memory breaks, it looses the ability to write. Always. (If the controller chip doesn't fail or the memory gets physically damaged, the data stays on in pretty much forever)

If you've valuable data on the key, you can try to mount the FS using an alternative superblock.
Otherwise you can run badblocks in rw mode to isolate the bad blocks and create a new filesystem, https://wiki.archlinux.org/index.php/Ba … ad_sectors to re-use the key.

However, since the wear is unpredictable, I would not trust that key w/ my life anymore.

Offline

Board footer

Powered by FluxBB