SATA hard drive stops working...

Zepp · 2008-08-03 20:20:53

The sata hard disk became inaccessible all of a sudden.

This is what I found in dmesg:

kjournald starting. Commit interval 5 seconds
EXT3 FS on sdb1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
warning: `smbd' uses 32-bit capabilities (legacy support in use)
EXT3-fs warning (device sdb1): empty_dir: bad directory (dir #60818152) - no `.' or `..'
hda-intel: Invalid position buffer, using LPIB read method instead.
ata3: exception Emask 0x10 SAct 0x0 SErr 0x90200 action 0xe frozen
ata3: irq_stat 0x00400000, PHY RDY changed
ata3: SError: { Persist PHYRdyChg 10B8B }
ata3: hard resetting link
ata3: SATA link down (SStatus 0 SControl 300)
ata3: failed to recover some devices, retrying in 5 secs
ata3: hard resetting link
ata3: SATA link down (SStatus 0 SControl 300)
ata3: failed to recover some devices, retrying in 5 secs
ata3: hard resetting link
ata3: SATA link down (SStatus 0 SControl 300)
ata3.00: disabled
sd 2:0:0:0: rejecting I/O to offline device
sd 2:0:0:0: [sdb] Result: hostbyte=0x01 driverbyte=0x00
end_request: I/O error, dev sdb, sector 304178327
sd 2:0:0:0: rejecting I/O to offline device
sd 2:0:0:0: rejecting I/O to offline device
sd 2:0:0:0: rejecting I/O to offline device
end_request: I/O error, dev sdb, sector 304178343
Buffer I/O error on device sdb1, logical block 38022285
lost page write due to I/O error on sdb1
Aborting journal on device sdb1.
sd 2:0:0:0: rejecting I/O to offline device
Buffer I/O error on device sdb1, logical block 38011454
lost page write due to I/O error on sdb1
EXT3-fs error (device sdb1) in ext3_reserve_inode_write: Journal has aborted
sd 2:0:0:0: rejecting I/O to offline device
Buffer I/O error on device sdb1, logical block 0
lost page write due to I/O error on sdb1
EXT3-fs error (device sdb1) in ext3_dirty_inode: Journal has aborted
------------[ cut here ]------------
WARNING: at fs/buffer.c:1183 mark_buffer_dirty+0x78/0x90()
Modules linked in: ext2 parport_pc ppdev lp parport serio_raw ppp_generic pcspkr psmouse k8temp i2c_piix4 i2c_core shpchp pci_hotplug sg thermal processor fan evdev button battery ac snd_hda_intel snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_hwdep snd_pcm_oss snd_pcm snd_timer snd_page_alloc snd_mixer_oss snd soundcore slhc rtc_cmos rtc_core rtc_lib ext3 jbd mbcache dm_mod usbhid hid ff_memless ehci_hcd sd_mod ohci_hcd usbcore atl1 mii ahci ata_generic pata_atiixp pata_acpi libata scsi_mod dock
Pid: 3158, comm: SABnzbd.py Not tainted 2.6.25-ARCH #1
Call Trace:
[<ffffffff8023a5f4>] warn_on_slowpath+0x64/0xb0
[<ffffffff8022f3b9>] enqueue_task+0x59/0x70
[<ffffffff802b3200>] filldir+0x0/0xf0
[<ffffffff880f6faf>] :ext3:ext3_count_free_inodes+0x3f/0x70
[<ffffffff802ccb78>] mark_buffer_dirty+0x78/0x90
[<ffffffff881018f7>] :ext3:ext3_commit_super+0x57/0xa0
[<ffffffff802b3200>] filldir+0x0/0xf0
[<ffffffff881034d2>] :ext3:ext3_handle_error+0x52/0xd0
[<ffffffff881035a6>] :ext3:__ext3_std_error+0x56/0x70
[<ffffffff802b3200>] filldir+0x0/0xf0
[<ffffffff88104fce>] :ext3:__ext3_journal_stop+0x3e/0x60
[<ffffffff802c7864>] __mark_inode_dirty+0x34/0x1d0
[<ffffffff802b348c>] vfs_readdir+0xdc/0xe0
[<ffffffff802b360f>] sys_getdents+0x9f/0xe0
[<ffffffff8020c5aa>] system_call_after_swapgs+0x8a/0x8f
---[ end trace 8502b8f5b8e06205 ]---
sd 2:0:0:0: rejecting I/O to offline device
Buffer I/O error on device sdb1, logical block 0
lost page write due to I/O error on sdb1
ext3_abort called.
EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only
ata3: EH complete
ata3.00: detaching (SCSI 2:0:0:0)
Buffer I/O error on device sdb1, logical block 120061954
lost page write due to I/O error on sdb1
sd 2:0:0:0: [sdb] Synchronizing SCSI cache
sd 2:0:0:0: [sdb] Result: hostbyte=0x04 driverbyte=0x00
sd 2:0:0:0: [sdb] Stopping disk
sd 2:0:0:0: [sdb] START_STOP FAILED
sd 2:0:0:0: [sdb] Result: hostbyte=0x04 driverbyte=0x00
EXT3-fs error (device sdb1): ext3_find_entry: reading directory #60030977 offset 0
EXT3-fs error (device sdb1): ext3_find_entry: reading directory #60030977 offset 0
EXT3-fs error (device sdb1): ext3_find_entry: reading directory #2 offset 0
EXT3-fs error (device sdb1): ext3_find_entry: reading directory #2 offset 0
Buffer I/O error on device sdb1, logical block 38011454
lost page write due to I/O error on sdb1

This is twice that this has happened now. Before it was running for about 20 days before this happened. It has only been 2 days now since then, and it appears to have happen again. I have the earlier /var/log/messages.log as well if needed.

Zepp · 2008-08-03 20:27:28

Hmm the /dev entry for the disk is gone now too.

JGC · 2008-08-03 20:29:11

What disk, what chipset and what's the quality of your sata cabling?
This could either be bad cabling, a dead disk, a shitty controler or a driver that doesn't work correctly.

Zepp · 2008-08-03 20:45:16

JGC wrote:

What disk, what chipset and what's the quality of your sata cabling?
This could either be bad cabling, a dead disk, a shitty controler or a driver that doesn't work correctly.

The disk is a seagate 500gb sata2. The machine is down at the moment and I am not at home to bring it back up to run lspci. But this is the motherboard specification with chipsets and such

http://www.ecs.com.tw/ECSWebSite/Produc … =1&LanID=0

As for the cables uhh there like the standard cables that came with the motherboard i think.

Last edited by Zepp (2008-08-03 20:47:48)

KimTjik · 2008-08-03 22:22:59

I know that this isn't of any help, but I've encountered a similar problem lately. This thread with no replies should have been updated to include some of the information you provide: http://bbs.archlinux.org/viewtopic.php?id=52354

I'm also using a Seagate, more precisely a Barracuda 7200.11 320GB. If the computer is left for a long time, for example over night, it now tends to freeze hard when I then start up some program. The output I get after a restart is pretty similar to yours. I haven't been able to figure out how to from a terminal check and correct the disk, but as can be seen by the thread above running the installation Arch-CD repairs the partitions.

I don't really use it as a server so I decided not to let it run inactive for such long periods of time. I've checked the hard-drive with SeaTools and it gives the drive green light. In rare occasions is doesn't load after GRUB (happened twice), but I don't know if that unresponsiveness has something in common with the freezing described above. I've checked stability of CPU and RAM as well and it does pass.

Another strange thing also happens if I first start up Fedora 7 64-bit on the system and then restart. The file system doesn't get corrupted, so that's good, but the boot process doesn't complete and then simply hangs with complaints and unfinished USB device configurations. That issue looks even more odd to me and I've never encountered such problems before.

I've no idea what the problem is. Could it even be a kernel issue, some new I/O instructions making my system "confused"? By the way the system in question has another chipset, a nVidia nForce 570. This is beyond the territory of my brain.

Last edited by KimTjik (2008-08-03 22:25:10)

lilsirecho · 2008-08-04 00:04:46

http://bbs.archlinux.org/viewtopic.php?id=52797

Zepp · 2008-08-04 00:25:20

lilsirecho wrote:

http://bbs.archlinux.org/viewtopic.php?id=52797

?

lilsirecho · 2008-08-04 00:28:32

Read that post, maybe?

Zepp · 2008-08-04 00:41:54

lilsirecho wrote:

Read that post, maybe?

I did, except it's a question with no real answers/information. Also this machine isn't a laptop and I did not configure any power saving features.

JGC · 2008-08-04 06:54:32

Seagate 7200.11 shouldn't be a problematic drive, I'm running 18 servers with those drives in a datacenter on Intel ICH9R controllers using AHCI. What driver does the ATI southbridge work with, does it use the ahci driver?
I would do a checkup on the cables, when I bought an Asrock board, the cables that came with it were so thin that SATA300 wasn't possible without corruption. The machine could work fine for hours, but after a few hours it would start resetting the SATA ports and after a while the disks wasn't even detected by BIOS anymore.

Zepp · 2008-08-04 07:02:50

They aren't 7200.11 they are 7200.10, little older. Still SATA2 though. The board uses an SB700 chipset, I am not sure what specific driver it uses offhand. The BIOS has the SATA interface in IDE mode as well at the moment, I was thinking maybe I should switch it into AHCI (it came in IDE mode, I only noticed it now when I was poking around). I will try different cables too if it fails again.

Zepp · 2008-08-04 07:05:59

This is the exact hard drive by the way, if it matters

http://www.seagate.com/ww/v/index.jsp?v … 04090aRCRD

JGC · 2008-08-04 08:22:59

We also have 25 servers running 7200.10 400GB drives running all in AHCI or SATA Enhanced mode on Intel ICH7/8 chipsets without problems, so the 7200.10 should be fine also. These machines run for over a year now.
Only problems I've seen with 7200.10 is the poor performance of the 3.AAK firmwares.

Zepp · 2008-08-04 08:27:20

ya i didn't have any problems with the hard drive before, it must be something with this computer. I just haven't figured out exactly what yet.

KimTjik · 2008-08-04 15:46:20

I did run both smartctl -t short and smartctl -t long and the drive passed. Still I had a lot of problem this morning get it up running, so something is definitely not alright. I suspect it's a mechanical error which affects the drives responsiveness. Nonetheless Raw_Read_Error_Rate and Hardware_ECC_Recovered shows some extremely high figures. Some say it might not by itself prove there's something wrong, but I start to wonder since it gives me so much trouble lately.

I'll contact Seagate to see how cooperative they are and what my chances are to use the guarantee (I mean in view of not being able to actually prove by tests that it's broken).

Diaz · 2008-08-04 16:08:25

you're scaring me, my hdds have been acting sometimes weird and are seagate 500G sata2 too, but the latest not the same as yours...

KimTjik · 2008-08-05 08:19:55

My hard drive is failing badly now. Like a miracle it did boot this morning, and I'm actually writing this from it, but yesterday evening I struggled to get it anywhere. Now it's time to make some crucial backups before it breaks into pieces, including some config files so I save time when the new hard drive arrives.

I've been experiencing one hard drive to slowly die on a Linux system, but this is my first with such erratic symptoms before any previous ones:
- doesn't do anything after grub
- kernel panic right after loading the kernel
- close to enter the final part of loading modules and suddenly different kinds of "Ooops: 000* *** FREEMPT SMP"
- similar to above but it ends with a "BUG: unable to handle kernel NULL pointer"
- or it freeze on checking a partition
- in the mix some kind of segmentation fault now and then as well

Just to make sure I let the computer run test # 5 of Memtest over night, because if the corruption had another cause, CPU or RAM, it should show (OK it's a RAM test, but a seriously faulty CPU would make eventually create errors as well). So here I am, but I hope Zepp has better luck with his drive.

KimTjik · 2008-08-08 08:17:54

I'm sorry to "high jack" your thread once more Zepp, but I have to report that in my case the motherboard proved to be the core of the problem. The hard drive is moved to another system and is performing well without any errors and file systems are intact.

Motherboard though - based on nVidia nForce 570 Ultra - has more or less died. I could run some tests on it before it started to really misbehave and the CPU and RAM are good and those instructions are executed normally, but communication with drives creates loads of errors. The chipset might have failed.

rooloo · 2008-08-08 17:00:22

KimTjik wrote:

I'm sorry to "high jack" your thread once more Zepp, but I have to report that in my case the motherboard proved to be the core of the problem. The hard drive is moved to another system and is performing well without any errors and file systems are intact.
Motherboard though - based on nVidia nForce 570 Ultra - has more or less died. I could run some tests on it before it started to really misbehave and the CPU and RAM are good and those instructions are executed normally, but communication with drives creates loads of errors. The chipset might have failed.

just to throw you another bone, I have seen power supply units cause such problems. Things like HD's spinning up put a serious strain on a failing PSU.

Don't under estimate the power of a failing power supply. It could empty your wallet.

KimTjik · 2008-08-08 17:32:06

Thanks for your concern roolo!

My background is in the OC camp spiced with an obsession about PSU quality, so don't get me started! . Of course I'll check this unit once more. It's a Silverstone Nightjar ST30NF 300W. It's an amazing unit actually. Rated at 300W it will without trouble give you 400W+ stable. The low rating is probably just a precaution since it's a fan-less unit. But even though it hasn't shown any signs of weakness who knows? The system in question has a pretty simple nVidia 7600GT, so the PSU isn't getting any real stress, especially in view of how seldom I use 3D applications like games.

I have troubleshoot different devices with another PS unit, a Tagan 2-Force II 600W and the issue with the motherboard is consistent. Let's see what the outcome will be, because besides that I'll have one Corsair 620HXEU, Fortron Epsilon and PC Power & Cooling S61EPS for testing! Something must work I hope. Seriously I hope I can avoid taking apart more computers to test everything.

If I'm mistaken about the cause of the trouble I'll give a report.

arew264 · 2008-08-11 02:50:56

To the OP, Zepp
I would blame the motherboard in your case as well. I've been building systems for a while, and I can conclusively tell you that ECS boards are SH*T. I still have one next to me that was taken out of the case. It was exchanged 8 times before I found a working one (my dad refused to get anything else because it was a "good deal"), and the SATA and onboard audio never worked from day 1. I almost think they must have ripped out quality control to cut prices.
They're in Fry's Electronics to lure people in, and the sales people generally recommend that you buy a more expensive board. It's the same trick as they use with the "Great Quality" computers, only they actually warn you not to buy those.

Unless you can find something else wrong, it's probably the mobo. Even if you do find something else, you want to get away from ECS.

Zepp · 2008-08-11 05:21:02

I had an old ecs board for my duron cpu and has lasted this long without giving me any troubles. I don't expect amazing quality or anything, but at least working would be nice. It has been running for over 6 days now with no signs of trouble from the sata controller or the hdd.

arew264 · 2008-08-11 18:41:00

You may be okay then. I know that a lot of stuff on an ECS board is generally DOA, but I haven't seen one fail while in use. I somehow got the impression that this mobo was new.
I would still get away from ECS, but if it works for you... enjoy it.

Zepp · 2008-08-11 20:48:16

arew264 wrote:

You may be okay then. I know that a lot of stuff on an ECS board is generally DOA, but I haven't seen one fail while in use. I somehow got the impression that this mobo was new.
I would still get away from ECS, but if it works for you... enjoy it.

it is newish. It's about 2 months old

Arch Linux

#1 2008-08-03 20:20:53

SATA hard drive stops working...

#2 2008-08-03 20:27:28

Re: SATA hard drive stops working...

#3 2008-08-03 20:29:11

Re: SATA hard drive stops working...

#4 2008-08-03 20:45:16

Re: SATA hard drive stops working...

#5 2008-08-03 22:22:59

Re: SATA hard drive stops working...

#6 2008-08-04 00:04:46

Re: SATA hard drive stops working...

#7 2008-08-04 00:25:20

Re: SATA hard drive stops working...

#8 2008-08-04 00:28:32

Re: SATA hard drive stops working...

#9 2008-08-04 00:41:54

Re: SATA hard drive stops working...

#10 2008-08-04 06:54:32

Re: SATA hard drive stops working...

#11 2008-08-04 07:02:50

Re: SATA hard drive stops working...

#12 2008-08-04 07:05:59

Re: SATA hard drive stops working...

#13 2008-08-04 08:22:59

Re: SATA hard drive stops working...

#14 2008-08-04 08:27:20

Re: SATA hard drive stops working...

#15 2008-08-04 15:46:20

Re: SATA hard drive stops working...

#16 2008-08-04 16:08:25

Re: SATA hard drive stops working...

#17 2008-08-05 08:19:55

Re: SATA hard drive stops working...

#18 2008-08-08 08:17:54

Re: SATA hard drive stops working...

#19 2008-08-08 17:00:22

Re: SATA hard drive stops working...

#20 2008-08-08 17:32:06

Re: SATA hard drive stops working...

#21 2008-08-11 02:50:56

Re: SATA hard drive stops working...

#22 2008-08-11 05:21:02

Re: SATA hard drive stops working...

#23 2008-08-11 18:41:00

Re: SATA hard drive stops working...

#24 2008-08-11 20:48:16

Re: SATA hard drive stops working...

Board footer