[Solved] Harddisks: lot of output in both dmesg and kernel.log

evert_ · 2011-01-24 18:38:54

Situation:

Since a few weeks I'm having a new little home server. An intel atom d525 cpu with 2gb ram, which seems to be a nice compromise between speed, price and power consumption. I'm wanting to use this server mainly as file server (backup, bulk data), download server, automatic backups, video streaming for the TV and my computer, mpd server and just for some experiments.

For the data I've got 2 disks, 2Tb each in a mdadm raid1 set with a layer of lvm and ext4 as the filesystem. I've used a brand new usb stick of 8Gb for the / filesystem. I know usb sticks are not made to be used as a / filesystem, but it should work and when the stick wears out (years?) I can replace it easily and cheap.

First signs of problems:

After the arch install I went ahead and installed some services. Everything seemed to be fine and I was really happy with the new server. Until after 2-3 days uptime it 'crashed'. I didn't bother looking in the logs or whatever, so I just gave it a fine reset and it went up without troubles. Again 2-3 days later it crashed again. I plugged in the monitor again and it seemed to be the / file system giving some troubles. After some io errors on the / it remounted itself read only, which gives me an unusable server. After some ext4 checks, it happened again the reboot after.

After that I decided it must be either the usb stick or the usb controller be doing something fishy. I've put the / on an old 2gb usb stick and since then it seems to be running fine (uptime now a weak).

Untill the / is full:

Strange, the 2gb / is full whilst a week ago when moving the arch install from the old usb stick, it only used about 1300Mb. After some smart usage of du and find I found out that the logs were to blame. Both the kernel.log en everything.log were 300+mb.

I hope there is somebody here who can help me figure out why the logs are getting filled with these messages and how to fix it. And to tell me if there is a serious problem with the raid1 set (mounted on /home), since I need to be able to rely on it to some extends.

raid set:

       0       8        1        0      active sync   /dev/sda1
       1       8       17        1      active sync   /dev/sdb1

/dev/sdc1 is the usb stick mounted on /

dmesg:

nmbd(11212): dirtied inode 801 (wins.dat.11212) on sdc1
nmbd(11212): dirtied inode 801 (wins.dat.11212) on sdc1
jbd2/sdc1-8(725): WRITE block 1584712 on sdc1
nmbd(11212): dirtied inode 72329 (?) on sdc1
flush-8:32(1309): WRITE block 3072016 on sdc1
jbd2/sdc1-8(725): WRITE block 1584720 on sdc1
jbd2/sdc1-8(725): WRITE block 1584728 on sdc1
jbd2/sdc1-8(725): WRITE block 1584736 on sdc1
jbd2/sdc1-8(725): WRITE block 1584744 on sdc1
jbd2/sdc1-8(725): WRITE block 1584752 on sdc1
jbd2/sdc1-8(725): WRITE block 1584760 on sdc1
jbd2/sdc1-8(725): WRITE block 1584768 on sdc1
jbd2/sdc1-8(725): WRITE block 1584776 on sdc1
jbd2/sdc1-8(725): WRITE block 1584784 on sdc1
jbd2/sdc1-8(725): WRITE block 1584792 on sdc1
jbd2/sdc1-8(725): WRITE block 1584800 on sdc1
syslog-ng(1224): dirtied inode 5027 (syslog-ng.persist) on sdc1
syslog-ng(1224): dirtied inode 5027 (syslog-ng.persist) on sdc1
jbd2/sdc1-8(725): WRITE block 1584808 on sdc1
flush-8:32(1309): WRITE block 3668432 on sdc1
flush-8:32(1309): WRITE block 3668440 on sdc1
flush-8:32(1309): WRITE block 3668448 on sdc1
flush-8:32(1309): WRITE block 3668456 on sdc1
flush-8:32(1309): WRITE block 3668464 on sdc1
flush-8:32(1309): WRITE block 3668472 on sdc1
flush-8:32(1309): WRITE block 3668480 on sdc1
flush-8:32(1309): WRITE block 3668488 on sdc1
flush-8:32(1309): WRITE block 3668496 on sdc1
syslog-ng(1224): dirtied inode 72847 (kernel.log) on sdc1
syslog-ng(1224): dirtied inode 72847 (kernel.log) on sdc1
flush-8:32(1309): WRITE block 3684888 on sdc1
flush-8:32(1309): WRITE block 3684896 on sdc1
flush-8:32(1309): WRITE block 3684904 on sdc1
flush-8:32(1309): WRITE block 3684912 on sdc1
flush-8:32(1309): WRITE block 3684920 on sdc1
flush-8:32(1309): WRITE block 3684928 on sdc1
flush-8:32(1309): WRITE block 3684936 on sdc1
flush-8:32(1309): WRITE block 3684944 on sdc1
flush-8:32(1309): WRITE block 3684952 on sdc1
flush-8:32(1309): WRITE block 1309280 on sdc1
flush-8:32(1309): WRITE block 1136 on sdc1
syslog-ng(1224): dirtied inode 72851 (everything.log) on sdc1
syslog-ng(1224): dirtied inode 72851 (everything.log) on sdc1
syslog-ng(1224): dirtied inode 72851 (everything.log) on sdc1
dmesg(11213): READ block 265816 on sdc1

kernel.log

Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407424 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407432 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407440 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407448 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407456 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407464 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407472 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407480 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407488 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407496 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407504 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407512 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407520 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407528 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407536 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407544 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407552 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407560 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407568 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407576 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407584 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407592 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407600 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407608 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407616 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407624 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407632 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407640 on dm-0
Jan 24 19:34:58 nas kernel: python2(1584): WRITE block 2223407648 on dm-0
Jan 24 19:34:58 nas kernel: python2(1582): dirtied inode 40370181 (sabnzbd.log) on dm-0
Jan 24 19:34:58 nas kernel: python2(1582): dirtied inode 40370181 (sabnzbd.log) on dm-0
Jan 24 19:34:58 nas kernel: python2(1582): dirtied inode 40370181 (sabnzbd.log) on dm-0
Jan 24 19:34:58 nas kernel: python2(1582): dirtied inode 46139255 (flying.wild.alaska.s01e02.720p.hdtv.x264-momentum.part18.rar) on dm-0
Jan 24 19:34:58 nas kernel: python2(1582): dirtied inode 46139255 (flying.wild.alaska.s01e02.720p.hdtv.x264-momentum.part18.rar) on dm-0
Jan 24 19:34:58 nas kernel: python2(1582): dirtied inode 46139255 (flying.wild.alaska.s01e02.720p.hdtv.x264-momentum.part18.rar) on dm-0
Jan 24 19:34:58 nas kernel: md127_raid1(1074): WRITE block 8 on sdb1
Jan 24 19:34:58 nas kernel: md127_raid1(1074): WRITE block 8 on sda1
Jan 24 19:34:59 nas kernel: bash(11231): READ block 916240 on sdc1
Jan 24 19:34:59 nas kernel: tail(11231): READ block 916272 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 3692640 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 3709104 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588128 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588136 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588144 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588152 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588160 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588168 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588176 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588184 on sdc1
Jan 24 19:34:59 nas kernel: jbd2/sdc1-8(725): WRITE block 1588192 on sdc1

Especially the lines with 'dirtied inode' do worry me .

Ps: I do know that running archlinux on a server is not 'smart'. Same for using a usb flash device for /. Both are not considered smart choices, but for a home server which doesn't need to have a 100% uptime not always a bad choice and are not the topic of discussion .

Last edited by evert_ (2011-01-24 21:21:55)

Mr Green · 2011-01-24 18:49:34

The only way I can see you reducing problems is to move high read/write directories to a normal drive to reduce wear on your usb drive

Someone may jump in on this one but you may have more luck with a CF adaptor based system [internal]

evert_ · 2011-01-24 18:51:00

There are both log entries about the flash disk and the raid 1 set. The flash disk (sdc) entries don't seem to be critical, but I have no clue why they appeared...

I make a regular backup of my / to protect myself against loss of data due to wear some day. I've ran a little server in the past on a 4gb usb flash medium for several years, without any problems what so ever.

Last edited by evert_ (2011-01-24 18:52:54)

djgera · 2011-01-24 18:54:02

All is normal output. Seems that something that is running on your system, enabled block_dump (/proc/sys/vm/block_dump with 1) . Just disble it (with 0).

evert_ · 2011-01-24 21:21:16

Thanks djgera. I really do hope that you are sure all that output is normal. I set the block_dump to 0 and now the kernel.log is pretty silent since then. Thanks for the great and fast help!

Arch Linux

#1 2011-01-24 18:38:54

[Solved] Harddisks: lot of output in both dmesg and kernel.log

#2 2011-01-24 18:49:34

Re: [Solved] Harddisks: lot of output in both dmesg and kernel.log

#3 2011-01-24 18:51:00

Re: [Solved] Harddisks: lot of output in both dmesg and kernel.log

#4 2011-01-24 18:54:02

Re: [Solved] Harddisks: lot of output in both dmesg and kernel.log

#5 2011-01-24 21:21:16

Re: [Solved] Harddisks: lot of output in both dmesg and kernel.log

Board footer