You are not logged in.
- Topics: Active | Unanswered
#1 2020-12-20 13:00:28
- dviktor
- Member
- From: Moscow
- Registered: 2015-10-18
- Posts: 162
How to intercept constant disk access in RAID/dm-crypt setup?
I have two ST1000NM0008 drives in RAID 1 setup; also I've enrolled dm-crypt with LUKS 2 over that array. My plan is to use this storage as a weekly backup. As for now I haven't set up any backups yet so my drives are still doing nothing.
What concerns me is that someone constatly accesses them. At first I've heard heads parking/unparking events and rechecked SMART info - and yes, Load Cycle Count parameter has been increasing with every parking event. As these drives uses some kind of aggressive APM mode I've just disabled it via SeaChest tool to prolong heads machinery life. But this is a workaround, really. Also sometimes I can still hear disk access event so I want to know what can cause it. Because my plan is to use disk array as a weekly backup storage I only want it to be accessed once a week so the rest of the time drives can be spunned down with heads parked.
So if there any ways to monitor disk accesses exist?
Offline
#2 2020-12-20 18:24:17
- nomorewindows
- Member
- Registered: 2010-04-03
- Posts: 3,534
Re: How to intercept constant disk access in RAID/dm-crypt setup?
iotop for disk access
nethogs for network activitiy
I may have to CONSOLE you about your usage of ridiculously easy graphical interfaces...
Look ma, no mouse.
Offline
#3 2020-12-20 19:06:35
- frostschutz
- Member
- Registered: 2013-11-15
- Posts: 1,647
Re: How to intercept constant disk access in RAID/dm-crypt setup?
also /proc/sys/vm/block_dump but it will fill your syslog
there are plenty of things that trigger (read) access to all drives. searching for new/missing UUIDs is one...
some drives even consider smart queries to be disk accesses, my WD80EZAZ drives wake from standby if you smartctl -a ... very annoying, and those accesses don't show up in I/O logs since it's not really I/O... have to work around with -n standby option so smartd will leave drives alone
drives also do periodic scans by themselves (auto offline etc.), then ext4 has lazy initialization so it keeps drives busy for a while... for the sake of shaving a few seconds off the mkfs.
it can be difficult to track down what exactly causes things. since the drive itself won't tell you
Offline
#4 2020-12-24 19:30:20
- dviktor
- Member
- From: Moscow
- Registered: 2015-10-18
- Posts: 162
Re: How to intercept constant disk access in RAID/dm-crypt setup?
@nomorewindows I know about iotop but it's a bit inconvenient because I can't stare all the time into it to catch spontaneous (and pretty rare) disk access. or may be I don't know some crucial ways to do that with the help of iotop
@frostschutz thank you for the tip. I have smartd configured for periodical polling:
DEFAULT -o on -S on -a -I 190 -I 194 -s (S/../.././23|L/../../7/03) -m E-MAIL -M exec /usr/local/bin/smartdnotify
/dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_S3NXNF0J816592E -W 3,40,44
/dev/disk/by-id/ata-Samsung_SSD_860_EVO_1TB_S3YBNB0NA22225A -W 3,40,44
/dev/disk/by-id/ata-ST1000NM0008-2F2100_ZFA19JG2 -W 3,36,40
/dev/disk/by-id/ata-ST1000NM0008-2F2100_ZFA19JBM -W 3,36,40I think it can cause disk pings. Also I've disabled ext4 lazy init as soon as I've assembled RAID1 because noise from HDD was awful.
Offline
#5 2020-12-25 07:45:26
- cdwijs
- Member
- Registered: 2010-04-24
- Posts: 310
Re: How to intercept constant disk access in RAID/dm-crypt setup?
I use dstat. it can show io to each drive individually, system load, and network load:
dstat -tnN total -c -dD sda,sdc,sdb,sde --nocolor
----system---- -net/total- --total-cpu-usage-- --dsk/sda-- --dsk/sdb-- --dsk/sdd--
time | recv send|usr sys idl wai stl| read writ: read writ: read writ
25-12 06:54:11| 0 60B| 5 16 73 6 0| 0 456k: 88k 456k: 0 58M
25-12 06:54:12| 0 60B| 5 6 72 17 0| 48k 0 : 0 0 :4096B 97M
25-12 06:54:13| 0 60B| 5 13 70 12 0| 0 0 : 0 0 : 16k 105M
25-12 06:54:14| 0 60B| 5 15 76 3 0| 84k 0 : 0 0 : 0 44M
25-12 06:54:15| 0 0 | 6 14 69 11 0| 0 0 : 48k 0 : 0 100M
25-12 06:54:16| 0 60B| 6 11 73 10 0| 0 0 : 0 0 :8192B 101M
25-12 06:54:17| 0 60B| 5 13 72 10 0| 0 0 : 0 0 : 0 105M
25-12 06:54:18| 0 60B| 5 9 75 11 0| 0 0 : 0 0 : 0 109M
25-12 06:54:19| 0 60B| 3 6 78 13 0| 20k 0 : 0 0 : 16k 82M
25-12 06:54:20| 0 60B| 5 17 68 11 0| 24k 0 :7900k 0 : 0 105M
25-12 06:54:21| 0 60B| 6 19 66 9 0| 36k 0 : 0 0 : 0 102M
25-12 06:54:22| 0 60B| 5 11 74 10 0| 0 0 : 0 0 : 16k 104M
25-12 06:54:23| 0 60B| 5 13 71 12 0| 0 0 : 0 0 : 0 108M
25-12 06:54:24| 0 60B| 5 7 71 16 0| 0 0 : 352k 0 : 0 109M
25-12 06:54:25| 0 60B| 5 14 67 14 0| 24k 3400k: 768k 3400k: 32k 93M
25-12 06:54:26| 0 60B| 6 12 72 11 0| 0 0 : 32k 0 : 0 105M
25-12 06:54:27| 0 60B| 5 14 69 11 0| 0 0 : 28k 0 :8192B 99M
25-12 06:54:28| 0 60B| 5 19 67 9 0| 0 0 : 0 0 : 0 101M
25-12 06:54:29| 0 60B| 5 2 73 21 0| 0 0 : 0 0 : 0 102M
25-12 06:54:30| 0 60B| 3 1 73 23 0|2720k 0 : 0 0 : 0 108M
25-12 06:54:31| 0 60B| 3 2 73 22 0| 0 0 : 0 0 : 0 104M
25-12 06:54:32| 60B 60B| 5 13 79 4 0| 0 0 :3036k 0 : 0 39M
25-12 06:54:33| 0 60B| 5 17 76 2 0| 0 0 : 0 0 : 0 27M
----system---- -net/total- --total-cpu-usage-- --dsk/sda-- --dsk/sdb-- --dsk/sdd--
time | recv send|usr sys idl wai stl| read writ: read writ: read writ
25-12 06:54:34| 0 60B| 5 8 65 22 0| 0 0 : 0 0 : 152k 49M
25-12 06:54:35| 0 60B| 5 15 60 19 0| 16k 0 : 0 0 : 0 51M
25-12 06:54:36| 0 60B| 5 1 73 22 0| 0 0 : 0 0 : 0 110M
25-12 06:54:37| 0 60B| 3 3 72 23 0| 0 0 :4516k 0 : 0 103M
25-12 06:54:38| 0 60B| 5 13 76 6 0|4324k 0 :2276k 0 :4096B 68M
25-12 06:54:39| 0 60B| 5 13 71 10 0| 0 0 :1564k 0 : 32k 96M
25-12 06:54:40| 0 60B| 5 14 71 10 0| 108k 0 : 316k 0 : 0 97M
25-12 06:54:41| 0 60B| 6 15 68 11 0| 72k 0 : 0 0 : 32k 99M
25-12 06:54:42| 0 60B| 7 11 71 11 0| 0 0 : 44k 0 : 0 114MOffline