Arch Linux

mkkot

Member

From: Poland

Registered: 2009-12-20

Posts: 287

jbd2 and kworker constant disc activity

Hello,

I'm wondering why my HDD led blinks every 2-3 seconds or even more often. iotop -Pao shows only journaling and kworker. What could be wrong? Ext's4 minimal commit interval should write every 5 seconds but AFAIK only when there is something to write.

Total DISK READ :       0.00 B/s | Total DISK WRITE :       3.79 K/s
Actual DISK READ:       0.00 B/s | Actual DISK WRITE:       0.00 B/s
  PID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND          
  233 be/3 root          0.00 B     24.00 K  0.00 %  0.98 % [jbd2/sda10-8]
   71 be/4 root          0.00 B      0.00 B  0.00 %  0.02 % [kworker/u8:4]
11791 be/4 root          0.00 B      0.00 B  0.00 %  0.02 % [kworker/0:2]
11327 be/4 root          0.00 B      0.00 B  0.00 %  0.01 % [kworker/1:0]

aurabindo

Member

From: Toronto

Registered: 2012-05-26

Posts: 41

Website

Re: jbd2 and kworker constant disc activity

bump!, did you figure out what went wrong? I am experiencing the same issue on both arch (3.14), and ubuntu (3.13)

mkkot

Member

From: Poland

Registered: 2009-12-20

Posts: 287

Re: jbd2 and kworker constant disc activity

Not really. But you can see that kworker doesn't actually cause physical writtings to disc even though it does some I/O operations. Please detach any sata optical drives if you have because kernel driver can poll them which causes hdd blinking. I don't know how to disable this behaviour yet.

MDykstra

Member

Registered: 2014-03-12

Posts: 11

Re: jbd2 and kworker constant disc activity

I do get disk activity from this. It's mostly jdb2 which does the writing, probably in response to kworker making small writes constantly. Unfortunately my harddisk is making soft clicking noises because of it. It happens also in emergency runlevel with the system's journal logged in volatile memory. Apparently it's both for ext3 and ext4 partitions. So the same problem got introduced in both filesystem implementations or it happens due to something elsewhere in the kernel.

Also check: kworker, jdb2 constantly writing to disk

Last edited by MDykstra (2014-05-02 09:05:26)

MDykstra

Member

Registered: 2014-03-12

Posts: 11

Re: jbd2 and kworker constant disc activity

I used perf to look at this in emergency mode (see below), with journal logged in volatile memory.
The function ret_from_fork seems to be at the beginning of each kernel trace.
I also don't know why I have swapper process writing to disk; I don't have any swap mounted.
I also don't know if I should try something with hdparm, but I have the feeling the kernel
is doing something strange. After a few months it's getting on my nerves ...

-  87.79%    kworker/0:1  [kernel.kallsyms]  [k] blk_peek_request
     blk_peek_request
     0xffffffffa020eb4b
     __blk_run_queue
     blk_execute_rq_nowait
     blk_execute_rq
     0xffffffffa020cf3b
     0xffffffffa020d06c
     0xffffffffa03fb7eb
     0xffffffffa03e9058
     0xffffffffa03fbc7d
     disk_check_events
     disk_events_workfn
     process_one_work
     worker_thread
     kthread
     ret_from_fork
-   7.26%       kthreadd  [kernel.kallsyms]  [k] blk_peek_request
     blk_peek_request
     0xffffffffa020eb4b
     __blk_run_queue
     blk_execute_rq_nowait
     blk_execute_rq
     0xffffffffa020cf3b
     0xffffffffa020d06c
     0xffffffffa03fb7eb
     0xffffffffa03e9058
     0xffffffffa03fbc7d
     disk_check_events
     disk_events_workfn
     process_one_work
     worker_thread
     kthread
     ret_from_fork
-   3.89%  kworker/u16:2  [kernel.kallsyms]  [k] blk_peek_request
     blk_peek_request
     0xffffffffa020eb4b
     __blk_run_queue
     queue_unplugged
     blk_flush_plug_list
        72.73% blk_queue_bio
           generic_make_request
           submit_bio
           _submit_bh
           __block_write_full_page
           block_write_full_page_endio
           block_write_full_page
           blkdev_writepage
           __writepage
           write_cache_pages
           generic_writepages
           do_writepages
           __writeback_single_inode
           writeback_sb_inodes
           __writeback_inodes_wb
           wb_writeback
           bdi_writeback_workfn
           process_one_work
           worker_thread
           kthread
           ret_from_fork 
        27.27% blk_finish_plug
           generic_writepages
           do_writepages
           __writeback_single_inode
           writeback_sb_inodes
           __writeback_inodes_wb
           wb_writeback
           bdi_writeback_workfn
           process_one_work
           worker_thread
           kthread
           ret_from_fork
-   0.71%    jbd2/sda3-8  [kernel.kallsyms]  [k] blk_peek_request
     blk_peek_request
     0xffffffffa020eb4b
     __blk_run_queue
        50.00% queue_unplugged
           blk_flush_plug_list
              50.00% io_schedule
                 sleep_on_page
                 __wait_on_bit
                 wait_on_page_bit
                 filemap_fdatawait_range
                 filemap_fdatawait
                 0xffffffffa041644f
                 0xffffffffa041cc3c
                 kthread
                 ret_from_fork
              50.00% blk_finish_plug
                 0xffffffffa0415af6
                 0xffffffffa041cc3c
                 kthread
                 ret_from_fork
        25.00% cfq_insert_request
           __elv_add_request
           blk_flush_plug_list
           blk_finish_plug
           generic_writepages
           0xffffffffa0415a43
           0xffffffffa041cc3c
           kthread
           ret_from_fork
        25.00% blk_queue_bio
           generic_make_request
           submit_bio
           _submit_bh
           submit_bh
           0xffffffffa0415583
           0xffffffffa0416887
           0xffffffffa041cc3c
           kthread
           ret_from_fork 
-   0.35%        swapper  [kernel.kallsyms]  [k] blk_peek_request
     blk_peek_request
     0xffffffffa020eb4b
     __blk_run_queue
     blk_run_queue
     0xffffffffa020d98d
     0xffffffffa020f407
     0xffffffffa020f692
     0xffffffffa02065f2
     0xffffffffa020f38e
     blk_done_softirq
     __do_softirq
     irq_exit
     do_IRQ
     ret_from_intr
     cpuidle_idle_call
     arch_cpu_idle
     cpu_startup_entry
     rest_init
     start_kernel
     x86_64_start_reservations
     x86_64_start_kernel