You are not logged in.

#1 2018-11-10 10:12:26

Papi
Member
Registered: 2018-11-10
Posts: 5

[SOLVED] Random I/O freezes

Hi,
I am using ArchLinux since a few years now. I have problem with random I/O freezes since end of the September. I've tried to downgrade kernel version, but it didn't help.

Error description
* system normally works, but I can't load page in browser, open new terminal window and some apps become transparent
* display works smoothly and responsively
* music and videos normally plays when it's cached in RAM, but I can't load next song or watch next part of video when it needs to be downloaded from internet (Spotify, YouTube)
* dmesg and journalctl is empty, no errors
* ioping shows normal values when I/O is freezed (about 200μs (best) - 2ms (worst))
* dstat shows missed x ticks very often
* dstat, iostat and iotop shows very high peak of I/O usage when it stops to be freezed

Possible causes
It happens very randomly. I/O freezes every 2 minutes or 2 hours. One freeze took usually from few seconds to half a minute.
Usually it happens immediately after boot when I start a lot of apps at once (Chrome, Telegram, Slack, Caprine, Spotify, Thunderbird)

Laptop information
Model: Dell Precision 3530
CPU: Intel Core Processor i5-8400H (4 Core, 8MB Cache, 2.50GHz, 4.2GHz Turbo, 35W), vPro
GPU: Nvidia Quadro P600 w/ 4GB GDDR5 (disabled), Intel® UHD Graphics 630
RAM: 16GB, DDR4-2666MHz SDRAM, 2 DIMMS, Non-ECC
SSD: Kingston HyperX Savage 480GB, 2.5", SHSS37A/480G

System information
* Kernel: 4.18.16-arch1-1-ARCH
* Filesystem: btrfs

I've also tried to benchmark my disk using dd if=/dev/zero of=file bs=4M count=10000 status=progress
* Blue - normal system
* Orange - Arch Linux from USB with mounted my btrfs filesystem

https://i.imgur.com/2YiVPnw.png
(X-axis - record from dstat, Y-axis - MB/s)

Peaks in a blue line happens after freeze.

Mod note: Please don't embed large images in your posts, use URLs or linked thumbnails -- https://wiki.archlinux.org/index.php/Co … s_and_code -- WorMzy

Last edited by Papi (2018-11-15 07:59:56)

Offline

#2 2018-11-10 10:47:36

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 8,345
Website

Re: [SOLVED] Random I/O freezes

Hi Papi, welcome to the Arch Linux forums.

Please provide a full journal from an affected session (e.g. use 'journalctl -b > journal', and then a pastebin client to upload it). The output from

smartctl -a /dev/sda

(assuming your SSD is enumerated as sda)

and

btrfs filesystem show
btrfs device stats /

Sakura:-
Mobo: MSI X299 TOMAHAWK ARCTIC // Processor: Intel Core i7-7820X 3.6GHz // GFX: nVidia GeForce GTX 970 // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 5x 1TB HDD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Offline

#3 2018-11-10 11:57:57

Papi
Member
Registered: 2018-11-10
Posts: 5

Re: [SOLVED] Random I/O freezes

journalctl -b: https://ptpb.pw/5fbe
smartctl -a /dev/sda: https://ptpb.pw/E1ce

btrfs filesystem show

Label: none  uuid: 60ed664d-a9be-4576-a56f-6072defcc202
	Total devices 1 FS bytes used 69.87GiB
	devid    1 size 186.26GiB used 72.02GiB path /dev/sda4

btrfs device stats /

[/dev/sda4].write_io_errs    0
[/dev/sda4].read_io_errs     0
[/dev/sda4].flush_io_errs    0
[/dev/sda4].corruption_errs  0
[/dev/sda4].generation_errs  0

Offline

#4 2018-11-10 15:13:56

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 8,345
Website

Re: [SOLVED] Random I/O freezes

Okay, disk and filesystem look fine, so let's rule those out for now.

Some notes about your journal:

  • You haven't setup early microcode loading. This is highly recommended, although I can't say whether it will actually help in this case. See this.

  • Your default IO scheduler is cfq, but you have an SSD. Again this isn't necessarily a problem, but I would recommend using noop for the SSD. See this.

  • I see a lot of CPU overheating events, some lasting a few seconds, one lasting exactly a minute, do these times correspond at all to the behaviour you are seeing?

    lis 09 00:06:25 archlinux kernel: CPU0: Core temperature above threshold, cpu clock throttled (total events = 1)
    lis 09 00:06:36 archlinux kernel: CPU0: Core temperature/speed normal
    lis 09 10:20:35 archlinux kernel: CPU0: Core temperature above threshold, cpu clock throttled (total events = 156)
    lis 09 10:20:44 archlinux kernel: CPU0: Core temperature/speed normal
    lis 09 12:09:44 archlinux kernel: CPU0: Core temperature above threshold, cpu clock throttled (total events = 646)
    lis 09 12:09:47 archlinux kernel: CPU0: Core temperature/speed normal
    lis 09 13:16:50 archlinux kernel: CPU0: Core temperature above threshold, cpu clock throttled (total events = 821)
    lis 09 13:17:09 archlinux kernel: CPU0: Core temperature/speed normal
    lis 10 10:12:29 archlinux kernel: CPU0: Core temperature above threshold, cpu clock throttled (total events = 861)
    lis 10 10:13:29 archlinux kernel: CPU0: Core temperature/speed normal

Sakura:-
Mobo: MSI X299 TOMAHAWK ARCTIC // Processor: Intel Core i7-7820X 3.6GHz // GFX: nVidia GeForce GTX 970 // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 5x 1TB HDD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Offline

#5 2018-11-10 17:00:37

Papi
Member
Registered: 2018-11-10
Posts: 5

Re: [SOLVED] Random I/O freezes

* I have early microcode loading.
* I've changed I/O scheduler to noop. Thanks for the tip.
* Times of CPU overheating events doesn't correspond to my issue. I don't know why they happens, because my CPU temperature is between 40 and 70 °C

Offline

#6 2018-11-13 22:19:32

Papi
Member
Registered: 2018-11-10
Posts: 5

Re: [SOLVED] Random I/O freezes

I did next experiments and here is dmesg output: https://ptpb.pw/fSLE.
There are a lot of messages like `INFO: task btrfs-transacti:247 blocked for more than 120 seconds.`

Offline

#7 2018-11-15 07:59:43

Papi
Member
Registered: 2018-11-10
Posts: 5

Re: [SOLVED] Random I/O freezes

Solved...
I think it was issue in my btrfs filesystem. I've switched from btrfs to ext4 and now it's ok.

Possible fix for others: Create new btrfs filesystem and migrate your data with rsync or something like that.

Offline

Board footer

Powered by FluxBB