Arch Linux

weirddan455

Member

Registered: 2012-04-15

Posts: 209

TCP Out of Memory

So my network just broke.  This is the first time I've run into a bug in the actual kernel.  I'm actually not even sure where to report kernel bugs.  I guess I'll Google that after this post.  I'm posting here in case it's something Arch specifc.  When it happened, my system had been up maybe 3 or 4 days.  Usually I'll reboot every day or two so that may be why I've only run into this just now.  It broke while I was in the middle of downloading updates through pacman.  After it broke, I could still ping just fine but anything else was basically a no-go.  I loaded up Wireshark while trying to load a webpage through Firefox and I could see the SYN, SYN-ACK, ACK handshake and I also saw a TLS/SSL handshake but the webpage never loaded.  Also pacman could no longer connect to my mirror... it just timed out.  After a restart all was fine again.

Also the "out of memory" error is confusing because at the time I was using only ~1GB RAM out of 16GB available so the system was nowhere near out of memory.

My network card:

$ lspci | grep Ethernet
07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)

It's using the r8169 kernel module as the driver and I'm running kernel 3.17.2 (the "linux" package from the [core] repo.)

And here's the error from dmesg:

[322438.563394] TCP: out of memory -- consider tuning tcp_mem
[322475.994051] TCP: out of memory -- consider tuning tcp_mem
[322479.087904] TCP: out of memory -- consider tuning tcp_mem
[322486.753438] TCP: out of memory -- consider tuning tcp_mem
[322618.973412] ------------[ cut here ]------------
[322618.973426] WARNING: CPU: 0 PID: 0 at net/core/stream.c:201 sk_stream_kill_queues+0x151/0x160()
[322618.973427] Modules linked in: fuse bridge stp llc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec iTCO_wdt iTCO_vendor_support nvidia(PO) snd_hwdep snd_pcm mxm_wmi evdev drm ecb serio_raw snd_timer btusb coretemp hwmon intel_rapl x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm r8169 pcspkr mousedev joydev bluetooth snd mac_hid mii rfkill soundcore thermal shpchp fan mei_me mei i2c_i801 button i2c_core lpc_ich battery wmi tpm_infineon video tpm_tis tpm processor ext4 crc16 mbcache jbd2 algif_skcipher af_alg dm_crypt dm_mod sr_mod cdrom sd_mod crc_t10dif ata_generic pata_acpi hid_generic usbhid hid atkbd libps2 crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper
[322618.973464]  ablk_helper cryptd ahci ata_piix libahci libata scsi_mod ehci_pci xhci_hcd ehci_hcd usbcore usb_common i8042 serio
[322618.973471] CPU: 0 PID: 0 Comm: swapper/0 Tainted: P           O   3.17.2-1-ARCH #1
[322618.973473] Hardware name: MSI MS-7681/Z68A-GD65 (G3) (MS-7681), BIOS V25.8 01/11/2013
[322618.973474]  0000000000000000 67bcc9c1036c0b93 ffff88041ec03d40 ffffffff815367d0
[322618.973476]  0000000000000000 ffff88041ec03d78 ffffffff8107054d ffff88030f0dc600
[322618.973478]  ffff88030f0dc780 ffff88030f0dc728 0000000000000000 ffff88030f0dc600
[322618.973480] Call Trace:
[322618.973482]  <IRQ>  [<ffffffff815367d0>] dump_stack+0x4d/0x6f
[322618.973488]  [<ffffffff8107054d>] warn_slowpath_common+0x7d/0xa0
[322618.973490]  [<ffffffff8107067a>] warn_slowpath_null+0x1a/0x20
[322618.973493]  [<ffffffff81438a31>] sk_stream_kill_queues+0x151/0x160
[322618.973496]  [<ffffffff814890a6>] inet_csk_destroy_sock+0x56/0x170
[322618.973498]  [<ffffffff8148a9a7>] tcp_done+0x67/0xa0
[322618.973500]  [<ffffffff8149cd61>] tcp_write_err+0x31/0x50
[322618.973502]  [<ffffffff8149d628>] tcp_retransmit_timer+0x208/0x770
[322618.973504]  [<ffffffff8149dd40>] ? tcp_write_timer_handler+0x1b0/0x1b0
[322618.973508]  [<ffffffff8149dc20>] tcp_write_timer_handler+0x90/0x1b0
[322618.973510]  [<ffffffff8149ddbc>] tcp_write_timer+0x7c/0x80
[322618.973512]  [<ffffffff810d5aa6>] call_timer_fn+0x36/0x160
[322618.973514]  [<ffffffff8149dd40>] ? tcp_write_timer_handler+0x1b0/0x1b0
[322618.973516]  [<ffffffff810d64d4>] run_timer_softirq+0x274/0x320
[322618.973519]  [<ffffffff81074402>] __do_softirq+0xf2/0x2e0
[322618.973521]  [<ffffffff81074746>] irq_exit+0x86/0xb0
[322618.973524]  [<ffffffff8153f334>] smp_apic_timer_interrupt+0x44/0x50
[322618.973526]  [<ffffffff8153d5cd>] apic_timer_interrupt+0x6d/0x80
[322618.973527]  <EOI>  [<ffffffff813f27f2>] ? cpuidle_enter_state+0x62/0x1a0
[322618.973532]  [<ffffffff813f2a17>] cpuidle_enter+0x17/0x20
[322618.973535]  [<ffffffff810afd1c>] cpu_startup_entry+0x3dc/0x430
[322618.973538]  [<ffffffff8152cab4>] rest_init+0x84/0x90
[322618.973540]  [<ffffffff818fbff1>] start_kernel+0x471/0x492
[322618.973542]  [<ffffffff818fb120>] ? early_idt_handlers+0x120/0x120
[322618.973544]  [<ffffffff818fb4d7>] x86_64_start_reservations+0x2a/0x2c
[322618.973546]  [<ffffffff818fb62b>] x86_64_start_kernel+0x152/0x175
[322618.973547] ---[ end trace 3468a80001ace10a ]---
[322618.973550] ------------[ cut here ]------------
[322618.973553] WARNING: CPU: 0 PID: 0 at net/ipv4/af_inet.c:153 inet_sock_destruct+0x191/0x200()
[322618.973554] Modules linked in: fuse bridge stp llc snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec iTCO_wdt iTCO_vendor_support nvidia(PO) snd_hwdep snd_pcm mxm_wmi evdev drm ecb serio_raw snd_timer btusb coretemp hwmon intel_rapl x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm r8169 pcspkr mousedev joydev bluetooth snd mac_hid mii rfkill soundcore thermal shpchp fan mei_me mei i2c_i801 button i2c_core lpc_ich battery wmi tpm_infineon video tpm_tis tpm processor ext4 crc16 mbcache jbd2 algif_skcipher af_alg dm_crypt dm_mod sr_mod cdrom sd_mod crc_t10dif ata_generic pata_acpi hid_generic usbhid hid atkbd libps2 crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper
[322618.973577]  ablk_helper cryptd ahci ata_piix libahci libata scsi_mod ehci_pci xhci_hcd ehci_hcd usbcore usb_common i8042 serio
[322618.973582] CPU: 0 PID: 0 Comm: swapper/0 Tainted: P        W  O   3.17.2-1-ARCH #1
[322618.973583] Hardware name: MSI MS-7681/Z68A-GD65 (G3) (MS-7681), BIOS V25.8 01/11/2013
[322618.973584]  0000000000000000 67bcc9c1036c0b93 ffff88041ec03db0 ffffffff815367d0
[322618.973586]  0000000000000000 ffff88041ec03de8 ffffffff8107054d ffff88030f0dc600
[322618.973588]  ffff88030f0dc780 ffff88030f0dc66c ffffffff8149dd40 ffff88030f0dc600
[322618.973589] Call Trace:
[322618.973590]  <IRQ>  [<ffffffff815367d0>] dump_stack+0x4d/0x6f
[322618.973593]  [<ffffffff8107054d>] warn_slowpath_common+0x7d/0xa0
[322618.973595]  [<ffffffff8149dd40>] ? tcp_write_timer_handler+0x1b0/0x1b0
[322618.973597]  [<ffffffff8107067a>] warn_slowpath_null+0x1a/0x20
[322618.973599]  [<ffffffff814b5281>] inet_sock_destruct+0x191/0x200
[322618.973601]  [<ffffffff8142d70d>] __sk_free+0x1d/0x130
[322618.973603]  [<ffffffff8142d839>] sk_free+0x19/0x20
[322618.973605]  [<ffffffff8149dda8>] tcp_write_timer+0x68/0x80
[322618.973607]  [<ffffffff810d5aa6>] call_timer_fn+0x36/0x160
[322618.973609]  [<ffffffff8149dd40>] ? tcp_write_timer_handler+0x1b0/0x1b0
[322618.973611]  [<ffffffff810d64d4>] run_timer_softirq+0x274/0x320
[322618.973613]  [<ffffffff81074402>] __do_softirq+0xf2/0x2e0
[322618.973615]  [<ffffffff81074746>] irq_exit+0x86/0xb0
[322618.973617]  [<ffffffff8153f334>] smp_apic_timer_interrupt+0x44/0x50
[322618.973619]  [<ffffffff8153d5cd>] apic_timer_interrupt+0x6d/0x80
[322618.973620]  <EOI>  [<ffffffff813f27f2>] ? cpuidle_enter_state+0x62/0x1a0
[322618.973623]  [<ffffffff813f2a17>] cpuidle_enter+0x17/0x20
[322618.973625]  [<ffffffff810afd1c>] cpu_startup_entry+0x3dc/0x430
[322618.973627]  [<ffffffff8152cab4>] rest_init+0x84/0x90
[322618.973629]  [<ffffffff818fbff1>] start_kernel+0x471/0x492
[322618.973632]  [<ffffffff818fb120>] ? early_idt_handlers+0x120/0x120
[322618.973634]  [<ffffffff818fb4d7>] x86_64_start_reservations+0x2a/0x2c
[322618.973635]  [<ffffffff818fb62b>] x86_64_start_kernel+0x152/0x175
[322618.973637] ---[ end trace 3468a80001ace10b ]---
[322654.327487] Netfilter messages via NETLINK v0.30.
[322655.186174] cfg80211: Calling CRDA to update world regulatory domain

EDIT:  I found a bug reported on this several months back with no solution.  Seems it might be related to birdge networking.  I added my log to the report.

https://bugzilla.kernel.org/show_bug.cgi?id=77481

Last edited by weirddan455 (2014-11-10 22:21:18)