You are not logged in.

#1 2019-06-01 15:17:05

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Is there a memory regression in kernel 5.1.5

Hi,

I experienced a dramatic regression with regard to memory usage starting with my latest kernel upgrade (now on 5.1.5-arch1).

I'm running arch linux on a Zenbook UX21A with only 4 GB of RAM, but it was never a problem.

Now, if I start firefox or chromium and open more than 5 tabs, system goes up to 3GB memory usage or more and stalls almost immediately.

Anyone experienced this recently?

BTW: is there a mechanism in linux that would simply automatically kill the offending application instead of letting it impact the entire system?

Thanks!

Offline

#2 2019-06-02 09:19:40

frostschutz
Member
Registered: 2013-11-15
Posts: 1,409

Re: Is there a memory regression in kernel 5.1.5

I have no such issue on two machines, one 16GB RAM the other 3GB.

It's possible to get hangs or crashes if you are using too much/too large `tmpfs`. Make sure the Size of all your tmpfs instances added together does not exceed 50% of RAM. By default each instance gets 50% RAM ... if you actually fill up two of them, then your system is 100% dead.

BTW: is there a mechanism in linux that would simply automatically kill the offending application instead of letting it impact the entire system?

that's what the OOM killer is supposed to do but it can't free up tmpfs...

Last edited by frostschutz (2019-06-02 09:20:59)

Online

#3 2019-06-02 10:03:45

sinx
Member
Registered: 2019-06-02
Posts: 17

Re: Is there a memory regression in kernel 5.1.5

Hello,
I do not see any increased memory usage on my system.
Are you sure your system is crashing when you use too much ram, or just become insanely slower ? If you have a swap partition it starts to be used when you fill up your real ram and so some operations slow down from your ram speed to your hard disk speed (it's really really slower...).
In all cases i think you can save your computer from a crash or an extreme slow down by killing a task using too much ram when your ram is nearly full.
The command :

 top -b -o %MEM -n 1 

prints the top ram-using process and the free ram you have. It can be used to kill a process which uses a lot of ram only when there is no more ram left (in fact just before).
I have just made a bash script which seems working to do this (I tested 2-3 times). Call it once and it kills the process using the more ram, only if needed. You can call it periodically in a daemon or whatever you want :

#!/bin/bash

LIMIT=200 

if [[ `top -b -o %MEM -n 1 | sed -n 4p | awk '{print $6;}'` -lt $LIMIT ]]; then
  kill `top -b -o %MEM -n 1 | sed -n 8p | awk '{print $1;}'`
fi

You can change the value of LIMIT which is the minimum free ram you want to keep (in MB).
Note that this script is not safe at all and can kill any process even if it is an important one, if it uses too much ram.

Offline

#4 2019-06-02 10:09:34

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Is there a memory regression in kernel 5.1.5

Another is this little script:

#!/bin/bash
# Display the top applications of memory usage
# http://www.cyberciti.biz/faq/linux-check-memory-usage/#comment-51021
 
while read command percent rss; do 
  if [[ "${command}" != "COMMAND" ]]; then 
    rss="$(bc <<< "scale=2;${rss}/1024")" 
  fi
  printf " %-26s%-8s%s\n" "${command}" "${percent}" "${rss} MB" \
  | sed 's/COMMAND/PROGRAM/' | sed 's/RSS MB/#MEM/'
done < <(ps -A --sort -rss -o comm,pmem,rss | head -n 20)

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Online

#5 2019-06-02 12:53:40

latalante1
Member
Registered: 2018-08-30
Posts: 110

Re: Is there a memory regression in kernel 5.1.5

You checked exactly what changes the creators of webapp, websites that you visit make. What has changed in their code in between? Do you know?
The number of open tabs is no good measure. For example, these two will take up much more memory than 50 forum pages like this.
https://bellard.org/jslinux/vm.html?cpu … g&mem=1024
https://chromium.googlesource.com/chrom … er&n=10000

The kernel developers would be the last to be targeted.

Get interested in something like Zram. It has been in the kernel for years.
https://kernelnewbies.org/Linux_2_6_33# … d_swapping

You should have done this for a long time, if you do not intend to buy additional memory.

Offline

#6 2019-06-02 19:44:15

tydynrain
Member
From: Lower Puna, Big Island Hawai'i
Registered: 2017-10-26
Posts: 115
Website

Re: Is there a memory regression in kernel 5.1.5

Get interested in something like Zram. It has been in the kernel for years.
https://kernelnewbies.org/Linux_2_6_33# … d_swapping

I've been using Zram on my systems very soon after I began using Arch, and I'd say that it definitely improves the situation.


Registered Linux User: #623501 | Arch Linux Principles: Simplicity - Modernity - Pragmatism - User Centrality - Versatility => KISS
Arch Linux, the most exciting thing since Linus created Linux and married it with GNU/GPL.
Arch Linux for Life, Arch Linux Forever!

Offline

#7 2019-06-02 21:04:04

latalante1
Member
Registered: 2018-08-30
Posts: 110

Re: Is there a memory regression in kernel 5.1.5

Once for testing purposes it can be one command. Permanently, not much more work.

modprobe zram && echo 2G > /sys/block/zram0/disksize && mkswap --label zram0 /dev/zram0 && swapon --priority 100 /dev/zram0 && echo 2G > /sys/block/zram1/disksize && mkswap --label zram1 /dev/zram1 && swapon --priority 100 /dev/zram1

If you have four cores, you can create 4 swaps by 1G (there can be even more, maybe less). Compression ratio is at a minimum 1:3 (lzo, lz4).
If you have a fast ssd (persistent) disk or nvme you can use it Zram writeback feature.
You need to test and test in search of the optimal configuration.

Offline

#8 2019-06-03 20:33:27

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: Is there a memory regression in kernel 5.1.5

frostschutz wrote:

I have no such issue on two machines, one 16GB RAM the other 3GB.

It's possible to get hangs or crashes if you are using too much/too large `tmpfs`. Make sure the Size of all your tmpfs instances added together does not exceed 50% of RAM. By default each instance gets 50% RAM ... if you actually fill up two of them, then your system is 100% dead.

Thanks frostschutz, here's how mine look:

df -h| grep tmpfs
tmpfs             1.9G   68M  1.9G   4% /dev/shm
tmpfs             1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs             1.9G   94M  1.8G   5% /tmp
tmpfs             384M   16K  383M   1% /run/user/1000

I didn't set up any of these manually, and I'm not sure what they do (except /tmp). Anything suspicious here?

Offline

#9 2019-06-03 20:37:18

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: Is there a memory regression in kernel 5.1.5

sinx wrote:

Hello,
I do not see any increased memory usage on my system.
Are you sure your system is crashing when you use too much ram, or just become insanely slower ? If you have a swap partition it starts to be used when you fill up your real ram and so some operations slow down from your ram speed to your hard disk speed (it's really really slower...).

Thanks, sinx: actually I don't have a swap partition, and indeed: it is probably not crashing, but becoming "just" absolutely unresponsive, — amounting to the same thing though, as the only exit is a hard reset then.

Offline

#10 2019-06-03 20:39:42

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: Is there a memory regression in kernel 5.1.5

graysky wrote:

Another is this little script:

#!/bin/bash
# Display the top applications of memory usage
# http://www.cyberciti.biz/faq/linux-check-memory-usage/#comment-51021
 
while read command percent rss; do 
  if [[ "${command}" != "COMMAND" ]]; then 
    rss="$(bc <<< "scale=2;${rss}/1024")" 
  fi
  printf " %-26s%-8s%s\n" "${command}" "${percent}" "${rss} MB" \
  | sed 's/COMMAND/PROGRAM/' | sed 's/RSS MB/#MEM/'
done < <(ps -A --sort -rss -o comm,pmem,rss | head -n 20)

Thanks I'll try this immediately!! that's the thing I was looking for.

Offline

#11 2019-06-03 20:50:45

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: Is there a memory regression in kernel 5.1.5

latalante1 wrote:

You checked exactly what changes the creators of webapp, websites that you visit make. What has changed in their code in between? Do you know?
The number of open tabs is no good measure. For example, these two will take up much more memory than 50 forum pages like this.
https://bellard.org/jslinux/vm.html?cpu … g&mem=1024
https://chromium.googlesource.com/chrom … er&n=10000

No, I can't say at all:  but I didn't change my surfing habits and the memory situation happens independent of website; there has to be something local to my machine that behaves differently, otherwise the occurrence of several webites all going rogue at the same time would be to improbable.

Get interested in something like Zram. It has been in the kernel for years.
https://kernelnewbies.org/Linux_2_6_33# … d_swapping

You should have done this for a long time, if you do not intend to buy additional memory.

Yes, that sounds good, I'give it a try! Unforunately, these Zenbooks do not allow a RAM upgrade, I would have done it long before! Thanks!

Offline

Board footer

Powered by FluxBB