high ram usage after a latest update?

seth · 2021-11-19 15:59:15

so what is GTT ? should I know what it is as a user?

See the wikipedia link. It's what I think you think it is and users usually don't have to be aware of it, no.

but because dota had a update before

Sounds more like related to the file cache, which is unrelated to VRAM & GTT (and likely not an issue either)

Again: the interesting question here is whether if you've "lost" RAM to GTT and you allocate a specific amount of RAM that will require some GTT to be returned but NOT so much that you run OOM no matter what (why FF etc. are a bad test because you cannot really control their memhoggery), does that release some GTT back to the available RAM (after the allocating process has died and freed it)
iow: the head/tail allocation test.

rezad · 2021-11-19 17:02:09

seth wrote:

so what is GTT ? should I know what it is as a user?
See the wikipedia link. It's what I think you think it is and users usually don't have to be aware of it, no.
but because dota had a update before
Sounds more like related to the file cache, which is unrelated to VRAM & GTT (and likely not an issue either)
Again: the interesting question here is whether if you've "lost" RAM to GTT and you allocate a specific amount of RAM that will require some GTT to be returned but NOT so much that you run OOM no matter what (why FF etc. are a bad test because you cannot really control their memhoggery), does that release some GTT back to the available RAM (after the allocating process has died and freed it)
iow: the head/tail allocation test.

I said dota had update but that was not important. what was important is that after running dota once and playing I restarted dota and it seems to not use the used ram from last time so it may accumulate the ram usage which is bad. but with benchmark I am not sure if that happens and I will test that too.

so you want me to do this:
assign 2GB in bios to vram
right the benchamrk that seems to have the most effect (1GB before 6.5GB after)
then use that command to test
?

also would you mind telling me the exact command to use with parameters ?
this one ?
cat <( </dev/zero head -c 500m) <(sleep SECONDS) | tail

I am not a pro user even though "I user arch btw"® ™
and what I am supposed to be doing when you say stress?
fill ram fast? or go near the edge and the use ,say 1gb , and see if system releases ram from the graphic usage wastage?
I think my firefox test showed that it doesnt release back but I will test what you will put in the next post.

I think it will hang again but do I need to check anything else?
maybe have dmesg open and watch for any error before the hang?

Last edited by rezad (2021-11-19 17:10:39)

rezad · 2021-11-19 17:49:59

ok I tested but something happened which I think you thought it would.

I ran the benchmark and it was not accumulative. so running it twice would only use the 5GB (ram usage goes from 2.5 to 7.5GB)
but then I was testing with dd command with "dd if=/dev/zero bs=1G of=/dev/null" and I saw that using then in many konsole tab would not increase the ram usage that much.
so I was opening more tab and increase the 1GB parameteres and it said an error for more than 13GB (maybe dd checks total ram size) so I used 13GB but still ram usage would be 9.5GB. I got tired of that and closed all the dd commands.
now the weird thing happened.
the ram was released back! so the usage became just 2.5GB.
so there is hope that system does release it back.
but before that happened system hangs and I almost restarted but a OOM (i think that is called) happened and brave browser addon and tab crashed and dmesg said this [ +0.113095] systemd-journald[319]: /dev/kmsg buffer overrun, some messages lost.
and

 oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service/app.slice/app-org.kde.konsole-34168698ff7649b88919b5408453a84c.scope,task=tail,pid=52891,uid=1000
[  +0.000031] Out of memory: Killed process 52891 (tail) total-vm:10299312kB, anon-rss:10293896kB, file-rss:4kB, shmem-rss:0kB, UID:1000 pgtables:20196kB oom_score_adj:0"

also a simpler one is this one after using the gpu benchmark

</dev/zero head -c 15000m | tail
zsh: broken pipe head -c 15000m < /dev/zero |
zsh: killed tail

this caused the RAM to go full and then it crashed and the ram usage that was 7.5 after gpu benchmark went down to 2.5GB

so maybe there is hope.
but how do I make this work with firefox.
with firefox running out of memory after the gpu benchmark (or game) system just hanged.
should I have waited more for that and maybe it would recover?

this I think is still an issue when ram usage is shown as 7.5GB if running another app (tail) and crashing it would release it back. or maybe that 7.5GB is deceptive.

my main point I dont see where that RAM is and what is using it. which is weird for me on linux.
if this is cache as you say, shouldnt closing the app that uses it also remove that cache? and this happens with gpu things not filesystem things so I dont think it is directory cache. though I am a noob with these issues and words.

Last edited by rezad (2021-11-19 18:01:26)

seth · 2021-11-20 16:37:18

You can still run perfectly OOM w/ FF (or other clients) despite the GTT being returned to the system - or maybe (FF uses GL acceleration?!) in combination.

should I have waited more for that and maybe it would recover?

Could imply thrashing, (check "sudo iotop") ie. the kernel juggles files in and out of the RAM cache (because it can't eg. drop anon pages lacking the swap space to do so) - still OOM, though.
Feel encouraged to add a swap file to increase your available memory and see whether that helps you w/ FF.

In very doubt, you can always abuse head to explicitly reclaim your RAM from the GTT…

the ram usage that was 7.5 after gpu benchmark went down to 2.5GB

The aggregated values are not all that helpful (the fancy RAM meters for desktop widgets are that: "fancy")
The GTT should™ only take ~3GB, so 2.5+3=5.5, that means 2GB would be taken by sth. different
=> /proc/meminfo before and after the memory release for more insight on that.

rezad · 2021-11-21 00:06:01

seth wrote:

You can still run perfectly OOM w/ FF (or other clients) despite the GTT being returned to the system - or maybe (FF uses GL acceleration?!) in combination.
should I have waited more for that and maybe it would recover?
Could imply thrashing, (check "sudo iotop") ie. the kernel juggles files in and out of the RAM cache (because it can't eg. drop anon pages lacking the swap space to do so) - still OOM, though.
Feel encouraged to add a swap file to increase your available memory and see whether that helps you w/ FF.
In very doubt, you can always abuse head to explicitly reclaim your RAM from the GTT…
the ram usage that was 7.5 after gpu benchmark went down to 2.5GB
The aggregated values are not all that helpful (the fancy RAM meters for desktop widgets are that: "fancy")
The GTT should™ only take ~3GB, so 2.5+3=5.5, that means 2GB would be taken by sth. different
=> /proc/meminfo before and after the memory release for more insight on that.

sigh.
I think I add that swap file.
btw the fancy ram apps include htop too?
when I can get the non-fancy ones? ony /proc/meninfo?
where I can see exactly what app (or kernel thing?) is using that wasted ram that gets released bad when I abuse the tail app?

seth · 2021-11-21 08:24:57

htop tells you more than the fancy dials, but you didn't communicate that information
The "wasted" ram will be in the GTT or file cache (other stuff cannot be released) - the file cache is visible in "free -h" (and more detailed in proc/meminfo) - the GTT apparently nowhere (at least no generic tool) - but it's assigned to your GPU and likely claimed by the GPUs kernel module.

Arch Linux

#26 2021-11-19 15:59:15

Re: high ram usage after a latest update?

#27 2021-11-19 17:02:09

Re: high ram usage after a latest update?

#28 2021-11-19 17:49:59

Re: high ram usage after a latest update?

#29 2021-11-20 16:37:18

Re: high ram usage after a latest update?

#30 2021-11-21 00:06:01

Re: high ram usage after a latest update?

#31 2021-11-21 08:24:57

Re: high ram usage after a latest update?

Board footer