Reserved memory: is it really hardware?

Hanabishi · 2022-09-14 13:50:06

I think many people noticed that you have less usable RAM than physically installed. On my desktop PC I always seen ~15.5G total (according to kernel log, meminfo or any external tool) out of 16G installed.
It may seem negligible being like ~3% loss, but in absolute terms 500M is fairly solid amount and seems too high for "basic hardware needs".

For years I kinda supposed this is normal, but eventually I ended up diving in it. And yes, I found some related topics here (1, 2) and read the articles.
Answering the obvious question: I have Ryzen 1600 installed which has no integrated graphics (the whole 1st gen ryzen series has no iGPUs whatsoever), so this is definitely not the case.

Just to check my sanity, other PC on different platform with completely different hardware states 3.74G usable out of 4G. In this case its more painful ~7% loss, but the absolute value is lower though. This PC has iGPU, but in bios dedicated memory for it set to the lowest possible amount of 32M, so it should be negligible.

And turns out this is not so "hardware". I discovered that iommu=off kernel parameter actually reduces the reserved memory amount by ~100M on my PC and now I have 15.6G of usable RAM. On the other PC it results in 3.80G usable which is ~60M increase.
I tried to tweak some parameters to push it further, but no luck:

Memory: 16249860K/16688604K available (16391K kernel code, 2091K rwdata, 11104K rodata, 2124K init, 3412K bss, 438484K reserved, 0K cma-reserved)

438484K reserved still seems too much and I can't find a clear explanation for where it goes.

Just for reference, I even installed Windows to check does it behaves the same way, as I don't remember having such issue with it in the past.
And the answer is: not really. Windows reports only 84 MB of hardware reserved RAM and states that ~15.9G is usable. This is definitely seems more reasonable for hardware needs.

So the question is: does someone can explain it and maybe recommend some tweaks?

Last edited by Hanabishi (2022-09-14 13:50:27)

mpan · 2022-09-14 16:13:49

RAM addresses are reserved based on what is reported by the firmware.⁽¹⁾ There is nothing you can do about that, other than physically removing devices or turning off some features. The second option is mostly of historical significance, as modern computers do not offer that possibility. To satisfy your curiosity, you may see /proc/iomem, as it may contain some useful information:

sudo cat /proc/iomem

You did not say, what exactly are the issues you are experiencing due to that 500MB being reserved. So it’s impossible to help you solving those issues. But, whatever they are, if 500M out of 16G missing is causing any noticeable trouble, I would say it’s a sign of major problem elsewhere. Because the expected effect is like… dunno… 0.1% decrease in disk I/O performance?
____
⁽¹⁾ Or BIOS if you have an older system. With traditional BIOS systems it was data passed directly from hardware. Nowadays the reserved regions map may be augmented by firmware.

Hanabishi · 2022-09-14 16:38:36

RAM addresses are reserved based on what is reported by the firmware.⁽¹⁾ There is nothing you can do about that, other than physically removing devices or turning off some features.

I read the articles and understand that. The actual question is which features reserve the memory (like iommu) and can they be disabled.

To satisfy your curiosity, you may see /proc/iomem, as it may contain some useful information:

I already digged in it, but many lines just say "Reserved". If someone can figure out something from it, you are welcome:

00000000-00000fff : Reserved
00001000-0009ffff : System RAM
000a0000-000fffff : Reserved
  00000000-00000000 : PCI Bus 0000:00
  000a0000-000dffff : PCI Bus 0000:00
    000c0000-000cebff : Video ROM
  000f0000-000fffff : System ROM
00100000-09cfefff : System RAM
09cff000-09ffffff : Reserved
0a000000-0a1fffff : System RAM
0a200000-0a20afff : ACPI Non-volatile Storage
0a20b000-0affffff : System RAM
0b000000-0b01ffff : Reserved
0b020000-bc456017 : System RAM
bc456018-bc473257 : System RAM
bc473258-bc474017 : System RAM
bc474018-bc482057 : System RAM
bc482058-c3282fff : System RAM
c3283000-c3283fff : Reserved
c3284000-c87f5fff : System RAM
c87f6000-c87f6fff : Reserved
c87f7000-ca383fff : System RAM
ca384000-ca6adfff : Reserved
ca6ae000-ca6e7fff : ACPI Tables
ca6e8000-cad87fff : ACPI Non-volatile Storage
cad88000-cb9fefff : Reserved
cb9ff000-ccffffff : System RAM
cd000000-cfffffff : Reserved
d0000000-fec2ffff : PCI Bus 0000:00
  e0000000-f01fffff : PCI Bus 0000:07
    e0000000-efffffff : 0000:07:00.0
    f0000000-f01fffff : 0000:07:00.0
  f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f]
    f8000000-fbffffff : Reserved
      f8000000-fbffffff : pnp 00:00
  fc900000-fcbfffff : PCI Bus 0000:08
    fc900000-fc9fffff : 0000:08:00.3
      fc900000-fc9fffff : xhci-hcd
    fca00000-fcafffff : 0000:08:00.2
      fca00000-fcafffff : ccp
    fcb00000-fcb01fff : 0000:08:00.2
      fcb00000-fcb01fff : ccp
  fcc00000-fcdfffff : PCI Bus 0000:01
    fcc00000-fccfffff : PCI Bus 0000:02
      fcc00000-fccfffff : PCI Bus 0000:03
        fcc00000-fcc03fff : 0000:03:00.0
        fcc04000-fcc04fff : 0000:03:00.0
          fcc04000-fcc04fff : r8169
    fcd00000-fcd7ffff : 0000:01:00.1
    fcd80000-fcd9ffff : 0000:01:00.1
      fcd80000-fcd9ffff : ahci
    fcda0000-fcda7fff : 0000:01:00.0
      fcda0000-fcda7fff : xhci-hcd
  fce00000-fcefffff : PCI Bus 0000:09
    fce00000-fce07fff : 0000:09:00.3
      fce00000-fce07fff : ICH HD audio
    fce08000-fce08fff : 0000:09:00.2
      fce08000-fce08fff : ahci
  fcf00000-fcffffff : PCI Bus 0000:07
    fcf00000-fcf3ffff : 0000:07:00.0
    fcf60000-fcf63fff : 0000:07:00.1
      fcf60000-fcf63fff : ICH HD audio
  fd100000-fd1fffff : Reserved
  fea00000-fea0ffff : Reserved
  feb00000-feb00007 : SB800 TCO
  feb80000-fec01fff : Reserved
    feb80000-febfffff : pnp 00:01
    fec00000-fec003ff : IOAPIC 0
    fec01000-fec013ff : IOAPIC 1
  fec10000-fec10fff : Reserved
    fec10000-fec10fff : pnp 00:04
fec30000-fec30fff : Reserved
  fec30000-fec30fff : AMDIF030:00
    fec30000-fec30fff : AMDIF030:00 AMDIF030:00
fed00000-fed00fff : Reserved
  fed00000-fed003ff : HPET 0
    fed00000-fed003ff : PNP0103:00
fed40000-fed44fff : Reserved
fed80000-fed8ffff : Reserved
  fed81500-fed818ff : AMDI0030:00
fedc0000-fedc0fff : pnp 00:04
fedc2000-fedcffff : Reserved
fedd4000-fedd5fff : Reserved
fee00000-ffffffff : PCI Bus 0000:00
  fee00000-fee00fff : Local APIC
    fee00000-fee00fff : pnp 00:04
  ff000000-ffffffff : Reserved
    ff000000-ffffffff : pnp 00:04
100000000-42f37ffff : System RAM
  12c800000-12d801e4b : Kernel code
  12da00000-12e4d7fff : Kernel rodata
  12e600000-12e80ad7f : Kernel data
  12eeab000-12f1fffff : Kernel bss
42f380000-42fffffff : Reserved

You did not say, what exactly are the issues you are experiencing due to that 500MB being reserved.

There are no particular issues with it (beside the memory wastage). I'm just curious and want an explanation. And if "There is nothing you can do about that", how Windows manages to use like 5x less memory for it.
Also I'm interested how it affects other people with different hardware sets.

Last edited by Hanabishi (2022-09-14 17:05:37)

Lone_Wolf · 2022-09-15 09:26:40

The actual question is which features reserve the memory (like iommu) and ~~and can they be~~ what functionality will I miss when they are disabled ?

With IOMMU it depends on age and brand/type of processor .
AMD has made extensive use of IOMMU for 64 bit OSes since they launched their first 64-bit processor ( 2005? unsure) , intel didn't support it before approx 2015 .

You can try disabling it, but don't be surprised if performance goes down or weird things happen.

Hanabishi · 2022-09-15 09:53:12

what functionality will I miss when they are disabled ?

I want to know possible options anyway and then decide.

You can try disabling it, but don't be surprised if performance goes down or weird things happen.

Sure, if some weird stuff will happen, I will just turn it back, not a big deal. As for performance, at least wiki info states the opposite.

sabroad · 2022-09-15 11:52:02

Hanabishi wrote:

Memory: 16249860K/16688604K available (16391K kernel code, 2091K rwdata, 11104K rodata, 2124K init, 3412K bss, 438484K reserved, 0K cma-reserved)

Linux includes kernel memory in "Reserved" memory.

Hanabishi wrote:

Windows reports only 84 MB of hardware reserved RAM

Windows doesn't include kernel memory in what it calls "Hardware Reserved" memory.

Why did you think these were the same thing?

Hanabishi · 2022-09-15 12:14:07

Linux includes kernel memory in "Reserved" memory.

This is reasonable assumption, but I already tested it too. Tweaking kernel buffers, loading/unloading kernel modules does not affect the value. It always stays exactly the same.
It even stays the same when I boot the Arch installation ISO. 500M just for the kernel, without even running a graphical environment, seems too much anyway, I think.
It only changes when I run the system on different hardware. As I described above, running on other PC makes this value 2x less without any clear reason.

Why did you think these were the same thing?

I don't know. Thats why the topic was created, because I did not found any explanation in the net.

Last edited by Hanabishi (2022-09-15 12:30:33)

sabroad · 2022-09-15 12:43:08

Hanabishi wrote:

It only changes when I run the system on different hardware. As I described above, running on other PC makes this value 2x less without any clear reason.

The kernel maintains data structures [1] that tracks how memory is being used. When more memory is added to a system, the kernel will need to reserve more memory for these internal data structures - as much as is needed to track how memory is being /and will be/ used.

So it's only natural that the kernel will use less memory on a 4GB system than a 16GB system.

Edit: [1] dive on struct page: https://blogs.oracle.com/linux/post/min … e-overhead

Last edited by sabroad (2022-09-15 13:17:20)

Hanabishi · 2022-09-15 13:18:19

Indeed. When I remove a stick and leave only 8G, the value slightly drops to 344824K reserved (iommu enabled). There is obvious correlation here.
Well, I think such explanation is fair enough.

So the second half: any other kernel options to tickle it?

sabroad · 2022-09-15 16:19:41

We're into the tall grass of diminishing returns now.

Hanabishi wrote:

438484K reserved still seems too much and I can't find a clear explanation for where it goes.

Let's summarise:

438484K reserved
  189867K PCI Bus 0000:00
    64965K 0000:07:00.0 graphics AGP aperture (probably)
  229376K struct page
  19241K other reserved memory (kernel + hardware)

Bearing in mind that it's the BIOS that controls and sets out hardware address space, and "struct page" is memory bookkeeping, do you want to look for gains on a 16G system in the 19M "other" category?

Last edited by sabroad (2022-09-15 16:36:20)

Hanabishi · 2022-09-15 16:48:33

Thanks for the summary. I already realized some pci relation too.

Bearing in mind that it's the BIOS that controls and sets out hardware address space

Some pci kernel options actually have effect. E.g. pci=nommconf slightly reduces the value, but pci=nobar slightly increases it.
I just experimenting with it, but yeah, I doubt that significant difference can be achieved.

Arch Linux

#1 2022-09-14 13:50:06

Reserved memory: is it really hardware?

#2 2022-09-14 16:13:49

Re: Reserved memory: is it really hardware?

#3 2022-09-14 16:38:36

Re: Reserved memory: is it really hardware?

#4 2022-09-15 09:26:40

Re: Reserved memory: is it really hardware?

#5 2022-09-15 09:53:12

Re: Reserved memory: is it really hardware?

#6 2022-09-15 11:52:02

Re: Reserved memory: is it really hardware?

#7 2022-09-15 12:14:07

Re: Reserved memory: is it really hardware?

#8 2022-09-15 12:43:08

Re: Reserved memory: is it really hardware?

#9 2022-09-15 13:18:19

Re: Reserved memory: is it really hardware?

#10 2022-09-15 16:19:41

Re: Reserved memory: is it really hardware?

#11 2022-09-15 16:48:33

Re: Reserved memory: is it really hardware?

Board footer