You are not logged in.

#1 2004-12-13 16:51:11

ErikV
Member
Registered: 2004-12-13
Posts: 10

big mem usage by pacman closes X

hi folks,

Not sure whether this is the right forum part, and didn't use the search although I've seen this (http://bbs.archlinux.org/viewtopic.php? … ec932ca868)

The thing is this:

I've read some topic about having a local mirror and started expirimenting.

[root@scrappy-x86 erik]# pacman -Sl testing current extra | awk -F " " '{print $2}' ORS=" " > bla

[root@scrappy-x86 erik]# pacman -Sw `more ba`

First I had to delete autorespond cuz the dependencies couldn't be met. Second time it used memory, more memory, more memory until I ran out of memory and swap (112mb ram, 256mb swap), then the kernel killed X according to /var/log/kernel

Dec 13 14:28:52 scrappy-x86 kernel: __alloc_pages: 0-order allocation failed (gfp=0x1f0/0)
Dec 13 14:28:52 scrappy-x86 kernel: __alloc_pages: 0-order allocation failed (gfp=0x1f0/0)
Dec 13 14:28:52 scrappy-x86 kernel: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
Dec 13 14:28:52 scrappy-x86 kernel: __alloc_pages: 0-order allocation failed (gfp=0x1f0/0)
Dec 13 14:28:52 scrappy-x86 kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
Dec 13 14:28:52 scrappy-x86 kernel: VM: killing process X
Dec 13 14:28:52 scrappy-x86 kernel: __alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
Dec 13 14:28:52 scrappy-x86 kernel: VM: killing process xfce4-panel

Now, I don't want a local repo or something, but I want to know if it's usual the kernel closes down apps other than the one is using too much mem, and doesn't close pacman. Next time I ran pacman I had to remove pacman.lck

Oh, and yes, pacman is fully updated and for some reason there's nothing left in /var/log/pacman.log after this event.

/edit
/off-topic
w000t, 1 4m us3r 2600

Offline

#2 2004-12-13 16:59:01

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: big mem usage by pacman closes X

hmmm, do you have a swap partition?
things like this happen when there is no swap available for the kernel

[offtopic]why is it that people dislike swap partitions?[/offtopic]

Offline

#3 2004-12-13 17:04:35

ErikV
Member
Registered: 2004-12-13
Posts: 10

Re: big mem usage by pacman closes X

yep, I've 256 megs of swap on a separate swap partition, but why doesn't the kernel close pacman down instead of X?

Offline

#4 2004-12-13 17:05:57

kakabaratruskia
Member
From: Santiago, Chile
Registered: 2003-08-24
Posts: 596

Re: big mem usage by pacman closes X

In the pacman webpage there's some information about a bug in libtar, that can cause what you are saying. There's also a patch.
http://archlinux.org/pacman/


And where were all the sportsmen who always pulled you though?
They're all resting down in Cornwall
writing up their memoirs for a paper-back edition
of the Boy Scout Manual.

Offline

#5 2004-12-13 20:05:50

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: big mem usage by pacman closes X

ErikV wrote:

yep, I've 256 megs of swap on a separate swap partition, but why doesn't the kernel close pacman down instead of X?

I would assume the kernel says "pacman is requesting more memory, but I'm out - let's kill some processes so I can give pacman more memory"

Offline

#6 2004-12-14 07:30:26

apeiro
Daddy
From: Victoria, BC, Canada
Registered: 2002-08-12
Posts: 771
Website

Re: big mem usage by pacman closes X

There are a couple memleaks in the dependency-resolution functions in the current version.

I'll release another version soon with fixes included.

Offline

#7 2004-12-14 09:56:37

ErikV
Member
Registered: 2004-12-13
Posts: 10

Re: big mem usage by pacman closes X

phrakture wrote:
ErikV wrote:

yep, I've 256 megs of swap on a separate swap partition, but why doesn't the kernel close pacman down instead of X?

I would assume the kernel says "pacman is requesting more memory, but I'm out - let's kill some processes so I can give pacman more memory"

I thought the linux kernel was smart enough not to let down half the system because of one single app, like in win98 where 1 app which crashes takes your whole system with it.

Offline

#8 2004-12-14 10:22:14

i3839
Member
Registered: 2004-02-04
Posts: 1,185

Re: big mem usage by pacman closes X

OOM handling is tricky and not that well yet. They're working on it, and there seems to be good progress recently, so hopefully it will be much better in newer kernels.

A good start is to disable memory overcommit with `echo 2 > /proc/sys/vm/overcommit_memory`. See for more info linux-2.*/Documentation/vm/overcommit-accounting.

Offline

#9 2004-12-14 15:42:42

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: big mem usage by pacman closes X

ErikV wrote:
phrakture wrote:

I would assume the kernel says "pacman is requesting more memory, but I'm out - let's kill some processes so I can give pacman more memory"

I thought the linux kernel was smart enough not to let down half the system because of one single app, like in win98 where 1 app which crashes takes your whole system with it.

well, it's not taking down your system - it's taking down X... X is just a graphical app... now if it dropped syslogd and other system daemons I'd be worried

Offline

#10 2004-12-14 17:13:17

ErikV
Member
Registered: 2004-12-13
Posts: 10

Re: big mem usage by pacman closes X

sure, it 'only' closes X, but as X is my working environment, it's closing my working env so I can't work further until I do something about

Offline

#11 2004-12-14 18:17:33

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: big mem usage by pacman closes X

ErikV wrote:

sure, it 'only' closes X, but as X is my working environment, it's closing my working env so I can't work further until I do something about

no, I'm not saying it's not bad but here's the thing: you claimed "linux is supposed to be stable so it doesn't crash when things use alot of memory" - that is 100% true... the linux kernel will save itself.  If the kernel and system processes are in trouble of being consumed, the kernel starts wiping out processes... one may happen to be X.
This is much like Aron Ralston, that hiker who got his arm caught under a boulder.... he was in trouble of starving to death, so he cut his own arm off with a pocket knife and walked 2 miles to safety.... yeah X may seem important, but at least the system didn't hardlock (the analogy: yeah an arm may seem important, but at least he didn't starve to death)...

Offline

#12 2004-12-14 19:00:30

ErikV
Member
Registered: 2004-12-13
Posts: 10

Re: big mem usage by pacman closes X

to stay in your story, you first try to move the boulder, and since the kernel is somelike almighty it actually can move the boulder (ie close pacman)

It's like a patient comes to the doctor with an awful pain in his arm. The doctor examines him and looks for the source of the pain, and takes away the pain. (like putting it in chalc (good english?) when it's broken). And he doesn't give him first morfine and let him leave.

Offline

#13 2004-12-14 19:24:09

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: big mem usage by pacman closes X

well here's the thing though - the kernel doesn't know pacman is doing anything wrong - it just knows it wants more memory and the kernel can't provide it...
if there was something in the code that said "hey kernel, I'm a memory leak!" then the kernel would probably want to kill that... but then again if that was in the code, someone would just fix the code itself.

the fact of the matter is that pacman (or libtar actually) is doing something and wants more memory - the only reason the kernel is killing things is because of pacman, therefore it wouldn't kill pacman...

Offline

#14 2004-12-14 19:31:06

ErikV
Member
Registered: 2004-12-13
Posts: 10

Re: big mem usage by pacman closes X

But that should mean that if I make a program that consumes a couple of Gb of ram, the kernel would close down all (user?) programs at the end.

To me it looks far better that if a program wants more memory than is available in your pc the kernel closes that program.

Offline

#15 2004-12-14 19:56:12

i3839
Member
Registered: 2004-02-04
Posts: 1,185

Re: big mem usage by pacman closes X

ErikV wrote:

To me it looks far better that if a program wants more memory than is available in your pc the kernel closes that program.

One catch is that you can't know that the app asking for too much memory is the app causing the memory hogging. Program X and P are running at the same time (say, X and Pacman), P is eating all mem in big chunks, poor program X asks for a bit of mem that isn't available. What to do now?

There's virtual memory to cope with too. Say program A asks for a gig of mem, then it gets 1 gig virtual mem. Only when the app actually uses portions of that mem real ram is reserved and allocated (e.g. in top that's the VIRT versus RES value). So you promised one gig which may not be available later if overcommit is enabled, which it is by default. So prog B starts happily eating your mem, and suddenly A wants to use its promised mem: Ouch.
Easy sollution to avoid this things is to disable overcommit, but that makes running certain memory hogs like java impossible with not enough ram, as you can use less memory in total, thus your ram less efficiently.

So it boils down to the fact that the OOM killer must make a smart decision which is impossible to do always right, complicated by the fact that there's no ram, so doing anything fancy is out of the question.

That said, the current OOM handling is buggy as it kicks in too soon and makes very bad decisions, but as I said earlier, the nice kernel developers are working on it and seem to have fixed it. Now just hope the sollution will be in the next kernel version.

EDIT:
If an app asks for more memory than is really available nothing will be killed, but the app just won't get it. But as said above, it isn't as simple as that.

Offline

#16 2004-12-14 20:15:40

ErikV
Member
Registered: 2004-12-13
Posts: 10

Re: big mem usage by pacman closes X

ah, now I see.

Yep, phrakture you're right.
Learned something this day, something interesting instead of dull fysics and chemistry.
Thanks for the clear explanation.

Offline

#17 2004-12-15 02:03:47

kakabaratruskia
Member
From: Santiago, Chile
Registered: 2003-08-24
Posts: 596

Re: big mem usage by pacman closes X

ErikV wrote:

ah, now I see.

Yep, phrakture you're right.
Learned something this day, something interesting instead of dull fysics and chemistry.
Thanks for the clear explanation.

Your computer is mostly physics.


And where were all the sportsmen who always pulled you though?
They're all resting down in Cornwall
writing up their memoirs for a paper-back edition
of the Boy Scout Manual.

Offline

#18 2004-12-15 06:07:24

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: big mem usage by pacman closes X

kakabaratruskia wrote:

Your computer is mostly physics.

so is your brain

Offline

#19 2004-12-15 13:14:54

kakabaratruskia
Member
From: Santiago, Chile
Registered: 2003-08-24
Posts: 596

Re: big mem usage by pacman closes X

phrakture wrote:
kakabaratruskia wrote:

Your computer is mostly physics.

so is your brain

Well... matter is physics, but I would relate the brain with biology and maybe chemystry first.


And where were all the sportsmen who always pulled you though?
They're all resting down in Cornwall
writing up their memoirs for a paper-back edition
of the Boy Scout Manual.

Offline

#20 2004-12-15 19:39:15

ErikV
Member
Registered: 2004-12-13
Posts: 10

Re: big mem usage by pacman closes X

true..but that's where the word 'dull' comes in

Offline

Board footer

Powered by FluxBB