How does swap in Linux work?

meph · 2012-02-26 02:59:31

Ok so I just tried the same thing as you did, opened a big file several times in vim to check what's gonna happen.

[meph@meph-ntb ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          3798       3688        109          0          0        158
-/+ buffers/cache:       3528        269
Swap:         4628        435       4193

What happened when I opened it the last time was that my system completely froze and hdd went nuts, after a good minute or so it became responsive again. Sort of. Mouse cursor is lagging and sometimes freezes completely, everything takes a lot longer than it usually does but in the end it does what I want it to do. Can still switch between open windows, open new terminal sessions, actually right now I'm writing this with all those vims still open. It's not pleasant and I would probably bang my head against a wall after a while, but it more or less works. That looks like what you described, when you opened the iso images, doesn't it? There may be something specific to your virtualization software, if i remember correctly then it is possible to set up which application gets to be swapped and which don't. So if the VM application eats all your ram and pushes essential things into swap, then it might virtually lock up your system. This is just a wild theory though, we'd need a true guru to elaborate on this.

hdparm -t /dev/sda

What's the output when your run this while your system is working fine?

edit: and by the way no, I don't think that setting up swap file is needed. It was just an idea, it's unlikely to change anything. Spaghetti method

Last edited by meph (2012-02-26 03:01:04)

Lockheed · 2012-02-26 07:02:36

It would be a bugger if it turned out to be normal behaviour.

/dev/sda:
Timing buffered disk reads: 174 MB in 3.01 seconds = 57.79 MB/sec

Last edited by Lockheed (2012-02-26 07:03:45)

meph · 2012-02-26 11:07:23

Somewhere I read an opinion saying that linux actually having to use swap already is a plan B, and that the only possible solution when you regurarly run out of memory is to get more of it, swap won't save you. Makes some sense I guess.

So far, besides your subjective feeling, we haven't found anything wrong with your system. Your disk speed is normal as well. Might as well be normal behavior.

masteryod · 2012-02-26 13:19:20

meph wrote:

Out of topic comment to the SSD discussion. Lots of these are just rumors and urban legends, currently most SSD's have a lifespan of 1-5 million write cycles.[...]

I'm afraid that reality is much worse, for example, see second table: http://www.anandtech.com/show/5067/unde … tlc-nand/2 (I know this is just a single cell durability, not whole disk but situation is not very good and still SSDs (MLC = affordable to most of the people) are not recommended to work in high write/erase environment - it's interesting what future will bring in that manner)

meph · 2012-02-26 19:14:25

masteryod wrote:

meph wrote:
Out of topic comment to the SSD discussion. Lots of these are just rumors and urban legends, currently most SSD's have a lifespan of 1-5 million write cycles.[...]
I'm afraid that reality is much worse, for example, see second table: http://www.anandtech.com/show/5067/unde … tlc-nand/2 (I know this is just a single cell durability, not whole disk but situation is not very good and still SSDs (MLC = affordable to most of the people) are not recommended to work in high write/erase environment - it's interesting what future will bring in that manner)

That's some very interesting reading, definitely gave me something to think about. Check this article as well, it's on the same website, though a bit older. If you want tl;dr - 7GB write per day on a 203GB ssd means that if wear management is perfect, in 36 days you will write to every single cell (more info on this in the article). With 10.000 write cycles per cell it would mean that your sdd will wear out in 986 years. Talking about SLC/MLC/TLC kind of loses its significance. Obviously reality will be somewhat different, but you get the idea. You're right that it's something to be considered in high traffic environment, but in a personal desktop computer, based on these numbers, it's likely that your ssd will eventually die of something else, before it gets a chance to wear out.

On the other hand, I have to admit that I lack any serious knowledge about the matter, so what I said is just google and simple logic talking. But as of now, I yet have to see a real test that would prove ssd's wear out quickly, one backed up by solid numbers. Everytime I do a quick research on the matter, I end up with an article that says it's not as bad as people say, and it's backed up by math. If you have something to disprove that, then I'll happily be educated. The article you linked, however interesting it is, provides a lot of information on how things work internally, but lacks some information on how it will reflect into real life usage.

Either way, this is a very interesting discussion, but I guess we should move it elsewhere before it gets moved for us

Lockheed · 2012-02-26 19:53:23

Getting back on topic - I'm just really surprised this would be the normal behaviour for linux running on swap. Would that be one single area where Windows is superior to Linux?

lucke · 2012-02-26 20:08:34

Did you try it under Windows? You said that Windows behaved pretty much as my Arch when it runs out of memory.

Maybe it doesn't really have things to swap out? I don't normally use VMs myself, except for testing some distro every now and then. I don't have great problems when I max out my 2 GB. Maybe I could say something more about your situation if I saw the output of smem/free/dstat. I sometimes use the script pasted in https://bbs.archlinux.org/viewtopic.php … 3#p1062643 to see how my systems behave under great memory pressure.

meph · 2012-02-26 20:38:02

Windows pages stuff over time, while linux tries to avoid using swap for as long as it can and only do it at the very last moment. Windows' strategy probably is better to even out load over time, while linux tries to minimize disk activity, because disk already is a bottleneck for most systems. All this means that windows possibly handles alternating load situations better, because it did part of the job beforehand, when your system was idle. And let's face it, it's the most common situations for us desktop users. So it might be better, but only in situations when you actually do run out of ram. If you don't, and everything is done live in ram with minimal disk i/o, the linux approach is possibly better.

2 cents.

lucke · 2012-02-26 20:57:30

The swappiness determines whether Linux does it gradually or at the last moment.

Last edited by lucke (2012-02-26 22:21:00)

masteryod · 2012-02-26 21:35:05

@Lockheed: another workaround, do you have discrete graphic card with a lot of vram? you can do ramdisk on it and use as swap

@meph: yeah I know that real life predictions are sane but still SSD is new technology and current state of it is not very optimistic. I'm very curious about cell longevity issue because is decreasing faster than prices are dropping. And as today I would be more troubled about firmware bugs than lifespan

Lockheed · 2012-02-26 22:02:14

masteryod wrote:

@Lockheed: another workaround, do you have discrete graphic card with a lot of vram? you can do ramdisk on it and use as swap

It wouldn't be much. It's 128MB and I need it, anyway.

Changing swappiness from 10 to 50 did not make a difference to me.

meph · 2012-02-26 23:12:44

Swappiness indeed does affect this, but from my experience, it's still different. On a system with enough ram (say 4GB) you can set swappiness to 100, start running applications and your swap still will be empty. It does set swap aggressivity, but the general approach is still different to windows. For example, by default there's no preloading in linux - no prefetch or superfetch. That alone makes a lot of difference in memory usage.

@masteryod only time will tell I guess. What you say is true, on the other hand I believe that the current hdd situation can't last forever either. 5 years ago I wouldn't dream of having a 2TB hard drive in my home computer, but transfer speed is more or less the same. Realistically, show me a hard drive that will fully utilize SATA III. The hard drive technology just won't cut it anymore. With ssd you get much lower latency and random access time, no fragmentation (practically, not technically), even data transfer rates are continuously getting better. Right now I agree with you, I'm not using one, but in a few years time I'm pretty sure I will. I do believe they are the future

Arch Linux

#26 2012-02-26 02:59:31

Re: How does swap in Linux work?

#27 2012-02-26 07:02:36

Re: How does swap in Linux work?

#28 2012-02-26 11:07:23

Re: How does swap in Linux work?

#29 2012-02-26 13:19:20

Re: How does swap in Linux work?

#30 2012-02-26 19:14:25

Re: How does swap in Linux work?

#31 2012-02-26 19:53:23

Re: How does swap in Linux work?

#32 2012-02-26 20:08:34

Re: How does swap in Linux work?

#33 2012-02-26 20:38:02

Re: How does swap in Linux work?

#34 2012-02-26 20:57:30

Re: How does swap in Linux work?

#35 2012-02-26 21:35:05

Re: How does swap in Linux work?

#36 2012-02-26 22:02:14

Re: How does swap in Linux work?

#37 2012-02-26 23:12:44

Re: How does swap in Linux work?

Board footer