Swap performance with PCI solid-state storage: share your experiences?

karkhaz · 2014-08-01 05:05:18

Does anybody here have a swap partition(s) on a PCI-based solid-state device (like this one)? I'd like to know if it's noticable when your PC starts thrashing, how fast access to swapped out data is compared to data in memory, etc.

Context: my PC has a measly 32GB of RAM. I'm running experiments which memout even when I run them one at a time, and ideally I'd like to be able to run them in parallel... I tried using a SATA3-based solid state disk for a swap partition, but it makes the performance unbearable---my experiments spend most of their time in disk sleep, as reported by htop, waiting to retrieve data that's been swapped out. When I get my studentship, I may build a machine with 256GB or so of RAM, but that's really expensive and if I can get away with a fast swap device then I'd like to do so.

Last edited by karkhaz (2014-08-01 05:06:47)

jasonwryan · 2014-08-01 05:07:56

karkhaz wrote:

my PC has a measly 32GB of RAM.

Measly?

karkhaz · 2014-08-01 05:15:38

jasonwryan wrote:

Measly?

Of course it's overkill for general use...for my nicely tuned Arch setup, 4GB is more than enough for everything except experiments. But I'm running big ol' program analyses on large codebases (apache, linux, ...) and that can rinse out my RAM in moments.

The eventual solution is this motherboard and lots of 16GB sticks of RAM (32GB sticks cost far too much), some Xeons, and all the associated trimmings. But if somebody tells me that they've found PCI storage to be reasonably performant, then my wallet will be ever so slightly happier

Last edited by karkhaz (2014-08-01 05:23:30)

lucke · 2014-08-01 05:58:18

Consider using zswap to limit disk I/O. https://www.ibm.com/developerworks/comm … y7?lang=en

karkhaz · 2014-08-01 06:15:16

lucke wrote:

Consider using zswap to limit disk I/O. https://www.ibm.com/developerworks/comm … y7?lang=en

Wow, fascinating! I'm going to try this out right now. It won't solve all my problems, but may mitigate some of them significantly. Thanks!

Some time later... So this has been of some use, thanks. My problem is that I need _way_ more RAM than my machine has, and so eventually even the zswap space gets used up and I'm back to standard swap disk again. Nevertheless, I think I notice some improvements, and I feel like I can schedule parallel jobs. Still hoping to hear from folks who have used PCI storage before I mark this as solved, but thanks again!

Last edited by karkhaz (2014-08-01 06:59:35)

lucke · 2014-08-01 11:00:13

By default zswap uses up to 20% (I believe) of your RAM, you can change it via /sys/module/zswap/parameters/max_pool_percent. You can also change the compressor and change the size of compressed pages above which they get sent to the disk. There's also zram, which puts everything swapped out in RAM. Maybe you can achieve somewhat better results. You can use dstat to monitor paging and other things nicely; /sys/kernel/debug/zswap has zswap's stats.

Arch Linux

#1 2014-08-01 05:05:18

Swap performance with PCI solid-state storage: share your experiences?

#2 2014-08-01 05:07:56

Re: Swap performance with PCI solid-state storage: share your experiences?

#3 2014-08-01 05:15:38

Re: Swap performance with PCI solid-state storage: share your experiences?

#4 2014-08-01 05:58:18

Re: Swap performance with PCI solid-state storage: share your experiences?

#5 2014-08-01 06:15:16

Re: Swap performance with PCI solid-state storage: share your experiences?

#6 2014-08-01 11:00:13

Re: Swap performance with PCI solid-state storage: share your experiences?

Board footer