[Solved] importance of contiguous space for swap

bbus · 2021-12-21 17:00:26

The "X" problem is my configuration for hibernation has not been working reliably for the past month+. My laptop would rarely recover from "suspend-then-hibernate", with only slightly better odds if I called "systemctl hibernate" from command line directly.

I should have tested rolling back a combination of kernel and systemd, (I've rolled back the kernel multiple times in the past, successfully debugging other issues I've had) but by the time I got time to work on this, I was so frustrated I removed my original swap file before attempting those other routes.

So, my "Y" problem is to re-create a swap file to then be able to test if rolling back solves my issues.

My original swap file only had 23 extents, according to "filefrag -v /swapfile" (I created it when the laptop was brand new, immediately after installing arch), but now, now matter what I try (e.g. e4defrag, moving other files off to create a lot more space on the drive, etc.) I cannot get a new swapfile created without a minimum of 100 extents.

Is it absolutely necessary the swapfile is contiguous? Am I correct in assuming "dd" isn't creating a contiguous space because it is still being written to disk inside of the filesystem?

If I understand correctly, the number of extents shows how discontiguous the file is on disk. According to https://wiki.archlinux.org/title/Swap#Manually the swapfile needs to be contiguous because it won't query the filesystem on writes.
-- Another question I had was: does that mean if a different file was stored in between those extents, that the file would be clobbered by the kernel, while it tried to write to the swapfile for hibernation?

Last edited by bbus (2021-12-22 20:49:51)

Lone_Wolf · 2021-12-21 19:36:04

extents ≠ holes

In the past swapfiles on archlinux were often created with fallocate .
Around june 2020 kernel 5.7 complained about it and many of us (links at bottom of post) recreated the swapfile using dd.

I was one of them, created an 8 GiB swapfile that filefrag reports as having 223 extents.

Here's partial output :

# filefrag -v /swapfile 
Filesystem type is: ef53
File size of /swapfile is 8589934592 (2097152 blocks of 4096 bytes)
 ext:     logical_offset:        physical_offset: length:   expected: flags:
   0:        0..    2047:   16824320..  16826367:   2048:            
   1:     2048..    6143:   16818176..  16822271:   4096:   16826368:
   2:     6144..    8191:   16850944..  16852991:   2048:   16822272:
   3:     8192..   10239:   16855040..  16857087:   2048:   16852992:
   4:    10240..   14335:   16859136..  16863231:   4096:   16857088:
   5:    14336..   16383:   17424384..  17426431:   2048:   16863232:
   6:    16384..   20479:   17420288..  17424383:   4096:   17426432:
   7:    20480..   24575:   17459200..  17463295:   4096:   17424384:
   8:    24576..   30719:   17655808..  17661951:   6144:   17463296:
   9:    30720..   43007:   17682432..  17694719:  12288:   17661952:
  10:    43008..   45055:   17704960..  17707007:   2048:   17694720:

The other 200+ lines show the same thing : the extents are created contiguous .

https://bbs.archlinux.org/viewtopic.php?id=256614
https://bugs.archlinux.org/task/66921
https://bugs.archlinux.org/task/66979

bbus · 2021-12-21 21:40:49

Thanks for the links. I had originally used dd to create my swap, before it was popular, I guess. (I think the primary reason was that I was already very familiar and comfortable with dd, and had not {still haven't} ever used fallocate.)
My confusion then stems from the difference between logical offset and physical offset. I had assumed (a) "contiguous" meant it had to be in the physical offset space, because of the statement in https://man.archlinux.org/man/swapon.8#Files_with_holes:

The swap file implementation in the kernel expects to be able to write to the file directly, without the assistance of the filesystem.

I also assumed (b) the mapping from logical offset to physical offset was managed by the filesystem. One or both of those assumptions must be wrong?

After reading your post, I searched for more info on this, and found https://sites.cs.ucsb.edu/~rich/class/c … index.html. It is a lot to parse, and I'm only part of my way reading through it. In section F it has:

by knowing the inode number and the file system (identified by the device number on which the file system resides) in which that inode resides, the kernel can locate the file to which that inode refers

So that confuses me on what the relationship is between my incorrect assumptions, how "contiguous" is defined, and what the kernel is doing on its own, without the file system. Do you by chance have any short, clarifying ways to say it, or, some other sources I can look into?

edit:
Thanks for posting a sample of your filefrag output. The biggest reason I'm trying to understand all of this is extent 6 to 7 on mine goes from a physical offset near the end of the disk to one close to the beginning. Trying to understand how that doesn't mess up what seems to be a blind write on hibernation, and a blind read on wake, if it has to jump around the physical locations on the disk.

edit 2:
I've also just found this https://stackoverflow.com/a/47625268:

Most disks these days do logical I/O in hardware. Operating systems rarely have to do physical I/O these days.

So if the kernel only cares about contiguous logical offsets (as opposed to physical), and can do so without the help of the file system, where is the mapping contained? In the disk hardware controller? This seems like a strange solution, kind of hope this is wrong.

Last edited by bbus (2021-12-21 22:19:02)

loqs · 2021-12-22 01:38:36

Perhaps this comment from swapfile.c in the kernel documenting swap extents helps https://github.com/torvalds/linux/blob/ … le.c#L2355

bbus · 2021-12-22 02:24:25

Wow, yes, that does! If I understand correctly, it does lean on the filesystem at the "swapon" event, creating its list, but from then on it doesn't need the filesystem?
Does the swapon/swapoff get called at each boot/poweroff? If so, I wonder why it wouldn't make that list at mkswap, then only create the lock at swapon/swapoff?

Lone_Wolf · 2021-12-22 20:31:24

where is the mapping contained? In the disk hardware controller? This seems like a strange solution, kind of hope this is wrong.

Nope, it's right. To understand how that works you have to investigate the history of hard disc drives & controllers which began in the 1950s , so 60+ years ago.
Also investigate CHS (Cylinder Head Sector) , LBA (Logical Block Addressing) , MBR (Master Boot Record) and GPT (GUID Partition Table) .
(links at bottom)

A few personal notes that you may find interesting or useful :

In the 1980's / early 1990's hard disk controllers migrated from motherboards chipsets to being integrated in harddrives .
Tech people often claimed the hard disk controllers were more complicated then the cpus of that era.

For most purposes SSDs can and will be treated as fast harddrives

before Integrated Drive Electronics made things a lot easier the results of configuring wrong hdd characteristics in firmware varied from non-booting systems to loosing data when trying to write to non-existing drive areas.

https://en.wikipedia.org/wiki/Hard_disk_drive
https://en.wikipedia.org/wiki/Cylinder-head-sector
https://en.wikipedia.org/wiki/Logical_block_addressing
https://en.wikipedia.org/wiki/Master_boot_record
https://en.wikipedia.org/wiki/GUID_Partition_Table

bbus · 2021-12-22 20:49:33

Awesome, thank you for the notes and links! I'll read up on it.

Marking solved, and Happy Holidays

Arch Linux

#1 2021-12-21 17:00:26

[Solved] importance of contiguous space for swap

#2 2021-12-21 19:36:04

Re: [Solved] importance of contiguous space for swap

#3 2021-12-21 21:40:49

Re: [Solved] importance of contiguous space for swap

#4 2021-12-22 01:38:36

Re: [Solved] importance of contiguous space for swap

#5 2021-12-22 02:24:25

Re: [Solved] importance of contiguous space for swap

#6 2021-12-22 20:31:24

Re: [Solved] importance of contiguous space for swap

#7 2021-12-22 20:49:33

Re: [Solved] importance of contiguous space for swap

Board footer