You are not logged in.

#1 2013-10-10 20:06:24

tazmanian
Member
Registered: 2009-10-25
Posts: 39

[solved] /var mysteriously filling up

My /var partition keeps filling up. I have already resized the partition several times, and it fills up in a matter of minutes. I don't understand where the space is going. Below is the relevant excerpt from df:

/dev/mapper/var        754M  716M     0 100% /var
/dev/mapper/var.cache  1.7G  1.5G   70M  96% /var/cache

I have /var/cache on a separate partition, so the pacman cache isn't an issue. df reports 716 MB used. My first suspicion was logging, but journalctl reports using a modest 5 MB:

# journalctl --disk-usage
Journals take up 5.2M on disk.

The weird thing is, du only shows 363 MB in use, which is less than half the partition:

# du -xs /var
363M    /var

Anyone have ideas on how to debug this?

Last edited by tazmanian (2013-10-12 07:29:04)

Offline

#2 2013-10-10 20:23:54

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 12,136
Website

Re: [solved] /var mysteriously filling up

tazmanian wrote:

My /var partition keeps filling up. I have already resized the partition several times, and it fills up in a matter of minutes. I don't understand where the space is going. Below is the relevant excerpt from df:

/dev/mapper/var        754M  716M     0 100% /var
/dev/mapper/var.cache  1.7G  1.5G   70M  96% /var/cache

I have /var/cache on a separate partition, so the pacman cache isn't an issue. df reports 716 MB used. My first suspicion was logging, but journalctl reports using a modest 5 MB:

# journalctl --disk-usage
Journals take up 5.2M on disk.

The weird thing is, du only shows 363 MB in use, which is less than half the partition:

# du -xs /var
363M    /var

Anyone have ideas on how to debug this?

community/ncdu.

Last edited by WorMzy (2013-10-10 20:24:48)


Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Online

#3 2013-10-10 20:56:07

hiciu
Member
Registered: 2010-08-11
Posts: 84

Re: [solved] /var mysteriously filling up

tazmanian wrote:

The weird thing is, du only shows 363 MB in use, which is less than half the partition:

# du -xs /var
363M    /var

Anyone have ideas on how to debug this?

In Unix you can do something like this:

#define FILE "/path/to/file"
int fd = open(FILE, O_WRONLY | O_CREAT | O_TRUNC);
remove(FILE);
char buf[1024] = {0};
while (0) {
   write(fd, buf, sizeof(buf));
}

This will create file, delete it and write to deleted file. df will report space on disk is taken, but du and ncdu report that space is empty. This technique is used when program needs to use some space as a buffer and it don't need that data in memory.

You can check for open but deleted file with `sudo lsof | grep deleted`.

Last edited by hiciu (2013-10-10 20:56:24)

Offline

#4 2013-10-10 21:43:39

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,621
Website

Re: [solved] /var mysteriously filling up

Use ncdu:

% sudo ncdu -x /var
ncdu 1.10 ~ Use the arrow keys to navigate, press ? for help                                                                    
--- /var -----------------------------------------------------------------------------------------------------------------------
    1.1GiB [##########] /cache                                                                                                  
   32.8MiB [          ] /lib
    1.2MiB [          ] /spool
    8.0KiB [          ] /db
@   4.0KiB [          ]  lock
@   4.0KiB [          ]  mail
@   4.0KiB [          ]  log
@   4.0KiB [          ]  run
    0.0  B [          ] /tmp
e   0.0  B [          ] /opt
e   0.0  B [          ] /lost+found
e   0.0  B [          ] /local
e   0.0  B [          ] /games
e   0.0  B [          ] /empty

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#5 2013-10-12 07:27:55

tazmanian
Member
Registered: 2009-10-25
Posts: 39

Re: [solved] /var mysteriously filling up

hiciu wrote:

You can check for open but deleted file with `sudo lsof | grep deleted`.

Thanks, this helped tremendously. I did think to check lsof for open deleted files, but didn't notice the "(deleted)" tag in the output. Looks like krunner is the culprit.

Offline

Board footer

Powered by FluxBB