Many wget instances to mirrors on mirrorlist (called by pkgfile)

ngoonee · 2010-04-13 23:27:21

Hi all,

Every once in a while I see my download speed hitting its limit (50-60 kb/s, slow I know) on my laptop. Just today I thought of running a proper netstat while this happens, I get the following:-

[ngoonee@ngoonee-laptop ~]$ sudo netstat -tp
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name   
tcp        0      0 192.168.1.8:35417       ftp.estpak.ee:www       ESTABLISHED 10292/wget          
tcp        0      0 192.168.1.8:52729       ftp.mfa.kfki.hu:ftp     ESTABLISHED 10296/wget          
tcp        0      0 192.168.1.8:32795       ftp.estpak.ee:www       ESTABLISHED 2743/wget           
tcp        0      0 192.168.1.8:34561       mirror.selfnet.de:ftp   ESTABLISHED 7633/wget           
tcp        0      0 192.168.1.8:42365       cesium.di.uminho.pt:ftp ESTABLISHED 21276/wget          
tcp        0      0 192.168.1.8:56139       dion.freedback.com:6127 ESTABLISHED 31449/wget          
tcp        0      0 192.168.1.8:42428       sagres.c3sl.ufpr.br:www ESTABLISHED 16771/wget          
tcp        0      0 192.168.1.8:50152       dion.freedback.com:ftp  ESTABLISHED 31449/wget          
tcp        0      0 192.168.1.8:54224       mail.giantix-server:www ESTABLISHED 8881/wget           
tcp        0      0 192.168.1.8:55165       cesium.di.uminho.:10646 ESTABLISHED 25373/wget          
tcp        0      0 192.168.1.8:60646       gv-in-f109.1e100.:imaps TIME_WAIT   -                   
tcp        0      0 192.168.1.8:37136       ftp.estpak.ee:21046     ESTABLISHED 13081/wget          
tcp        0      0 192.168.1.8:35421       ftp.estpak.ee:www       ESTABLISHED 27695/wget          
tcp        1      0 192.168.1.8:34770       ni-in-f104.1e100.:https CLOSE_WAIT  7837/e-calendar-fac 
tcp        0      0 192.168.1.8:59253       dion.freedback.co:52366 ESTABLISHED 4028/wget           
tcp        0      0 192.168.1.8:52462       mail.giantix-serv:37164 ESTABLISHED 3942/wget           
tcp        0      0 192.168.1.8:37325       ftp.estpak.ee:ftp       ESTABLISHED 13081/wget          
tcp        0      0 192.168.1.8:35560       mirror.selfnet.de:ftp   ESTABLISHED 14615/wget          
tcp        0      0 192.168.1.8:42207       mirror.selfnet.de:20289 ESTABLISHED 7633/wget           
tcp        0      0 192.168.1.8:42885       dion.freedback.com:ftp  ESTABLISHED 4028/wget           
tcp        0      0 192.168.1.8:57700       cesium.di.uminho.pt:ftp ESTABLISHED 25373/wget          
tcp        0      0 192.168.1.8:48856       ftp.estpak.ee:www       ESTABLISHED 15574/wget          
tcp        0      0 192.168.1.8:35416       ftp.estpak.ee:www       ESTABLISHED 6229/wget           
tcp        0      0 192.168.1.8:33698       cesium.di.uminho.:49751 ESTABLISHED 21276/wget          
tcp        0      0 192.168.1.8:42353       mail.giantix-server:ftp ESTABLISHED 3942/wget           
tcp        0      0 192.168.1.8:57301       ftp.mfa.kfki.hu:41730   ESTABLISHED 10296/wget          
tcp        0      0 192.168.1.8:38278       mirror.selfnet.de:53560 ESTABLISHED 14615/wget          
tcp        0      0 192.168.1.8:32802       ftp.estpak.ee:www       ESTABLISHED 7863/wget           
tcp        1      0 192.168.1.8:34769       ni-in-f104.1e100.:https CLOSE_WAIT  7837/e-calendar-fac

Basically all of them (which show the process wget) are Arch mirrors. Pacman is not running (based on ps aux), neither any of my pacman replacers/helpers.

How do I track down what is causing this? The wget processes are run by root, but can't be killed even by sudo kill -9.

EDIT: Actually the process gets killed but it just starts up again immediately (different PID).

EDIT2: Managed to really kill the processes (and get my download speed back to 0) by killing multiple instances of pkgfile. Curioser and curioser, I can't be the only user of pkgfile.

EDIT3: Running sudo pkgfile -u maxes out my connection as well, but it completes (slowly) over the period of maybe 3-5 minutes, while the previous issue had multiple pkgfile instances and lasted for an hour at least (from when I noticed). Perhaps cron related?

Last edited by ngoonee (2010-04-13 23:40:43)

Daenyth · 2010-04-13 23:51:24

Odd, I've never seen that before. There's a cron.daily entry to run pkgfile -u, as noted in the install message, but it should never have multiple instances running.

Hmm, I suppose it would be good to add locking in any case.

ngoonee · 2010-04-14 04:53:20

Unfortunately, I don't know how to reproduce the error, so the only 'confirmation' that something works is a negative 'hasn't happened yet' confirmation.

In any case, I guess by locking you simply me 'touch /var/pkgfile.pid' and an if check whether the file exists before running?

Daenyth · 2010-04-14 05:16:34

Something like that

Also, if you edit the cronjob to run with -dd as well, and save the output somewhere, it could potentially be useful

Last edited by Daenyth (2010-04-14 05:20:00)

ngoonee · 2010-04-14 15:43:43

Just updated to pkgtools-18-1, assume this includes that commit?

I will save the output to a log file until it happens again. If it ever does.

Daenyth · 2010-04-14 15:44:50

Yup, among other things. Changelog here

Arch Linux

#1 2010-04-13 23:27:21

Many wget instances to mirrors on mirrorlist (called by pkgfile)

#2 2010-04-13 23:51:24

Re: Many wget instances to mirrors on mirrorlist (called by pkgfile)

#3 2010-04-14 04:53:20

Re: Many wget instances to mirrors on mirrorlist (called by pkgfile)

#4 2010-04-14 05:16:34

Re: Many wget instances to mirrors on mirrorlist (called by pkgfile)

#5 2010-04-14 15:43:43

Re: Many wget instances to mirrors on mirrorlist (called by pkgfile)

#6 2010-04-14 15:44:50

Re: Many wget instances to mirrors on mirrorlist (called by pkgfile)

Board footer