You are not logged in.
Hello guys, I hope I posted this at the right place. I'm having a strange problem and I don't know what is causing it.
The story
I needed to upload a 900MB video so I tried to split it up first using the 'split' command in my home directory. After half an hour of execution my whole system started to lag... I then terminated the process and found out that I had probably run it with wrong arguments (stupid me). As a result I had millions of tiny files in my home directory (ranging form mv_xaa to mv_zzzbscltunz or similar...). I had a really bad time removing them. Lots of applications didn't work anymore because you couldn't list files in my home directory, 'rm mv_*' didn't work because it said that the argument list was too big. I ended up running commands like
find . -name 'mv_*' -exec rm {} \;
while true; do rm `echo m_* | head -c 1000000`; echo deleted 1 block; done # note: not totally safe
find . -name 'mv_*' -delete
Using filezilla GUI to remove the filesfor one whole night and then the files were gone. (By the way, which of those command do you think performs faster?)
The problem
Then, relieved that I finally got rid of those files, I saw that I still experience a long lag. The problem is that after I boot up and try to ls or use pcmanfm in my home directory, it takes a very long time. Other directories work fast as usual. If I 'ls' again in my home directory it works fast, probably due to some form of caching.
$cd # SOMEDIR #
$time ls
# OUTPUT #
real 0m0.225s
user 0m0.000s
sys 0m0.003s
$cd ~
$time ls
# OUTPUT (~20 files) #
real 1m46.271s
user 0m0.003s
sys 0m0.547sMy guess is that something on the filesystem is wrong. I use ext4. I though that I maybe have to defragment the home partition. Do you think that can help? How do I do it?
What do you think is causing the big ls lag?
Thanks for reading the long post.
Last edited by aero (2012-09-29 19:27:34)
Offline
Any chance you are getting I/O errors while talking to the disk? Check your logs.
Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
The shortest way to ruin a country is to give power to demagogues.— Dionysius of Halicarnassus
---
How to Ask Questions the Smart Way
Offline
I guess there are no disk I/O errors, because I didn't find anything in /var/log/messages.log. Is there another place I can look for errors?
Offline
try /var/log/everything.log
Offline
I suspect that the problem is because EXT filesystems do not ever reuse directory inodes once they are allocated. To check to see the space being used try ls -l .. while in the problem directory. If the size of the directory seems massive, your only solution is to create a new directory and move all the files to it and then delete the old directory. Obviously if this is your home directory you will need to take appropriate steps to make sure there are no login shells accessing the directory at the time.
Offline
I suspect that the problem is because EXT filesystems do not ever reuse directory inodes once they are allocated. To check to see the space being used try ls -l .. while in the problem directory. If the size of the directory seems massive, your only solution is to create a new directory and move all the files to it and then delete the old directory. Obviously if this is your home directory you will need to take appropriate steps to make sure there are no login shells accessing the directory at the time.
I think that might be exactly the problem. ls -l .. in ~ tells me
drwx-----x 92 myname users 63397888 Sep 27 12:15 mynameI will try to recreate the directory later today at home, where I can make backups first. Will then post the results here.
Offline
It worked!
time ls
# OUTPUT #
real 0m0.260s
user 0m0.003s
sys 0m0.000sWhat I did:
1. Backup all my files of my home directory in a separate hard drive
2. init 1
3. cd /home
4. mkdir foo
5. chown aero:users foo
6. chmod go-r foo
7. chmod g-x foo
8. mv aero/* foo
9. mv aero/.* foo # don't do that, it's wrong! use something like 'find . -mindepth 1 -maxdepth 1 -exec mv -t.. -- {} +'
10. rmdir aero
11. mv foo aero
12. reboot
Size is much better now:
$ ls -l /home
total 20
drwx-----x 91 aero users 4096 Sep 29 15:29 aero
drwx------ 2 root root 16384 Jun 29 2011 lost+foundThank you skunktrader!
Last edited by aero (2012-09-29 19:31:29)
Offline