I have had some more problems with quotas - on starting a vserver guest the quota seemed to exist, but I could still put files inside a vserver as much as the hd could fit.
With the help of Herbert, the vserver project lead, I was able to get quotas partly to work. We went through my configuration and finally he narrowed the problem down to the fact that the older vserver patches do not support quota property for reiserfs or xfs - only for ext3 and ext2. I had been using reiserfs exlusively. For a newer kernel, I had to get an experimental patch - patch-2.6.13-vs2.0.1-pre2.diff.
Now quotas are getting properly filled up and the attempt to add more files fails. But on deletion, space is not freed up. So now I am testing quotas with an ext3 partition and older patch, to find out if something is wrong with the experimental patch or the methods I am using.
The positive side of this quota problem has been that I succeeded in making dietlibc package and then making the vserver-tools package to have dietlibc support. This will improve vserver chroot's security.
EDIT: my experiment with ext3 was successful! I was able to free up space by deleting some files I downloaded before to fill up the quota. I tried this with the latest experimental patch for kernel 126.96.36.199, but it will probably work with older ones,too. EDIT: a quota test with kernel 188.8.131.52 was successful, too.
After a lot of testing, finally quotas seem to work right. The guide should not be followed to the letter when most of the programs in guest's /bin, /usr/bin dirs are as immutable, hard links. chxid -c <number> -R /wherever/vhost0/ will overwrite the xid of those immutable files and it will always be the latest vserver guest's xid. This will cause problems with older vservers because they cannot access files, like /sbin/init anymore and will fail to start.
I have been using just the other part. I made a dir prepre-start.d to /etc/vservers/<vservername>/scripts and put that one-liner script (/usr/sbin/vdlimit...) to a file. Only change I made was that I adde --xid <context id> to du part of that line (and I am using the previous modded version of du). That file is run on vserver start-up then and quotas work fine. I also added "vserver start <vsname> &>/dev/null" to rc.local for easily starting up the vserver guests.
See the infamous flower page for all the other options, like prepre-start.d - just remember to change the page style in Firefox
Finally on to the next levels! I got Apache to work as a reverse proxy and I am on my way finding proper mod_security rules. I found a site that seems to have collected together large rulesets from different areas. The areas presented there are for example: applications, bad user agents and comment spam prevention.
I am having troubles compiling vserver-tools. It is caused by the absence of libtool. I'd like to add the new PKGBUILD to the wiki at vserver site and also add the PKGBUILD for dietlibc.
Here is the clip form the last lines of compilation:
libtool: link: cannot find the library `/usr/lib/libstdc++.la' make: *** [src/vhashify] Error 1 make: Leaving directory `/home/sven/vserver/setup/util-vserver/src/util-vserver-0.30.208' make: *** [all-recursive] Error 1 make: Leaving directory `/home/sven/vserver/setup/util-vserver/src/util-vserver-0.30.208' make: *** [all] Error 2
How to fix this?
EDIT: Nevermind, the wiki got the answer I removed all .la - files from server and got it compiled nicely.