Virtual Cluster Performance

wsims · 2010-12-24 03:13:07

I'm attempting to setup a virtual cluster.
I've installed Arch in three VM's using the KVM wiki.
All three VM's are up and running on my LAN, but when accessing each VM via HTTP, one page will load fairly quickly, while the other two will load slowly, or timeout all together. Even when ssh'ing to each VM, things are "laggy".
All three connections are sharing one bridge (br0), each with their own tap interface (tap0, tap1, tap2).

Startup script for each VM (substituting tap1, tap2 for the 2nd and 3rd VM accordingly).

#!/bin/sh
ARGS="-hda win2k.img -boot c -net nic,vlan=0 -net tap,vlan=0,ifname=tap0,script=/etc/qemu-ifup -m 256 -localtime"
echo "Starting QEMU with..."
echo $ARGS
echo "...."
exec qemu $ARGS

Per the KVM wiki, I've saved this script in /etc/qemu-ifup

#!/bin/sh
echo "Executing /etc/qemu-ifup"
echo "Bringing up $1 for bridged mode..."
sudo /sbin/ifconfig $1 0.0.0.0 promisc up
echo "Adding $1 to br0..."
sudo /usr/sbin/brctl addif br0 $1
sleep 2

Notes:
-The host machine has a Quad Core 2.66Ghz processor with 3GB of RAM. The RAM does seem light, but each VM is only using 256MB each.
-At no point has the host pegged its processor or RAM
-All three VM's have LAN connectivity and access to the Internets
-ifconfig shows no packet collisions on any of the tun interfaces, br0, or eth0

ifconfig:

br0       Link encap:Ethernet  HWaddr 00:1A:92:0A:5D:4A  
          inet addr:192.168.1.145  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:63215855 errors:0 dropped:0 overruns:0 frame:0
          TX packets:90842468 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:23218456314 (22142.8 Mb)  TX bytes:121914827747 (116267.0 Mb)

eth0      Link encap:Ethernet  HWaddr 00:1A:92:0A:5D:4A  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:61986117 errors:0 dropped:0 overruns:0 frame:0
          TX packets:89457049 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:22970691309 (21906.5 Mb)  TX bytes:120961264355 (115357.6 Mb)
          Interrupt:21 Base address:0x8000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:706524 errors:0 dropped:0 overruns:0 frame:0
          TX packets:706524 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1575005790 (1502.0 Mb)  TX bytes:1575005790 (1502.0 Mb)

tap0      Link encap:Ethernet  HWaddr F2:90:73:25:02:68  
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:10825 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19804 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:944410 (922.2 Kb)  TX bytes:27208522 (25.9 Mb)

tap1      Link encap:Ethernet  HWaddr 7E:89:B4:F7:C9:11  
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:1361425 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1453316 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:1336475832 (1274.5 Mb)  TX bytes:1374656905 (1310.9 Mb)

tap2      Link encap:Ethernet  HWaddr 82:93:4C:FD:14:C2  
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:11589 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20474 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:1109099 (1.0 Mb)  TX bytes:27419303 (26.1 Mb)

Questions:
Would all three VM's sharing a single bridge (br0) be the "pinch point"?
Would installing a separate NIC for each VM help?
How does this all affect Net Neutrality?

wsims · 2010-12-24 03:30:36

Forgot to mention that each VM is sharing a single folder of content via NFS.

graysky · 2010-12-24 08:02:22

Your post intrigued me. While I can't answer your question I have a few of my own. What is the goal of your virtual cluster? I could be way off base here but asking is how we learn: why do you think that having a virtual cluster will offer better performance than host quad core machine?

wsims · 2010-12-24 14:12:22

The goal of the cluster is purely educational, but the idea was to spread out HTTP requests between the three VM's.
The next step would be to get DrQueue installed and configured. http://www.drqueue.org/cwebsite/

Eventually I'd like to do something like this guy did, but on a much smaller scale.
This is the story of Helmer. A linux cluster in a IKEA Helmer cabinet

Last edited by wsims (2010-12-24 14:13:58)

Arch Linux

#1 2010-12-24 03:13:07

Virtual Cluster Performance

#2 2010-12-24 03:30:36

Re: Virtual Cluster Performance

#3 2010-12-24 08:02:22

Re: Virtual Cluster Performance

#4 2010-12-24 14:12:22

Re: Virtual Cluster Performance

Board footer