kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

gernonimo · 2009-04-10 16:11:07

whats the difference between the two? Which one should be preferred?
thanks!

Mr.Elendig · 2009-04-10 18:34:07

I'm happy with qemu from extra, using qemu-kvm to run the vm.

bender02 · 2009-04-11 06:34:43

i think the two are being merged about now (with qemu-0.10.2). i'm also happily using qemu.

_lunix_ · 2009-04-13 03:25:27

yes they have been merged.

qemu-kvm

is now gone
and replaced with

qemu -enable-kvm

The other trouble I ran into is I have a 64bit cpu and was running 64bit guests with

qemu-kvm -hda foo.img

That now fails and I have to do

qemu-system-x86_64 -enable-kvm -hda foo.img

_lunix_ · 2009-04-13 04:43:50

ok. I was having trouble with 100% cpu with using the latest qemu ( with qemu -enable-kvm )
Just removed it and built KVM using ABS 'extra/x86_64/kvm/'
Now the CPU usage is fine but I can no longer use VDE.
*sigh*

Last edited by _lunix_ (2009-04-13 04:57:33)

stoffell · 2009-04-13 07:27:57

I don't think they have merged completely yet, is that possible?

I was trying to boot a Debian Lenny on a "Virtio block device", I just followed the howto on the linux-kvm page but the option "boot=on" does not exist in qemu 0.10.2-3. It does exist when I build the kvm 84-1 from AUR.

Or am I missing something? Can we update the wiki page on KVM/Qemu to clarify the differences?

cheers
Kristof

Kenni · 2009-04-13 19:54:47

I also have a lot of problems with QEMU & KVM & kernel 2.6.29 at the moment.

I'll try to sum up my findings:

1. The latest QEMU (qemu-0.10.2-3) with partial KVM support (should be fully merged with KVM hopefully in 3-6 months) doesn't seem to work. It will only boot with an unaccelerated "Pentium II" CPU, and it fails to boot with the SMP-option with more than one CPU. I opened a bugreport with this here:
http://bugs.archlinux.org/task/14166

2. As a temporary solution, a new KVM-package was created by tpowa and put into the extra repository. This KVM-package contains the userspace qemu-kvm build from the KVM-project (this was part of the QEMU package in Arch Linux up till qemu-0.10.1-1.

3. Both the old qemu-0.10.1-1 and the new kvm-84-2 package, which both supply the same qemu-kvm (KVM-84) userspace executable, works perfectly on a 2.6.28.x based kernel. However, I can't get any of them to boot my QEMU/KVM guests without the guest freezing on the new 2.6.29.1 kernel...

4. The latest qemu with partial KVM support requires kernel 2.6.29, so trying it out with 2.6.28 is not an option.

So right now I'm stuck at kernel 2.6.28.x + kvm-84-2 or qemu-0.10.1-1.

I suppose that [3] is an upstream kernel bug and I'm crossing my fingers that it will get solved in 2.6.29.2 - Is anyone else also having this issue??

stoffell · 2009-04-14 17:16:46

Kenni wrote:

3. Both the old qemu-0.10.1-1 and the new kvm-84-2 package, which both supply the same qemu-kvm (KVM-84) userspace executable, works perfectly on a 2.6.28.x based kernel. However, I can't get any of them to boot my QEMU/KVM guests without the guest freezing on the new 2.6.29.1 kernel...

I'm running 2.6.29 and using the kvm-84-2 package without a problem. I did however have the same issues you mentioned with qemu 0.10.2-3.

At the moment I'm using the latest kernel and kvm-84-2. That seems to work here..

Cheers,
Kristof.

Kenni · 2009-04-14 18:11:59

stoffell wrote:

I'm running 2.6.29 and using the kvm-84-2 package without a problem.

Now, that's interesting! What arguments do you give to qemu-kvm? And you're running the latest kernel package 2.6.29.1-3?

_lunix_ · 2009-04-14 23:03:07

stoffell wrote:

Kenni wrote:
3. Both the old qemu-0.10.1-1 and the new kvm-84-2 package, which both supply the same qemu-kvm (KVM-84) userspace executable, works perfectly on a 2.6.28.x based kernel. However, I can't get any of them to boot my QEMU/KVM guests without the guest freezing on the new 2.6.29.1 kernel...
I'm running 2.6.29 and using the kvm-84-2 package without a problem. I did however have the same issues you mentioned with qemu 0.10.2-3.
At the moment I'm using the latest kernel and kvm-84-2. That seems to work here..

Cheers,
Kristof.

It works with this combination for me too as long as I don't want to use VDE2.
If I use bridge networking its fine.

Kenni · 2009-04-14 23:25:45

_lunix_ wrote:

It works with this combination for me too as long as I don't want to use VDE2.
If I use bridge networking its fine.

Ok, can you please post your qemu-kvm command? I would like to try to track down if it's a specific argument causing the problems on my system.

Thank you...

_lunix_ · 2009-04-14 23:45:26

qemu-kvm -monitor unix:/home/vmuser/run/pclient1.sock,server,nowait -name puppetmaster -smp 1 -m 512 -vnc :1 -daemonize -localtime -usb -usbdevice tablet -net nic,vlan=0,macaddr=00:0A:2E:15:20:10 -net tap,vlan=0,ifname=tun0,script=/etc/qemu-ifup -pidfile /home/vmuser/run/puppetmaster.pid -boot c -drive index=0,media=disk,if=ide,file=/home/vmuser/vdisks/puppetmaster/disk1.img -drive index=1,media=disk,if=ide,file=/home/vmuser/vdisks/puppetmaster/swap.img

I then use

socat - unix-connect:run/pclient1.sock

to connect to the kvm console. ( normally done with Alt-2 when not running in daemon mode )

iBertus · 2009-04-15 00:06:07

I just noticed that the new qemu will not work when I set -m higher than 2047 but it works with qemu-kvm via tpowa's package.

stoffell · 2009-04-15 20:04:12

Kenni wrote:

stoffell wrote:
I'm running 2.6.29 and using the kvm-84-2 package without a problem.
Now, that's interesting! What arguments do you give to qemu-kvm? And you're running the latest kernel package 2.6.29.1-3?

# pacman -Ss kernel26
core/kernel26 2.6.29.1-3 (base)
$ pacman -Ss kvm
extra/kvm 84-2

And my kvm startup is: qemu-kvm -k en-us -hda winxp.qcow2 -m 512

I didn't try bridging yet, so it's a pretty simple kvm line for my test machine :-) but it works..

Oh, and my linux guest is started as: qemu-kvm -boot c -drive file=debianlenny.img,if=virtio,boot=on -m 256

cheers,
Kristof

Last edited by stoffell (2009-04-15 20:06:54)

Kenni · 2009-04-15 21:34:38

stoffell wrote:

# pacman -Ss kernel26
core/kernel26 2.6.29.1-3 (base)
$ pacman -Ss kvm
extra/kvm 84-2

pacman -Ss will just give you the latest version available in the repositories, not the installed version. pacman -Q will tell you the installed version.

Thank you for both your answers, I've tested some different combinations of arguments to a Windows XP guest on my machine, but it still freezes under boot with the 2.6.29.1-3 kernel, while it boots without any issues on the 2.6.28 kernel

I'll have to look deeper into the problem when I get some more time..

stoffell · 2009-04-16 05:42:11

Kenni wrote:

pacman -Ss will just give you the latest version available in the repositories, not the installed version. pacman -Q will tell you the installed version.
Thank you for both your answers, I've tested some different combinations of arguments to a Windows XP guest on my machine, but it still freezes under boot with the 2.6.29.1-3 kernel, while it boots without any issues on the 2.6.28 kernel
I'll have to look deeper into the problem when I get some more time..

Oh yeah, sorry.. Pacman -Qs gives the same results, system is fully updated. How about trying a fresh install of an XP guest to see if that makes it through the setup? (and using as much default parameters as possible)

cheers
Kristof

Kenni · 2009-04-16 09:06:56

stoffell wrote:

How about trying a fresh install of an XP guest to see if that makes it through the setup? (and using as much default parameters as possible)

Yeah, but the guest works perfectly on a kernel below 2.6.29, so I think I'll stick with a .28 kernel for now and test it again when newer kernel versions as well as KVM and upstream QEMU versions gets released. I'm not in need of any new features or bugfixes from the current 2.6.29 kernel, so it's not a problem at the moment.

Kenni · 2009-05-03 12:56:05

Kenni wrote:

Yeah, but the guest works perfectly on a kernel below 2.6.29, so I think I'll stick with a .28 kernel for now and test it again when newer kernel versions as well as KVM and upstream QEMU versions gets released. I'm not in need of any new features or bugfixes from the current 2.6.29 kernel, so it's not a problem at the moment.

I finally got the issue solved with help from a KVM developer...there is a bug in qemu, which detects the host CPU as having 64bit capabilities even though the host kernel is 32 bit...passing the parameter '-cpu qemu32' (or '-cpu qemu64,-nx') fixes the issue
http://thread.gmane.org/gmane.comp.emul … ocus=31770

EDIT: A patchset has now been released - I suppose these will be part of KVM-86, which will make the '-cpu qemu32' parameter unnecessary:
http://thread.gmane.org/gmane.comp.emul … evel/31775

Last edited by Kenni (2009-05-03 16:27:49)

slinkygn · 2009-05-03 17:14:52

Just moved to kernel 2.6.29.2 (there were in-kernel patches to KVM) and qemu 10.3. Processor now (properly) reports as a QEMU processor in guests. No Pentium II. SMP still does not work. Did not test anything outside of the default networking -- KVM web page shows virtualized networking as "in progress," so I'd imagine some of the issues might be known.

And has anyone noticed that we seem to be the only distro having these problems? Nobody seems to report anything like this upstream to qemu or KVM, and no other distro-specific forums seem to have much about it either...

Last edited by slinkygn (2009-05-03 17:17:34)

Kenni · 2009-05-03 19:21:35

slinkygn wrote:

Just moved to kernel 2.6.29.2 (there were in-kernel patches to KVM) and qemu 10.3. Processor now (properly) reports as a QEMU processor in guests. No Pentium II. SMP still does not work.

The QEMU CPU vs. Pentium II CPU report from the guest can't be used to determine if KVM is enabled, after I got my issue solved, I have a Windows XP reporting Pentium II, with KVM enabled.

slinkygn wrote:

And has anyone noticed that we seem to be the only distro having these problems? Nobody seems to report anything like this upstream to qemu or KVM, and no other distro-specific forums seem to have much about it either...

As discussed several times in this thread (it's actually the subject of the thread), there're two way to run KVM in Arch Linux:
1. Upstream QEMU, which just recently got partial KVM support (qemu package in Arch Linux).
2. Upstream KVM userspace, which is a fork of QEMU, which has "full" KVM functionality (kvm package in Arch Linux).

It sounds like you are running the first one, which has lots of missing functionality and bugs...

Last edited by Kenni (2009-05-03 19:25:43)

slinkygn · 2009-05-03 20:07:36

Kenni wrote:

The QEMU CPU vs. Pentium II CPU report from the guest can't be used to determine if KVM is enabled, after I got my issue solved, I have a Windows XP reporting Pentium II, with KVM enabled.

I did not say I used it to determine if KVM was enabled. I used info kvm at the qemu console to determine if it was enabled, which it was.

slinkygn wrote:
And has anyone noticed that we seem to be the only distro having these problems? Nobody seems to report anything like this upstream to qemu or KVM, and no other distro-specific forums seem to have much about it either...
As discussed several times in this thread (it's actually the subject of the thread), there're two way to run KVM in Arch Linux:
1. Upstream QEMU, which just recently got partial KVM support (qemu package in Arch Linux).
2. Upstream KVM userspace, which is a fork of QEMU, which has "full" KVM functionality (kvm package in Arch Linux).
It sounds like you are running the first one, which has lots of missing functionality and bugs...

I'm sorry, I think you may have missed my point. The first one has a lot of missing functionality and bugs *for us*. No one outside of the Arch community is reporting these problems. Upstream, these bug reports don't even exist.

slinkygn · 2009-05-03 20:38:06

Kenni wrote:

As discussed several times in this thread (it's actually the subject of the thread), there're two way to run KVM in Arch Linux:
1. Upstream QEMU, which just recently got partial KVM support (qemu package in Arch Linux).
2. Upstream KVM userspace, which is a fork of QEMU, which has "full" KVM functionality (kvm package in Arch Linux).

Though while we're on tangents, I'm pretty confident that your description is anaccurate. I wouldn't bother with the details, except that maybe explaining the root of the issue better would also help us all come up with a solution. Hopefully Tobias can chime in and explain better if we're still misunderstanding.

Your 1) and 2) incorrectly separate QEMU and KVM. KVM has *always* required QEMU. KVM is and has always been solely the kernel-mode component, hence the name. It used to require a patched version of QEMU, which was submitted a while ago to Fabrice Bellard for inclusion upstream. The decision was made to hold off on inclusion until it was "stable." KVM finally got inclusion in the kernel, as it met stability requirements, and then the patch was included in QEMU mainline. KVM is not and never was a "fork" of QEMU.

Of course, even saying there are "two ways" to run KVM in Arch is fundamentally misled -- I could simply download the tarballs and run make on it myself, and compile the binaries from scratch. Which is what the package manager does for you; you're just picking two different packages as your "two ways." So, the correct way to state what you were trying to state is that there are two primary (and a number more in AUR) packages provided in the repository:

1) The first is the package labeled "qemu-<version>," which provides the latest build of qemu (10.0.3 as of now).
2) The second is the package labeled "kvm-<version>," which provides (as of kvm-85) a modified version of qemu 10.0.0 that has KVM support on by default instead of off. There may be other changes. No version of KVM is included with "kvm-<version>;" it still uses the KVM module from the kernel. Unarchive the package and look for yourself. And while you're at it, you can run qemu-kvm -h and check out the version number.

So the only visible difference is that you're running an older version of qemu, your binary name has been changed, and you don't have to type -enable-kvm. It is very possible (even likely; I would bet on it) that Tobias also included some of the post-10.0 patches in with that, but we don't know which offhand.

So, yes, the patched version of 10.0.0 in the repo works better (supports >=2gb, runs SMP) than 10.0.3. Now, as the >2gb change is not a bug, but a documented change, I'd tend to think that the QEMU devs had a reason for limiting that. I also would rather not run anything less than 10.0.3 considering that the last changelog noted a fix for a qcow2 corruption issue. But note that no one but Arch users are complaining about SMP not working. I want to know why that is. I suppose it's possible that no one else has noticed, but it seems pretty unlikely, doesn't it? It may, instead, be a build issue. I would just think that it would be useful to find out.

Kenni · 2009-05-03 20:44:37

slinkygn wrote:

I'm sorry, I think you may have missed my point. The first one has a lot of missing functionality and bugs *for us*. No one outside of the Arch community is reporting these problems. Upstream, these bug reports don't even exist.

Sorry, I get your point now. I've actually also been thinking about why nobody in the QEMU forums have posted issues about it, but I've convinced myself that no KVM users are using upstream QEMU, as all distributions are using KVM userspace and not QEMU upstream.

I think that the reason why we don't see the bug reports upstream, is because they are well known by the developers. Until all of the KVM userspace code is moved to QEMU upstream, it doesn't really make sense to track bugs, IMHO.

I suppose that people only use the KVM functionality in QEMU upstream if they're interested in testing it out. The only reason why Arch Linux users are complaining about these problems, is because the Arch Linux QEMU package recently changed from using latest KVM userspace to using upstream QEMU, which perhaps was a bit hasty decision, as QEMU upstream wasn't ready yet.

Kenni · 2009-05-03 21:03:26

slinkygn, I didn't refresh the page, so I didn't see your reply until afterwards.

Perhaps we don't agree on the wording, but essentially we mean the same...

However, I have a hard time believing that it should be a build problem causing the problems. If you have a look at the PKGBUILD, it seems very clean:
http://repos.archlinux.org/viewvc.cgi/q … iew=markup

A few comments on your feedback...:

Your 1) and 2) incorrectly separate QEMU and KVM. KVM has *always* required QEMU. KVM is and has always been solely the kernel-mode component, hence the name. It used to require a patched version of QEMU, which was submitted a while ago to Fabrice Bellard for inclusion upstream. The decision was made to hold off on inclusion until it was "stable." KVM finally got inclusion in the kernel, as it met stability requirements, and then the patch was included in QEMU mainline. KVM is not and never was a "fork" of QEMU.

I agree overall, but I would still call it a fork. The code QEMU was copied, modified and put into a separate project. Afterwards efforts were and are still made to merge the code together in a two-way manner, meaning that the KVM userspace today has reached v10.0.0 as well as QEMU upstream has gained initial KVM support. But it is not a patched QEMU v10.0.0, it's KVM userspace which has been patched up to have the same functionality as QEMU v10.0.0 does (in addition to all the KVM-related stuff not yet moved to QEMU upstream). Please correct me if I'm wrong.

So the only visible difference is that you're running an older version of qemu, your binary name has been changed, and you don't have to type -enable-kvm. It is very possible (even likely; I would bet on it) that Tobias also included some of the post-10.0 patches in with that, but we don't know which offhand.

No patches, have a look at the PKGBUILD, it fetches the source files directly from sourceforge:
http://repos.archlinux.org/viewvc.cgi/k … iew=markup

Last edited by Kenni (2009-05-03 22:13:25)

slinkygn · 2009-05-04 02:06:52

Yeah, my bad. The patches to qemu would come from upstream KVM, like they did before. Avi Kivity, not Tobias. My mistake there.

And yeah, I think some of those distinctions are us using different semantics. I'd call the qemu implementation that Avi maintains a fork -- but that is different to me than, as you worded it, kvm being a fork of qemu. When saying "kvm" and referring to a chunk of code, not what I'll call for this conversation's sake "the kvm project," has always referred to the kvm module -- the kernel code, again hence the name. Back when the userspace code was just a set of patches to qemu, this confusion was rarer. (Even after the kvm project started maintaining their own version of the code, they also maintained the qemu patch set to apply to the qemu project -- their version was just qemu, pre-patched. Avi discusses that philosophy of modifying the original code as little as possible in his most recent blog a few months ago.) The kvm userspace code is -- again, by Avi's own description -- qemu and libkvm. (http://www.linux-kvm.org/page/Code - "The kvm userspace code (libkvm and qemu) is available through a git tree.") kvm-the-project does maintain a prepatched qemu, but that doesn't make it kvm's code -- particularly when kvm's intent all along was to have their patches accepted by qemu, which they now have. So while you can say that kvm-the-project has within it a build of qemu -- and if you wanted, you could even call it a fork, though the maintainer declines to use the term -- qemu is not kvm.

I'd also add that though the gray area there exists, from a future-discussion perspective it does make things a lot clearer if we refer to kvm the module, the "pure" build of qemu, and kvm's qemu build (or qemu-kvm) as separate.

And by extension, no, kvm-userspace has not reached 10.0.0. There's a reason that the kvm userspace code just has build numbers. Its version number is 85. It would make a lot of sense to say that the kvm-maintained patches were modified and applied to qemu upstream to make qemu 10.0.0, but the post-10 stability patches for kvm were applied independently to qemu and qemu-kvm -- qemu-kvm basically getting just kvm-related stability patches, which are submitted upstream to qemu (and thus qemu-kvm moving up in build number), and qemu using the qemu-kvm patches as well as patches relating to other parts of the system.

Getting back to the meat of the question, it is important to note that the kvm we're getting from the arch package is NOT the kvm that you would get from building the download file. You called it clean, and you're absolutely right -- it's *too* clean. Download the KVM package from sourceforge, build, and do a make -n install; you get:

make -C kernel DESTDIR="" install
make -C libkvm DESTDIR="" install
make -C qemu DESTDIR="" install

Which makes sense if you read their docs -- the KVM build, as per KVM, come as a (independent of the kernel) set of modules and a QEMU build to go with those modules, just like it was before it was accepted by linux-kernel. (You could probably have noticed that kernel modules were being build when you run make, but there was so much stuff in there that I thought it easier to cut/paste the install phase for clarity.) The Arch PKGBUILD, however, just compiles qemu-kvm and its necessary libraries:

for dir in libkvm user qemu extboot; do
cd ${srcdir}/kvm-${pkgver}/${dir}
make || return 1
done

and then hand-installs those components without the kernel modules meant to go with it. Unless the package builder knows something the KVM developer doesn't, I wouldn't go around using that qemu build with whatever KVM module version from whatever different kernel version you may have willy-nilly.

If I had the hankering to run the KVM maintained userspace stuff, I'd do what they said and run their kernel stuff too. I think a proper KVM build should do that. But that's not my call, since I'm not the extra maintainer. As it stands, for me I'd rather use the in-kernel version, which I think will get the kernel-side patches that the KVM project's kernel code gets anyway since there is so much attention on the kernel, but also has the benefit of the qemu userspace code getting bug fixes from the qemu devs themselves. It sounded to me like that was the direction kvm itself was moving. But to each their own, of course. I'd just say better to stick all the way with one or all the way with the other method, and not just start making hybrids without a much deeper knowledge of the code than I think any of us have.

Last edited by slinkygn (2009-05-04 02:11:30)

Arch Linux

#1 2009-04-10 16:11:07

kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#2 2009-04-10 18:34:07

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#3 2009-04-11 06:34:43

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#4 2009-04-13 03:25:27

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#5 2009-04-13 04:43:50

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#6 2009-04-13 07:27:57

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#7 2009-04-13 19:54:47

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#8 2009-04-14 17:16:46

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#9 2009-04-14 18:11:59

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#10 2009-04-14 23:03:07

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#11 2009-04-14 23:25:45

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#12 2009-04-14 23:45:26

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#13 2009-04-15 00:06:07

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#14 2009-04-15 20:04:12

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#15 2009-04-15 21:34:38

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#16 2009-04-16 05:42:11

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#17 2009-04-16 09:06:56

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#18 2009-05-03 12:56:05

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#19 2009-05-03 17:14:52

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#20 2009-05-03 19:21:35

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#21 2009-05-03 20:07:36

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#22 2009-05-03 20:38:06

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#23 2009-05-03 20:44:37

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#24 2009-05-03 21:03:26

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

#25 2009-05-04 02:06:52

Re: kvm: qemu with option "-enable-kvm" or kvm-84 from AUR?

Board footer