KVM is Linux. Xen is Not.

Last year there was a lot of talk from Citrix, XenSource, and Oracle about the acceptance of certain Xen code and drivers into the Linux 3.0 kernel. I still hear it today. They’re implying that it means the Xen has been as tightly integrated into the Linux kernel as KVM. And I’m here to tell you that they are wrong.

Here’s what Oracle and Citrix have said on the matter:

“I’ve heard over the last few years, competitors use “There is no Xen support in Linux” as a tagline to create FUD with the Xen userbase and promote alternatives. Well, it’s all there people.”Wim Coekaert, Oracle

“During all the fuss of Citrix Synergy last week, an event of pretty earth-shattering importance occurred in the open source world: all key Xen code was accepted into the Linux mainline kernel.” Simon Crosby, Citrix

Wrong!

The Oracle and Citrix marketing on this issue are misleading to say the least. It implies that the recent acceptance of some Xen enablement code into the Linux kernel equates to integration, or that the Xen architecture doesn’t matter now that some of  their code is in Linux.

KVM is still the only Linux kernel-integrated hypervisor technology. End of sentence. End of story.

There is no special KVM enablement needed in the Linux kernel. KVM is integrated into the Linux kernel. KVM uses Linux for everything from device management to CPU scheduling, memory page sharing, better algorithms for making use of lots of cores and lots of memory, and tying into all the hardware virtualization technology AMD and Intel have been building into x86 for the past 5 years.

Xen is no more mainline or mainstream today in 2012 when it comes to Linux than it was in 2007 when KVM was accepted into the Linux kernel, or 2009 when Red Hat decided to move forward with KVM instead of Xen and integrated KVM into RHEL.

OK, then, what’s the Real Scoop?

Xen has still not been accepted (and probably never will be) into the Linux kernel, and cannot leverage directly Linux technologies such as transparent huge pages, CFS scheduler, paging, memory overcommitment with KSM, etc. for its hypervisor technology. You still need to install the Xen kernel on bare metal and build a special VM called a Dom0 to manage it and to provide device drivers. And you still have the suboptimal Xen architecture with Xen-Dom0-DomU. See the ugly diagrams below for more details.

The announcement of Xen and Linux 3.0 means two (and only two) things:

1. The paravirtualized drivers that Xen uses will be integrated into Linux starting with 3.0. This means that just like virtio (KVM/Libvirt) and pv-scsi (VMware), the pv-ops drivers used for accelerated disk and NIC will be available in any Linux 3.x+ guest without having to integrate drivers. This leaves Microsoft Hyper-V alone in requiring integrating drivers for guests.

2. A Linux 3.0 or higher guest can be used unmodified as a Dom0. Remember Dom0 is where the console operating system resides and is also a slave for certain IO from the guests. It is not Xen itself. Xen still resides on the bare metal and is still a separate project from Linux that has to duplicate effort to support hardware and new virtualization technologies.

In summary, it will be slightly easier to use future versions of Linux guests with current and future versions of Xen, and it will be slightly easier to use future versions of Linux as a Dom0 for current and future versions of Xen.

If you want to use a pre-3.0 Linux as a Dom0 it won’t work without modifications. If you want to use pre-3.0 Linux guests, you still need to hack in the drivers.

Not such a big deal after all. So does this change anything for KVM?

In a word, No.

One of KVM’s strengths is that it is integrated into the Linux kernel and therefore can utilize Linux features for things that hypervisors do besides just being a hypervisor.

Xen hypervisor is still a separate project and a completely separate code base, and features that KVM can use for virtualization and inherits from Linux (scheduling, paging, frequency scaling and hardware enablement, for example) will still need to be separately implemented in Xen. This is part of the design of Xen and is not going to change.

KVM Architecture

Elegant, isn’t it?

KVM Architecture

Xen Architecture

What a mess…

Xen Before Linux 3.0

Xen After Linux 3.0

What did they say? Word for word. I’m taking down names.

Here’s links to the various blogs Citrix and Oracle have put out:

http://blogs.citrix.com/2011/05/30/xen-celebrates-the-final-step-of-a-four-year-odyssey/

http://blogs.oracle.com/wim/entry/linux_mainline_contains_all_the

About these ads

25 thoughts on “KVM is Linux. Xen is Not.

  1. The sheer amount of FUD and ignorance in this post makes my eyes bleed.

    If one of my employees were to post something of this calibre about a competing solution I would be horribly embarassed.

  2. The amount of FUD and ignorance in this post makes my eyes bleed.

    If someone at my company was to post something of this calibre I would be horribly embarassed.

    • If there are any inaccuracies in the analysis, please let me know. As a service provider whose solutions are based on Xen, you are probably already aware of the architectural differences between the KVM and Xen platforms. As for FUD, I was addressing the FUD that Citrix and Oracle were circulating about KVM. Can it be that responding to FUD can be FUD to someone else?

  3. Actually – this reads like a political taken out of context blog entry, ignoring the real data., carefully selecting bits. If you actually read my blog you would have noticed that I was very careful to point out that the patches to have linux be a Dom0 kernel FOR Xen are in the kernel. So no where in my blog did I make any statement that Xen was in the kernel.

    Maybe next time you can leave a link to the original article in your blog as well so your readers can see the source and make up their minds instead.

    • I wrote this to address the confusion that most people had to the announcements from Oracle and Citrix: namely that Xen was now as integrated into Linux as KVM was and that there was no effective difference between Xen support in Linux and KVM support in Linux. It may not have been the intent, but it was definitely the impression that folks got and were asking questions about.

      I agree that readers should read all the relevant information. The link to the original blog was at the end of the posting. I have added the link to the quote at the top of the posting as well.

  4. Pingback: Open Source Virtualization – KVM | San Diego Linux – Linux and Virtualization Consultants

  5. (Posting here as well as on the follow-up)

    One big technical inaccuracy: “The paravirtualized drivers that Xen uses will be integrated into Linux starting with 3.0″, and “If you want to use pre-3.0 Linux guests, you still need to hack in the drivers.” Both of these are wrong. It looks like you’ve got the “consumer” and “provider” side mixed up. Drivers for guests (which you marked in red in your diagram) have been integrated into Linux for years; full 32-bit PV support in 2.6.24 and 64-bit 2.6.27, I believe. What’s made it into the kernel is the “provider” (or back-end, to use Xen terminology) in domain 0. (In KVM, I believe these are provided by qemu rather than by the kernel.)

    Another one I’m pretty sure is inaccurate: “KVM is in Linux”. OK, yes, the KVM kernel module does reside in Linux; but it can’t work on its own — it needs a user-space component in order to (as I understand it) do PCI emulation and provide the virtio back-ends. Now, there is a prototype user-space component in the Linux tree, but (as I understand it) RedHat actually uses qemu-kvm as the user-space component; and qemu-kvm is most decidedly *not* in Linux. So in your diagram, between the virtio front-end and the kernel, you should have a red slice for qemu.

    So “KVM is in Linux” is really just as inaccurate as “Xen is in Linux”. Linux now gives both virtualization platforms all the support that they need.

    (It should be clear I’m a Xen guy, so feel free to correct my understanding of how RedHat uses KVM.)

  6. Nothing has been said about performances here, and it isn’t a given, at least to me, that a smoother architecture with all the related plusses can be taken as an indicator of better performances. Can somebody with knowledge elaborate on this subject? Thanks.

    • I personally don’t see the problem with this article as I see great value for XEN not being built on top of Linux – mainly from a security perspective – and on the other side I see some advantage for KVM to be built on Linux. It simply depend on your needs. I personally believe we need more Hypervisors with evermore defined differences.

      Joannes, you’ll find a lot of benchmark on Phoronix. Seems like there’s not clear winner on that side (performance). Here’s one: http://www.phoronix.com/scan.php?page=article&item=ubuntu1210_xenkvm_preview

    • I personally don’t see the problem with this article as I see great value for XEN not being built on top of Linux – mainly from a security perspective – and on the other side I see some advantage for KVM to be built on Linux. It simply depends on your needs. I personally believe we need more Hypervisors with evermore defined differences.

      Joannes, you’ll find a lot of benchmark on Phoronix. Seems like there’s not clear winner on that side (performance). Here’s one: http://www.phoronix.com/scan.php?page=article&item=ubuntu1210_xenkvm_preview

      • That phoronix review is simply terrible. No HVM PV Drivers loaded for xen, but KVM gets virtio. No xen PV guest only HVM which makes xen much slower.

  7. Pingback: a guestOS process occupies VCPU at any given time?

  8. “Xen still resides on the bare metal and is still a separate project from Linux that has to duplicate effort to support hardware and new virtualization technologies.”

    And I’m so damn glad that is the way it is. I like Free software. Not free as in beer, and not free as in “we will always hate it because it wasn’t our idea and works on other OS, now that we copied every feature we can show how big ******* we are”.

  9. Pingback: Additional reading on open source virtualization | OSArchive

  10. Pingback: KVM/QEMU & Snapshots with libvirt – Final « An IT-related blog!

  11. Pingback: IT Efektivně | Open source virtualizace pro pitomce

  12. Hi,

    first of all many thanks for sharing the KVM/XEN diagram and for pointing out the changes with Kernel 3.0. There are various debates about the different hypervisors on the net, with everyone pointing out, why theirs being the best. Something I personally had to learn is stop being a fan and act completely rational to reach goals/milestones.
    We should be gladful for the competition ideed.

    Think of the following sceanario for a moment to see, what I mean:
    ————————————————————————————-

    - You are a big hoster and you have many customers from different business areas such scientific computing, banking, telco etc.
    - You have alot software licenses and stacks to support, which costs you

    So your customers and workloads have different needs, that can simply not be covered by “one fits all architecture”. As a serous hoster you can not afford to be uncertified or unsupported. As well there is some sort of vendor lock-in with the hypervisors and/or license benefits. Also it may be easier, when doing support calls.

    So maybe you will have different farms, like eg.

    x86 Virtualization Scope
    ===============================================================
    Hyper-V / Windows For SQL Server, SSAS, IIS, Biztalk, …
    Oracle VM / Oracle Linux For Oracle Database, Weblogic, Siebel, Hyperion, …
    VmWare ESX For VDI, Client Virtualization, CAD/CAM, other apps
    LXC / Solaris Zones For Web Servers / Sub-Partitioning VM’s

    There is many concepts as well for other platforms like e.g. IBM Power, Sparc or Itanium. They all have some more or less silmilar ideas. Stuff is just called diffently (eg. a Solaris Container is called WPAR at IBM or we call it Secure Resource Partitions at HP).

    I encountered many people still do not understand the concept of their hypervisor / virtualization technology they use.

    Example:

    If you think you’re smart get biggest Server known to mankind, put all different workloads on it and then wonder why it won’t fly automagically as you have enabled dynamic resource optimization, go back to square 1:

    - Putting 100 Windows VM’s all with Mcfee Full Scan Scheduled at 10 PM together with your 20 SAP ERP Linux VM’s is going to end in a hopeless battle for I/O and CPU

    - Putting Oracle Database on VmWare ESX is going to ruin you from license point of if you use CPU licensing, since Oracle does only support Hard Partitioning under their own hypervisor Oracle VM under x86 and as a hoster you often license by CPU as it is often the only option to not get sued when hosting web applications accessing a database.

    - Creating a VM for app just to logically model / isolate going to give much hypervisor overhead with increasing concurrency (yes, a hypervisor needs to schedule you cpu time manage your memory blocks ,etc). So judge well, between consolidation to an existing resource or creating a new container/zone or even a entire VM/Partition.

    - Create an isolation zone inside your farm, so you can take out problematic from the main farm VM’s for analysis and tuning.

    And most. but important:

    - Look at the resources and skills you have in house.
    - Take your time to form a clear vision.
    - Track / Review it regularly.

    Hope this helps to find a way to achieve your goals and to provide a good service.

    Regards

    Efstathios

  13. You drew Dom0 and the XEN Hypervisor as two separate blocks. Thats funny. But I understand, you had to to be able to add the words “what a mess”. You also really glossed over how nowadays you can directly allocate IO devices to the guests. Oh wait, you omited glossing over it, but you wrote this in February 2012, so your information was not too far outdated. You really do try hard though to make it sound difficult to make a guest into a DomU (Modify it… sound scary). I thoroughly enjoyed reading this, thank you.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s