From Fedora Project Wiki
(Note on Dom0 not being ready)
m (→‎Documentation: -- www.xen.org became www.xenproject.org. Fixing this)
 
(89 intermediate revisions by 11 users not shown)
Line 1: Line 1:
= Feature Name =
= Xen Pvops Dom0 =


== Summary ==
[[Features/XenPvops | pvops-based]] {{package|kernel}} to serve as Dom0 for a Xen-based system. Currently Fedora releases greater than 8 contain the Xen hypervisor and tools, but not a Xen Dom0-capable kernel.
== Owner ==


* Name: W. Michael Petullo
* Email: mike@flyn.org
* People Involved: Michael Young, Eduardo Habkost, Mark McLoughlin, Stephen Tweedie, Chris Wright, Juan Quintela, Markus Armbruster, Glauber Costa, Daniel Berrange et al.


== Summary ==
== Current status ==
 
* Targeted release: [[Releases/16 | Fedora 16 ]]
* Last updated: 2011-09-15
* Percentage of completion: 90%


Dom0 support for [[Features/XenPvops|  pvops-based kernel-xen]] .
In order to implement this feature, it is necessary to get full Xen Dom0 support upstream and in Fedora. The core Dom0 support has been accepted upstream and was released in Linux 2.6.37. Linux 2.6.39 brought support for the Xen network backend driver. Linux 3.0 brought the block backend driver. Linux 3.1 will bring the pci backend driver (for pci passthru).


== Owner ==
Open bugs:


* Name: EduardoHabkost, MarkMcLoughlin
* power management, [https://bugzilla.redhat.com/show_bug.cgi?id=692179 Bugzilla #692179]
* bridging in NetworkManager, [https://bugzilla.redhat.com/show_bug.cgi?id=199246 Bugzilla #199246]
* GRUB 2 has extraneous menu entries after installing Xen, [https://bugzilla.redhat.com/show_bug.cgi?id=739134 Bugzilla #739134]


Others involved: Stephen Tweedie, Chris Wright, Juan Quintela, Markus Armbruster, Glauber Costa, DanielBerrange, ...
== Detailed Description ==


== Current status ==
Xen is a hypervisor-based type-1 virtualization solution. The Xen hypervisor requires a standard operating system that runs in a privileged domain, Dom0. The Dom0 operating system provides driver and guest management support to Xen and other guest operating systems running in the DomU domain. Fedora 8 provided a kernel that could serve as a Dom0 guest. However, newer Fedora releases dropped support for Dom0 (although they can run in DomU). The Xen project is presently pushing the features required for a Dom0 Linux kernel upstream. Once these features are available in the mainline kernel, it follows that Fedora could support Xen Dom0.
* Targeted release: Unknown


Work on getting Dom0 support in the upstream kernel has pretty much stalled for the last number of months so this feature is postponed until work upstream restarts.
== Benefit to Fedora ==


[[Releases/10|  Fedora 10]] will ship with Xen i686 and x86_64 DomU support enabled in the stock kernel - i.e. there is no more kernel-xen.
Fedora will benefit from a Xen Dom0-capable kernel. Xen has proven to be a competitive virtualization solution in real-world installations. Xen is different enough from KVM that the two technologies could complement each other within the Fedora Project. There are ongoing benchmarks that intend to define the relative performance of the two technologies.


* Last updated: 2008-07-30
== Scope ==
* Percentage of completion: 30%


Tracker bug for [[Releases/10|  Fedora 10]] Xen kernel work : [https://bugzilla.redhat.com/show_bug.cgi?id=F10_XenPvOps]
* Implement Dom0 support in upstream kernel (Complete)
* Implement Dom0 backend drivers support in upstream kernel; fix remaining issues (Complete)
* Modify 'grubby' to support GRUB syntax required by Xen (Complete)
* Make 'new-kernel-pkg' aware of Xen using the HYPERVISOR option in /etc/sysconfig/kernel (Complete)
* Ensure Fedora's virtualization tools support Xen
* Possibly implement support for bridging in NetworkManager
* Possibly implement support in Anaconda


=== i386 Dom0 ===
== How To Test ==


* Not yet in rawhide kernel-xen
* Install a Fedora Dom0 kernel and {{package|xen}} (kernel-PAE is required for i686)
* Boots, though bare metal is now non-functional until hacks are cleaned up
* Reboot and confirm the Dom0 kernel can boot on bare metal (no Xen) by selecting the appropriate GRUB menu option
* DMA (and thus real hardware drivers) working (if it doesn't straddle page boundaries)
* Reboot and confirm grub.conf boots the new kernel using Xen by selecting the appropriate GRUB menu option
* Core needs major cleanup
* Observe presence of Xen via /sys/hypervisor/
* Status at  http://lists.xensource.com/archives/html/xen-devel/2008-02/msg00047.html
* Run 'xm list' and observe Domain-0
* http://lists.xensource.com/archives/html/xen-devel/2008-02/msg00242.html
* Use 'virt-install -l http://download.fedoraproject.org/pub/fedora/linux/releases/15/Fedora/x86_64 --ram 1024 --disk /var/lib/libvirt/images/f15.img --name F15' to deploy a Fedora paravirtualized guest
* No backend drivers yet
* Use 'virt-install' to deploy a Fedora fully virtualized guest
* Conflicts with x86_64 DomU patches
* Use 'virsh' to save & restore guests
* Balloon down memory of a guest with 'virsh'


=== x86_64 Dom0 ===
== User Experience ==


* No active work yet....
* Stock Fedora kernel supports both Xen dom0 and bare metal
* All features and hardware supported on bare metal also work on Xen Dom0
* Can enable and disable the use of the Xen hypervisor via an /etc/sysconfig/kernel setting
* Can manage Xen guests using Fedora virtualization tools


== Detailed Description ==
== Dependencies ==


See [[Features/XenPvops]]  for a rationale. This feature just continues to paravirt_ops Xen effort to bring it back to feature parity with Fedora 8 Xen.
* {{package|kernel}}
* {{package|grubby}}
* Fedora virtualization tools
* (possibly) {{package|anaconda}}
* (possibly) {{package|NetworkManager}}


== Scope ==
== Contingency Plan ==


Work items:
There is presently some uncertainty as to when the backend driver support that the Dom0 guest provides DomU guests will be upstream. This may make the upstream kernel that is chosen by Fedora 16 or it may not. Regardless, we will have to touch a lot of subsystems, so it makes sense to start early.


* Clean up and finish Dom0 boot patches (StephenTweedie, JuanQuintela, MarkMcLoughlin)
Until all of the requisite Xen components are upstream, we can continue to develop this feature using third party kernels. These third party kernels will be configured to be as close as possible to the forthcoming upstream kernel. In addition, Linux 2.6.39 does contain many of the backend drivers. It may be used with configurations that do not require the block backend driver.
** Backend drivers blkback/blktap and netback requiring: (StephenTweedie, JuanQuintela, MarkMcLoughlin)
** Some ballooning bits
** Foreign pages (i.e. PAGE_FOREIGN and VM_FOREIGN)
** zap_pte()
** Grant table ops
** Save / restore
** Memory ballooning
* x86_64: (DONE)
** Latest x86.git has useful patches on paravirt_ops pagetable handling
** ia32 emulation support
** SMP support
** Debug other problems that may appear
** Cleanup of code for inclusion upstream
** Some code need to be changed to be enabled at runtime if running under xen, instead of being enabled at compile time (pvops infra-structure may need to be extended to make it possible, on some cases)


Merged kernel: (DONE)
If all of the Xen backend drivers don't make Fedora 16, then little is lost. We can continue to test the Dom0 kernel (with a subset of its eventual features) and work on the supporting infrastructure. It would be useful for our effort to have a working Dom0, grubby, etc in Fedora 16 even if other features (most notably the block backend driver) do not make it. In the worst case, we may delay the announcement of the Dom0 feature to Fedora 17, but can include all progress completed in Fedora 16.


* Merge kernel-xen into 'kernel' RPM (MarkMcLoughlin)
== Documentation ==
* Make bzImage bootable so the same kernel can boot on Xen and bare-metal
** Patches for DomU bzImage boot are in xen-unstable and x86.git
** Adding Dom0 bzImage boot to the HV is a bit harder
* Remove images/xen/{initrd,vmlinux} from install trees (JeremyKatz ?)
* Make 'new-kernel-pkg' aware of HYPERVISOR setting in /etc/sysconfig/xen (DanielBerrange)
* Make virt-install use images/pxeboot/{initrd,vmlinux} for DomU installs (DanielBerrange)
* Make 'xen' SRPM build a xen-hypervisor containing the HV (DanielBerrange)


Nice to have:
See also:


* CPU hotplug
* Xen, http://www.xenproject.org/
* Migration support
* 32-on-64 support (both DomU and Dom0)
* LiveCD with pvops enabled kernel as DomU ?
* Kexec on pvops dom0


Unlikely to get: (ie feature regressions)
* The Xen mailing lists, hosted at xen.org


* PCI device passthrough to DomU
* The fedora-xen mailing list, hosted by Fedora


Finished items:
* "Dom0 xen support in Fedora 15?" on fedora-xen mailing list, http://lists.fedoraproject.org/pipermail/xen/2010-November/005205.html (note the latest kernel updates for Fedora 15 can be used as a dom0)


* x86_64 support upstream (see above)
* Michael Young's third-party Dom0 kernels, http://fedorapeople.org/~myoung/dom0/, provided a Dom0-capable kernel for Fedora before Linus accepted key Xen features (not compatible with Fedora 15+)
* Merged kernel (see above)
* First-cut of Dom0 boot (StephenTweedie, JuanQuintela, MarkMcLoughlin)
* ACPI
* ioremap
* IRQ routing
* DMA/contiguous pages
* MTRR
* privcmd interface, other /proc/xen bits, /sys/hypervisor, /dev/evtchn (MarkMcLoughlin)


== Test Plan ==
* Stefano Stabellini's announcement that Xen Dom0 support has been included in Linux 2.6.27, http://blog.xen.org/index.php/2011/01/14/linux-2-6-37-first-upstream-linux-kernel-to-work-as-dom0/


* Install a Fedora 10 host/Dom0 system
In order to configure Rawhide to boot in Dom0:
* Turn on Xen Hypervisor by setting 'HYPERVISOR=/boot/xen-3.2.gz' in /etc/sysconfig/kernel and running some tool (TBD)
* Turn on Xen Userspace by running 'chkconfig xend on'
* Reboot
* Observe presence of Xen via /sys/hypervisor/
* Run 'xm list' and observe Domain-0
* Use virt-install to deploy a Fedora 9/10 paravirt guest
* Use virt-install to deploy a Fedora 8 paravirt guest
* Use virt-install to deploy a Fedora 8 fullyvirt guest
* Use 'virsh' to save & restore a PV guest and HVM guest
* Balloon down memory of a guest with virsh.


== User Experience ==
1. yum install xen.


* Xen-ified kernel has same version as bare metal kernel
2. Edit /etc/default/grub to set "GRUB_DEFAULT="Xen 4.1.1". Update your GRUB 2 configuration using:
* All kernel updates support Xen
* All hardware that works in baremetal works in Xen Dom0
* No separate kernel RPM required for Xen DomU or Dom0
* Can enable/disable use of HV via a /etc/sysconfig setting


== Dependencies ==
<pre>grub2-mkconfig -o /boot/grub2/grub.cfg</pre>.


* Update to anaconda to not build a 'images/xen/' directory anymore
3. Configure a bridge device by setting /etc/sysconfig/network-scripts/ifcfg-virbr0:
* Update to 'new-kernel-pkg' to know how to install HV


== Contingency Plan ==
<pre>
DEVICE=virbr0
TYPE=Bridge
ONBOOT=yes
USERCTL=no
BOOTPROTO=dhcp
NM_CONTROLLED=no
</pre>


If Dom0 support is still not ready for Fedora 10 GA, the options are:
4. Configure eth0 to be a member of the bridge:


* Continue to ship with only DomU support
<pre>
* Drop Xen support altogether
DEVICE=eth0
HWADDR=[XX:XX:XX:XX:XX:XX]
TYPE=Ethernet
ONBOOT=yes
USERCTL=no
BOOTPROTO=none
BRIDGE=virbr0
NM_CONTROLLED=no
</pre>


== Documentation ==
5. Reboot and select your Xen kernel in the GRUB 2 boot menu


== Release Notes ==
== Release Notes ==


----
The Fedora Linux kernel may now provide the basis for a Xen-based virtualization solution. Xen is a hypervisor-based type-1 virtualization platform. The kernel now has the ability to boot in Xen's Dom0, a privileged domain that allows Fedora to provide driver and guest management support to Xen and other non-privileged, guest operating systems.
[[Category:ProposedFeature]]
<!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this feature, indicate them here.  You can also link to upstream documentation if it satisfies this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release. -->
 
== Comments and Discussion ==
 
* See [[Talk:Features/XenPvopsDom0]]  <!-- This adds a link to the "discussion" tab associated with your page.  This provides the ability to have ongoing comments or conversation without bogging down the main feature page -->
 
[[Category:Virtualization|XenPvopsDom0]]
[[Category:FeatureAcceptedF16]]
<!-- See https://fedorahosted.org/fesco/ticket/523 -->

Latest revision as of 13:12, 28 June 2013

Xen Pvops Dom0

Summary

pvops-based kernel to serve as Dom0 for a Xen-based system. Currently Fedora releases greater than 8 contain the Xen hypervisor and tools, but not a Xen Dom0-capable kernel.

Owner

  • Name: W. Michael Petullo
  • Email: mike@flyn.org
  • People Involved: Michael Young, Eduardo Habkost, Mark McLoughlin, Stephen Tweedie, Chris Wright, Juan Quintela, Markus Armbruster, Glauber Costa, Daniel Berrange et al.

Current status

  • Targeted release: Fedora 16
  • Last updated: 2011-09-15
  • Percentage of completion: 90%

In order to implement this feature, it is necessary to get full Xen Dom0 support upstream and in Fedora. The core Dom0 support has been accepted upstream and was released in Linux 2.6.37. Linux 2.6.39 brought support for the Xen network backend driver. Linux 3.0 brought the block backend driver. Linux 3.1 will bring the pci backend driver (for pci passthru).

Open bugs:

Detailed Description

Xen is a hypervisor-based type-1 virtualization solution. The Xen hypervisor requires a standard operating system that runs in a privileged domain, Dom0. The Dom0 operating system provides driver and guest management support to Xen and other guest operating systems running in the DomU domain. Fedora 8 provided a kernel that could serve as a Dom0 guest. However, newer Fedora releases dropped support for Dom0 (although they can run in DomU). The Xen project is presently pushing the features required for a Dom0 Linux kernel upstream. Once these features are available in the mainline kernel, it follows that Fedora could support Xen Dom0.

Benefit to Fedora

Fedora will benefit from a Xen Dom0-capable kernel. Xen has proven to be a competitive virtualization solution in real-world installations. Xen is different enough from KVM that the two technologies could complement each other within the Fedora Project. There are ongoing benchmarks that intend to define the relative performance of the two technologies.

Scope

  • Implement Dom0 support in upstream kernel (Complete)
  • Implement Dom0 backend drivers support in upstream kernel; fix remaining issues (Complete)
  • Modify 'grubby' to support GRUB syntax required by Xen (Complete)
  • Make 'new-kernel-pkg' aware of Xen using the HYPERVISOR option in /etc/sysconfig/kernel (Complete)
  • Ensure Fedora's virtualization tools support Xen
  • Possibly implement support for bridging in NetworkManager
  • Possibly implement support in Anaconda

How To Test

  • Install a Fedora Dom0 kernel and xen (kernel-PAE is required for i686)
  • Reboot and confirm the Dom0 kernel can boot on bare metal (no Xen) by selecting the appropriate GRUB menu option
  • Reboot and confirm grub.conf boots the new kernel using Xen by selecting the appropriate GRUB menu option
  • Observe presence of Xen via /sys/hypervisor/
  • Run 'xm list' and observe Domain-0
  • Use 'virt-install -l http://download.fedoraproject.org/pub/fedora/linux/releases/15/Fedora/x86_64 --ram 1024 --disk /var/lib/libvirt/images/f15.img --name F15' to deploy a Fedora paravirtualized guest
  • Use 'virt-install' to deploy a Fedora fully virtualized guest
  • Use 'virsh' to save & restore guests
  • Balloon down memory of a guest with 'virsh'

User Experience

  • Stock Fedora kernel supports both Xen dom0 and bare metal
  • All features and hardware supported on bare metal also work on Xen Dom0
  • Can enable and disable the use of the Xen hypervisor via an /etc/sysconfig/kernel setting
  • Can manage Xen guests using Fedora virtualization tools

Dependencies

Contingency Plan

There is presently some uncertainty as to when the backend driver support that the Dom0 guest provides DomU guests will be upstream. This may make the upstream kernel that is chosen by Fedora 16 or it may not. Regardless, we will have to touch a lot of subsystems, so it makes sense to start early.

Until all of the requisite Xen components are upstream, we can continue to develop this feature using third party kernels. These third party kernels will be configured to be as close as possible to the forthcoming upstream kernel. In addition, Linux 2.6.39 does contain many of the backend drivers. It may be used with configurations that do not require the block backend driver.

If all of the Xen backend drivers don't make Fedora 16, then little is lost. We can continue to test the Dom0 kernel (with a subset of its eventual features) and work on the supporting infrastructure. It would be useful for our effort to have a working Dom0, grubby, etc in Fedora 16 even if other features (most notably the block backend driver) do not make it. In the worst case, we may delay the announcement of the Dom0 feature to Fedora 17, but can include all progress completed in Fedora 16.

Documentation

See also:

  • The Xen mailing lists, hosted at xen.org
  • The fedora-xen mailing list, hosted by Fedora
  • Michael Young's third-party Dom0 kernels, http://fedorapeople.org/~myoung/dom0/, provided a Dom0-capable kernel for Fedora before Linus accepted key Xen features (not compatible with Fedora 15+)

In order to configure Rawhide to boot in Dom0:

1. yum install xen.

2. Edit /etc/default/grub to set "GRUB_DEFAULT="Xen 4.1.1". Update your GRUB 2 configuration using:

grub2-mkconfig -o /boot/grub2/grub.cfg

.

3. Configure a bridge device by setting /etc/sysconfig/network-scripts/ifcfg-virbr0:

DEVICE=virbr0
TYPE=Bridge
ONBOOT=yes
USERCTL=no
BOOTPROTO=dhcp
NM_CONTROLLED=no

4. Configure eth0 to be a member of the bridge:

DEVICE=eth0
HWADDR=[XX:XX:XX:XX:XX:XX]
TYPE=Ethernet
ONBOOT=yes
USERCTL=no
BOOTPROTO=none
BRIDGE=virbr0
NM_CONTROLLED=no

5. Reboot and select your Xen kernel in the GRUB 2 boot menu

Release Notes

The Fedora Linux kernel may now provide the basis for a Xen-based virtualization solution. Xen is a hypervisor-based type-1 virtualization platform. The kernel now has the ability to boot in Xen's Dom0, a privileged domain that allows Fedora to provide driver and guest management support to Xen and other non-privileged, guest operating systems.

Comments and Discussion