From Fedora Project Wiki

(Test Cases)
(Undo revision 240753 by 74.8.142.91 (Talk))
 
(5 intermediate revisions by 5 users not shown)
Line 29: Line 29:
 
=== Test Cases ===
 
=== Test Cases ===
  
This is the procedure I used to create the initial
+
This is the procedure I used to create the initial patch which allows libvirt to recognize/generate a huge page backed guest xml definition.  NB: While fairly low-level and useful to unit test, it is however not a mechanism directly visible to a typical user.
patch which allows libvirt to recognize/generate
 
a huge page backed guest xml definition.  NB: While
 
fairly low-level and useful to unit test, it is
 
however not a mechanism directly visible to a
 
typical user.
 
  
The goal here was to allow libvirt to request guest
+
The goal here was to allow libvirt to request guest backing by huge pages, which are essentially of 2MB size vs. that of a standard 4KB page.  Doing so offers a significant performance benefit in certain application scenarios.
backing by huge pages, which are essentially of 2MB
 
size vs. that of a standard 4KB page.  Doing so
 
offers a significant performance benefit in certain
 
application scenarios.
 
  
 +
==== Prepare the Host ====
  
Prepare the Host
+
Populate the huge page pool of a size suitable to support the guest image(s) which will be created:
----------------
 
 
 
Populate the huge page pool of a size suitable to
 
support the guest image(s) which will be created:
 
  
 
     # grep Huge /proc/meminfo
 
     # grep Huge /proc/meminfo
Line 55: Line 43:
 
     HugePages_Surp:        0
 
     HugePages_Surp:        0
 
     Hugepagesize:      2048 kB
 
     Hugepagesize:      2048 kB
 
 
     # echo 500 > /proc/sys/vm/nr_hugepages
 
     # echo 500 > /proc/sys/vm/nr_hugepages
 
 
     # grep Huge /proc/meminfo
 
     # grep Huge /proc/meminfo
 
     HugePages_Total:    500
 
     HugePages_Total:    500
Line 65: Line 51:
 
     Hugepagesize:      2048 kB
 
     Hugepagesize:      2048 kB
  
Note the above may take a considerable amount of
+
Note the above may take a considerable amount of time on a machine with fragmented physical
time on a machine with fragmented physical
+
memory.  So it is best to do so as soon after boot as possible.  Also on machines with limited
memory.  So it is best to do so as soon after
+
memory, populating a smaller number of pages may be necessary.
boot as possible.  Also on machines with limited
 
memory, populating a smaller number of pages may
 
be necessary.
 
  
Having created the free huge page pool, mount hugetlbfs
+
Having created the free huge page pool, mount hugetlbfs on the host.  If the mount point doesn't exist, create it first:
on the host.  If the mount point doesn't exist,
 
create it first:
 
  
 
     # mkdir /dev/hugepages
 
     # mkdir /dev/hugepages
 
     # mount -t hugetlbfs hugetlbfs /dev/hugepages
 
     # mount -t hugetlbfs hugetlbfs /dev/hugepages
  
Note the mount above must be in place before
+
Note the mount above must be in place before launching libvirtd as the daemon currently checks
launcing libvirtd as the daemon currently checks
+
for a hugetlbfs mount only upon startup.  So if the daemon is currently running, restart it:
for a hugetlbfs mount only upon startup.  So if
 
the daemon is currently running, terminate it:
 
  
     # kill `cat /usr/local/var/run/libvirtd.pid`
+
     # service libvirtd restart
  
Then (re)start the daemon. Doing so in separate
+
Look in <code>/var/log/messages</code> for any errors.
window running in the foreground will allow
 
diagnostics to make their way to stdout:
 
  
    # <path_to_daemon>/libvirtd
+
==== Launch the Guest ====
 
 
 
 
Launch the Guest
 
----------------
 
  
 
To launch the guest conventionally from virsh:
 
To launch the guest conventionally from virsh:
  
     # <path_to_virsh>/virsh --connect qemu:///system
+
     # virsh define test-guest.xml
 +
    Domain foo defined from test-guest.xml
  
    virsh #
+
In the above example the guest is tagged with the name "foo" in the associated XML definition:
  
At this point a guest must be defined by specifying
+
     # virsh list --all
an XML definition (more on this below), for example:
 
 
 
     virsh # define /etc/libvirt/qemu/hp_danpb-on.xml
 
    Domain foo defined from /etc/libvirt/qemu/hp_danpb-on.xml
 
 
 
In the above example the guest is tagged with the
 
name "foo" in the associated XML definition:
 
 
 
    virsh # list --all
 
 
     Id Name                State
 
     Id Name                State
 
     ----------------------------------
 
     ----------------------------------
Line 118: Line 83:
 
The guest may be launched via:
 
The guest may be launched via:
  
     virsh # start foo
+
     # start foo
 
     Domain foo started
 
     Domain foo started
  
And a VNC connection to the guest console can be
+
And a VNC connection to the guest console can be made via:
made via:
 
  
     # vncviewer localhost:5900
+
     # virt-viewer foo
  
If all goes well the guest should launch successfully
+
If all goes well the guest should launch successfully with its image backed by huge pages.  [Note it won't unless the guest XML definition specifies huge page usage correctly as below.  But proceeding here is instructive in any event.]
with its image backed by huge pages.  [Note it won't
 
unless the guest XML definition specifies huge page
 
usage correctly as below.  But proceeding here is
 
instructive in any event.]
 
  
Successful launch of a huge page backed guest may be
+
Successful launch of a huge page backed guest may be evidenced by observing the huge page free pool decreasing:  
evidenced by observing the huge page free pool decreasing:
 
  
 
     # grep Huge /proc/meminfo
 
     # grep Huge /proc/meminfo
Line 142: Line 101:
 
     Hugepagesize:      2048 kB
 
     Hugepagesize:      2048 kB
  
In the likely case HugePages_Free == HugePages_Total
+
In the likely case HugePages_Free == HugePages_Total take a look at the XML definition for the guest, For example:
take a look at the XML definition for the guest, For
 
example:
 
  
     virsh # dumpxml foo
+
     # virsh dumpxml foo
 
     <domain type='qemu'>
 
     <domain type='qemu'>
 
       <name>foo</name>
 
       <name>foo</name>
Line 159: Line 116:
 
         :
 
         :
  
The above does not specify a memory backing
+
The above does not specify a memory backing mechanism and therefore defaults to backing by
mechanism and therefore defaults to backing by
+
4KB pages.  To specify huge page backing a &lt;memoryBacking&gt; clause is needed:
4KB pages.  To specify huge page backing a
 
<memoryBacking> clause is needed:
 
  
     virsh # dumpxml foo
+
     # virsh dumpxml foo
 
     <domain type='qemu'>
 
     <domain type='qemu'>
 
       <name>foo</name>
 
       <name>foo</name>
Line 180: Line 135:
 
         :
 
         :
  
 +
To add this to the XML definition, edit the corresponding file to add the &lt;memoryBacking&gt; clause as above use <code>virsh edit</code>:
  
To add this to the XML definition, edit the corresponding
+
     # virsh edit foo
file to add the <memoryBacking> clause as above, undefine,
+
     Domain foo XML configuration edited.
and redefine the guest:
 
 
 
     virsh # undefine foo
 
     Domain foo has been undefined
 
 
 
    virsh # define /etc/libvirt/qemu/hp_danpb-on.xml    [edited XML def]
 
    Domain foo defined from /etc/libvirt/qemu/hp_danpb-on.xml
 
 
 
    virsh # start foo
 
    Domain foo started
 
 
 
This should result in a huge page back guest launch
 
which may be verified as above.
 
  
 +
This should result in a huge page backed guest launch which may be verified as above.
  
Possible Caveat
+
==== Possible Caveat ====
---------------
 
  
There was a modification to the default disposition of
+
There was a modification to the default disposition of selinux genfscon fs types affecting (among others) hugetlbfs in the kernel 2.6.29-2.6.30 timeframe.  This manifests as failure of chcon(1) on hugetlbfs files. Correction requires a selinux policy change for hugetlbfs and a corresponding kernel fs change.  Neither of which have been conclusively tested as of this writing on prospective FC12.  Thus there is a possibility SELINUX may need to be disabled to allow successful launch of a huge page backed guest.
selinux genfscon fs types affecting (among others)
 
hugetlbfs in the kernel 2.6.29-2.6.30 timeframe.  This
 
manifests as failure of chcon(1) on hugetlbfs files.
 
Correction requires a selinux policy change for hugetlbfs
 
and a corresponding kernel fs change.  Neither of which
 
have been conclusively tested as of this writing on
 
prospective FC12.  Thus there is a possibility SELINUX
 
may need to be disabled to allow successful launch of a
 
huge page backed guest.
 
  
 
=== Issues that were identified ===
 
=== Issues that were identified ===
Line 218: Line 152:
 
| Tester || Description || Bug references || Status
 
| Tester || Description || Bug references || Status
 
|-  
 
|-  
| || || [http://bugzilla.redhat.com/XXXXXX #XXXXX] || '''ASSIGNED'''
+
| caiqian || Huge Page Backed Memory Failed for Kqemu Guests || [http://bugzilla.redhat.com/527670 #527670] || '''NEW'''
 
|}
 
|}
  
[[Category:Test Days]]
+
[[Category:Fedora_12_Test_Days]]
 
[[Category:Virtualization]]
 
[[Category:Virtualization]]

Latest revision as of 06:26, 8 June 2011

DATE TIME WHERE
Thursday Sep 17, 2009 All day #fedora-test-day (webchat)

What to test?[edit]

This part of today's Fedora Test Day will focus on testing the KVM Huge Page Backed Memory feature in Fedora 12.

If you come to this page after the test day is completed, your testing is still valuable, and you can use the information on this page to test huge pages support and provide feedback.

Who's available[edit]

John Cooper is your host for today.

The following people have also agreed to be available for testing, workarounds, bug fixes, and general discussion:

What's needed to test[edit]

Test Cases[edit]

This is the procedure I used to create the initial patch which allows libvirt to recognize/generate a huge page backed guest xml definition. NB: While fairly low-level and useful to unit test, it is however not a mechanism directly visible to a typical user.

The goal here was to allow libvirt to request guest backing by huge pages, which are essentially of 2MB size vs. that of a standard 4KB page. Doing so offers a significant performance benefit in certain application scenarios.

Prepare the Host[edit]

Populate the huge page pool of a size suitable to support the guest image(s) which will be created:

   # grep Huge /proc/meminfo
   HugePages_Total:       0
   HugePages_Free:        0
   HugePages_Rsvd:        0
   HugePages_Surp:        0
   Hugepagesize:       2048 kB
   # echo 500 > /proc/sys/vm/nr_hugepages
   # grep Huge /proc/meminfo
   HugePages_Total:     500
   HugePages_Free:      500
   HugePages_Rsvd:        0
   HugePages_Surp:        0
   Hugepagesize:       2048 kB

Note the above may take a considerable amount of time on a machine with fragmented physical memory. So it is best to do so as soon after boot as possible. Also on machines with limited memory, populating a smaller number of pages may be necessary.

Having created the free huge page pool, mount hugetlbfs on the host. If the mount point doesn't exist, create it first:

   # mkdir /dev/hugepages
   # mount -t hugetlbfs hugetlbfs /dev/hugepages

Note the mount above must be in place before launching libvirtd as the daemon currently checks for a hugetlbfs mount only upon startup. So if the daemon is currently running, restart it:

   # service libvirtd restart

Look in /var/log/messages for any errors.

Launch the Guest[edit]

To launch the guest conventionally from virsh:

   # virsh define test-guest.xml
   Domain foo defined from test-guest.xml

In the above example the guest is tagged with the name "foo" in the associated XML definition:

   # virsh list --all
    Id Name                 State
   ----------------------------------
     - foo                  shut off

The guest may be launched via:

   # start foo
   Domain foo started

And a VNC connection to the guest console can be made via:

   # virt-viewer foo

If all goes well the guest should launch successfully with its image backed by huge pages. [Note it won't unless the guest XML definition specifies huge page usage correctly as below. But proceeding here is instructive in any event.]

Successful launch of a huge page backed guest may be evidenced by observing the huge page free pool decreasing:

   # grep Huge /proc/meminfo
   HugePages_Total:     500
   HugePages_Free:      481
   HugePages_Rsvd:      247
   HugePages_Surp:        0
   Hugepagesize:       2048 kB

In the likely case HugePages_Free == HugePages_Total take a look at the XML definition for the guest, For example:

   # virsh dumpxml foo
   <domain type='qemu'>
     <name>foo</name>
     <uuid>4c58c2a6-1b52-688e-bcfb-e57159f50961</uuid>
     <memory>524288</memory>
     <currentMemory>524288</currentMemory>
     <vcpu>1</vcpu>
     <os>
       <type arch='x86_64' machine='pc'>hvm</type>
       <boot dev='hd'/>
     </os>
       :

The above does not specify a memory backing mechanism and therefore defaults to backing by 4KB pages. To specify huge page backing a <memoryBacking> clause is needed:

   # virsh dumpxml foo
   <domain type='qemu'>
     <name>foo</name>
     <uuid>4c58c2a6-1b52-688e-bcfb-e57159f50961</uuid>
     <memory>524288</memory>
     <currentMemory>524288</currentMemory>
     <memoryBacking>
       <hugepages/>
     </memoryBacking>
     <vcpu>1</vcpu>
     <os>
       <type arch='x86_64' machine='pc'>hvm</type>
       <boot dev='hd'/>
     </os>
       :

To add this to the XML definition, edit the corresponding file to add the <memoryBacking> clause as above use virsh edit:

   # virsh edit foo
   Domain foo XML configuration edited.

This should result in a huge page backed guest launch which may be verified as above.

Possible Caveat[edit]

There was a modification to the default disposition of selinux genfscon fs types affecting (among others) hugetlbfs in the kernel 2.6.29-2.6.30 timeframe. This manifests as failure of chcon(1) on hugetlbfs files. Correction requires a selinux policy change for hugetlbfs and a corresponding kernel fs change. Neither of which have been conclusively tested as of this writing on prospective FC12. Thus there is a possibility SELINUX may need to be disabled to allow successful launch of a huge page backed guest.

Issues that were identified[edit]

Tester Description Bug references Status
caiqian Huge Page Backed Memory Failed for Kqemu Guests #527670 NEW