Skip to content

Guidelines to submit patch in xen-devel…

Please try to follow the guidelines below.  This will make things
easier on the maintainers.  Not all of these guidelines matter for every
trivial patch so apply some common sense.

1.    Always _test_ your changes, however small, on at least 4 or
5 people, preferably many more.

2.    Make sure your changes compile correctly in multiple
configurations. For example, both 32- and 64-bit x86.

3.    Make a patch available to the relevant maintainer in the list. Use
‘diff -u’ to make the patch easy to merge. Be prepared to get your
changes sent back with seemingly silly requests about formatting
and variable names.  These aren’t as silly as they seem. One
job the maintainersdo is to keep things looking the same.

PLEASE try to include any credit lines you want added with the
patch. It avoids people being missed off by mistake and makes
it easier to know who wants adding and who doesn’t.

PLEASE document known bugs. If it doesn’t work for everything
or does something very odd once a month document it.

PLEASE remember that submissions must be made under the terms
of the “Developer’s Certificate of Origin” (DCO) and should include
a Signed-off-by: line.

4.    Make sure you have the right to send any changes you make. If you
do changes at work you may find your employer owns the patch
not you.

5.    Happy hacking.

check for more information.

Abstract for OLS

Huge pages are the memory pages of size 2MB. Huge pages improve the performance of the system by 15-20% since number of page walks required for translation from a virtual address to physical 2MB page are reduced as compared to page walks required for translation from a virtual address to physical 4KB page. Also the number of TLB entries per 2MB chunk in memory is reduced by a factor of 512 as compared to 4KB pages.
In the context of virtualization, i.e. Xen hyper-visor,we propose an innovative design and implementation to support huge pages for paravirtualized guest paging operations.

Our solution presented in this paper is unique in the way that it modifies a layer of memory management at the level of physical memory in Xen. The solution allocates 2MB pages, i.e. ,huge page, along with the normal 4 KB pages before a domain boots. As the design partitions a domain’s committed memory into 2MB chunks as per demand before a domain boots, the availability of the huge pages increases and more number of actual physical huge pages can be provided to the paravirtualized domain. This increases the performance of the applications hosted on the guest operating system which require the huge page support. This design solves the problem of availability of 2MB chunk in guest’s physical address space (virtualized) as well as the Xen’s physical address space.

Xen Installation

Installing xen for first time is headache. Choosing right combination of kernel and xen is important and must.  Wrong combination may waste your number of fruitful days.

Here is the link to install xen-4.0.1 on Fedora-13

Reserving Hugepages for MySql

To get a hugepage support for mysql, follow the step given below:

1. Allocate Hugepages for Mysql using following command

[ashwin@wildfire]$ echo 40 > /proc/sys/vm/nr_hugepages

Other way to configure Hugepages

[ashwin@wildfire]$ sysctl -w vm.nr_hugepages=40

2. Check the Status of Allocation

[ashwin@wildfire]$ grep -i huge /proc/meminfo

HugePages_Total: 40

HugePages_Free: 40

HugePages_Rsvd: 0

Hugepagesize: 2048 kB

HugePages_Total : Total Number of Hugepages in pools
HugePages_Free  : Number of free Hugepages(unallocated) in pool
HugePages_Rsvd  : Number of Committed Hugepages to kernel      
Hugepagesize    : Size of Hugepages

3. Get mysql process id using following command

[ashwin@wildfire]$id mysql

uid=27(mysql) gid=27(mysql) groups=27(mysql)

4. Do following changes

[ashwin@wildfire]$ vi /etc/sysctl.conf

Add the following 4 lines :




kernel.shmmax = 68719476736

kernel.shmall = 4294967296


[ashwin@wildfire]$ sysctl -p

[ashwin@wildfire]$vi /etc/my.cnf









[ashwin@wildfire]$vi /etc/security/limits.conf



@mysql soft memlock unlimited

@mysql hard memlock unlimited


5. Restart Mysql server

[ashwin@wildfire]$service mysqld restart


6. Check whether the hugepages are configured to mysql.

[ashwin@wildfire]$ grep -i huge /proc/meminfo

HugePages_Total: 40

HugePages_Free: 40

HugePages_Rsvd: 40

Hugepagesize: 2048 kB

New Implementation for Allocation of Hugepages in Xen

We have implemented hugepage support for guests in following manner

In our implementation we added a parameter hugepage_num which is specified in the config file of the DomU. It is the number of hugepages that the guest is guaranteed to receive whenever the kernel asks for hugepage by using its boot time parameter or reserving after booting (eg. Using echo XX > /proc/sys/vm/nr_hugepages). During creation of the domain we reserve MFN’s for these hugepages and store them in the list. The listhead of this list is inside the domain structure with name “hugepage_list”. When the domain is booting, at that time the memory seen by the kernel is less amount required for hugepages. The function reserve_hugepage_range is called as a initcall. Before this function the xen_extra_mem_start points to this apparent end of the memory. In this function we reserve the PFN range for the hugepages which are going to be allocated by kernel by incrementing the xen_extra_mem_start. We maintain these PFNs as pages in “xen_hugepfn_list” in the kernel. Now when the kernel requests for hugepages, it makes a hypercall to xen hypervisor. Hypervisor will return mfn from the link-list and these will will be mapped to p2m mapping table in kernel. Like this kernel can allocate hugepages in virtualized  environment.

Enabling Printk’s in Paging

OMG!!! Debugging kernel code is really difficult but debugging paging in kernel code is more difficult. Try this,before pagetables are setup put printk in buddy allocator code and check what will happen. Don’t waste time in exploring this, kernel will get crash as pagetables are not setup and libraries are not initialized. But still you want to debug the code after page tables are setup. Follow the steps given below :

1. Add below declaration in any of the file (say mm.h)
int post_printk=0;

2. Define following macro in your code

#define MY_PRINTK(print_this)  if(post_printk) printk(“\n%s\n”print_this)

In kernel programming always put “\n” at the end of printks which ensure that printk’s will be printed even though kernel is crashed.

3. Now set post_printk=1 when you want to enable printks.

4. So rather than putting printks use the macro definition defined above.

MY_PRINTK(“This is my message in printk”);

Tried Solutions for Hugepage Allocation in Xen

1)Allocating pages during xen-boot time:
When Xen get-booted we have reserved few 2MB (order = 9) from the alloc calls explained earlier. We figure out these mfn of the reserved pool which lies in 1-to-1 domain region. What we did was when domain request a hugepage it makes and hyper-call so that we can allocate page from this pool. This method failed because phy_to_machine mapping was not done.

2)Case 1 by  mapping pages on kernel side .
Using set phy_to_mach at kernel side the mfns returned by our hypercall from xen were mapped to physical space in kernel . This approach failed since phy_to_mach was mapping not done on xen side.

3)Make mapping on xen side  before sending mfn to kernel. This case is under testing and partially implemented .