Error 9090: Xen 4.2 with Centos 6.6 but Ubuntu/Xen works fine on same hardware

Question

Using DVD1 from the current Centos 6.6 x86_64 installation, I can install Centos 6.6 minimal to install successfully on a Intel NUC computer with 20GB Flash and 4 GB RAM. The kernel is 2.6.32-504.el6.x86_64. Xen4CentOS will not boot, but loading Ubuntu/Xen on the same system works so it's not the hardware (tested on two identical hardware devices, same results).

I cannot get the Xen4CentOS installation (xen-4.2.5-37.el6) to work for kernel 3.10.56.11-el6.centos.alt.X86_64 following the installation http://wiki.xenproject.org/wiki/Xen4CentOS as my reference. There are sporadic questions about the error that occurs '9090' upon reboot but no one solidly asks the question or provides sufficient information on how they get the error in more than 2-3 lines of text, which I found insufficient for reproduction of the problem. (humbly asking that you take care not to mark this down or previously discussed by doing a quick search on 9090, I've spent a day on this and could use some hints)

The steps I followed were:

1) Burn DVD1, DVD2 from Centos Mirrors for installation (CentOS 6.6)
2) Install 'minimal' CentOS installation following all defaults with only the CentOS Repo, this required only DVD1 for the 'minimal' install
3) Disabled SE Linux and rebooted (to eliminate any concern, I know it's a bit harsh but its for troubleshooting)
4) Reboot into CentOS 6.6 - 2.6 Kernel
5) yum install centos-release-xen (accept 1 pkg install)
6) yum install xen (accept 36 packages install)
7) /usr/bin/grub-bootxen.sh
8) reboot
9) Upon selecting the 3.10 kernel => Get "Invalid magic number: 9090 Error 13: Invalid or unsupported executable format 

The first indication that the instructions are not working with my hardware is "This should produce an entry in /boot/grub/grub.conf which looks something like:". Grub on Centos 6.6 does not put the configuration file in that directory on my architecture, however it puts it here /boot/efi/EFI/redhat/grub.conf and puts a symbolic link here /etc/grub.conf.

Here is the grub file:

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,1)
#          kernel /vmlinuz-version ro root=/dev/mapper/vg_chassis0-lv_root
#          initrd /initrd-[generic-]version.img
#boot=/dev/sda1
device (hd0) HD(1,800,64000,6d6512a3-0f84-44db-a808-1191d98cd207)
default=0
timeout=5
splashimage=(hd0,1)/grub/splash.xpm.gz
hiddenmenu
title CentOS (3.10.56-11.el6.centos.alt.x86_64)
    root (hd0,1)
    kernel /xen.gz dom0_mem=1024M,max:1024M loglvl=all guest_loglvl=all
    module /vmlinuz-3.10.56-11.el6.centos.alt.x86_64 ro root=/dev/mapper/vg_chassis0-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=vg_chassis0/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_chassis0/lv_root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
    module /initramfs-3.10.56-11.el6.centos.alt.x86_64.img
title CentOS 6 (2.6.32-504.el6.x86_64)
    root (hd0,1)
    kernel /vmlinuz-2.6.32-504.el6.x86_64 ro root=/dev/mapper/vg_chassis0-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=vg_chassis0/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_chassis0/lv_root  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
    initrd /initramfs-2.6.32-504.el6.x86_64.img

Here are the system mounts:

Filesystem           1K-blocks    Used Available Use% Mounted on
/dev/mapper/vg_chassis0-lv_root
                      25131356  958760  22889320   5% /
tmpfs                  1683088       0   1683088   0% /dev/shm
/dev/sda2               487652   74666    387386  17% /boot
/dev/sda1               204580     260    204320   1% /boot/efi

I have looked at the grub file, the grub-bootxen.sh bash script. Repeated installations reproduce the same result. I'm grateful to anyone out there who might have a hint at how to get past this one or have something I might check...

Answer 1

Of all things...

If the exact same process above is followed, however one uses the Centos 6.6 Netboot install and selects: X86_64, with minimal install and loading the installation from the URL (not from a DVD version) it works! The URL I used was:

http://mirror.centos.org/centos/6/os/x86_64

both 'xl list' and 'xm list' show the expected results.

For clarity here are the new steps:

1) Download and install 'netboot' version of (CentOS 6.6)  
2) Install 'minimal' CentOS installation following all defaults with only the CentOS Repo, this required entering a URL for the location of the software: mirror.centos.org/centos/6/x86_64
3) Rebooted (no change to SE Linux)
4) Reboot into CentOS 6.6 - 2.6 Kernel
5) yum install centos-release-xen (accept 1 pkg install)
6) yum install xen (accept 36 packages install)
7) /usr/bin/grub-bootxen.sh
8) checked grub.conf file was present according to Xen4QuickStart under the /boot/grub/, which it was exactly as they described
9) reboot
10) Login and run 'xl list', 'xm list' showing expected results, 

Note: no changes to BIOS on the Intel NUC, no updates to BIOS required, no change to GRUB, nor to any configuration, simply an install from netboot. My thanks to another user who had this problem sometime back who suggested it may be related to EFI and that install from the site may resolve it.

Answer 2

I have exactly the same problem. Could not find any solution yet but I think that this is not a minor problem and maybe GRUB-related. Replacing xen.gz with another proofed version did not work either.

My steps during installation were the same but I was installing from the full DVD image.

System:

• i7-5820K
• Gigabyte X99-UD4H
• 32GB DDR4-Ram
• 512GB Samsung SSD

The system mounts:

Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/xen-root 10190136 4860724 4805124 51% / tmpfs 16351060 76 16350984 1% /dev/shm /dev/sda2 198337 80825 107272 43% /boot /dev/sda1 204580 376 204204 1% /boot/efi

I also tried multiple installations. File system of /boot I tested: ext2, ext4.