This particular dom0 is running Ubuntu 8.10 on an amd64 kernel using an lvm volume group called ‘vg0’. Adjust accordingly. Note that the credentials for this particular live cd are:
unprivileged username: jack
unprivileged password: jack
privileged username: root
privileged password: opensolaris
I have been meaning to add a Solaris machine to my network for a while. I stopped by #opensolaris on freenode, and was helped a great deal by a guy that goes by the nick “pino42.” The gist is this:
- Download the ISO:
cjac@dom0:~$ sudo mkdir /var/xen/osol && cd /var/xen/osol cjac@dom0:/var/xen/osol$ sudo wget -O osol-0906-ai-x86.iso http://dlc.sun.com/osol/opensolaris/2009/06/osol-0906-ai-x86.iso
- Mount it on /mnt/osol/:
cjac@dom0:/var/xen/osol$ sudo losetup /dev/loop0 osol-0906-x86.iso cjac@dom0:/var/xen/osol$ sudo mkdir /mnt/osol cjac@dom0:/var/xen/osol$ sudo mount /dev/loop0 /mnt/osol
- Copy the live kernel and ram disk from the iso so that Xen can boot from them:
cjac@dom0:/var/xen/osol$ sudo mkdir /boot/osol cjac@dom0:/var/xen/osol$ sudo cp /mnt/osol/platform/i86xpv/kernel/amd64/unix /boot/osol/ cjac@dom0:/var/xen/osol$ sudo cp /mnt/osol/boot/amd64/x86.microroot /boot/osol/
- Create the target lv to which we’ll install the OS:
cjac@dom0:/var/xen/osol$ sudo lvcreate /dev/vg0 -n solaris0-disk -L 4G
- Create the xen domU .cfg file for the live CD/installer:
cjac@dom0:/var/xen/osol$ sudo -s root@dom0:/var/xen/osol# cat > /etc/xen/solaris-live.cfg name = "solaris-live" vcpus = 1 memory = "768" # 512 will work, but it's slow kernel = "/boot/osol/unix" ramdisk = "/boot/osol/x86.microroot" extra = "/mnt/osol/platform/i86xpv/kernel/amd64/unix -kd - nowin -B install_media=cdrom" disk = ['phy:/dev/loop0,6:cdrom,r','phy:vg0/solaris0-disk,0,w'] vif = ['bridge=net,mac=00:16:3e:aa:b9:bc'] on_shutdown = "destroy" on_reboot = "destroy" on_crash = "preserve" ^D root@dom0:/var/xen/osol# exit
- Make sure you’ve got a DHCP server on the bridge you decide to use for ‘vif’. Setting this up is left as an exercise for the reader.
- All should be set to launch the live installer. Do so:
cjac@dom0:/var/xen/osol$ sudo xm create -c solaris-live.cfg
- At the kernel debugger prompt, continue:
Started domain solaris-live Loading kmdb... Welcome to kmdb Loaded modules: [ unix krtld genunix ] [0]> :c
- After selecting your keyboard layout and desktop language, log in:
opensolaris console login: jack Password: jack
- You will know that the DHCP server has assigned you an address when you see a message similar to the following on the console:
Jun 12 10:21:07 opensolaris in.routed[688]: route 0.0.0.0/8 --> 0.0.0.0 nexthop is not directly connected
The message will probably be different if you happen to have a RIP server functioning on the same link as the DHCP server. But I do not.
To discover the IP address which has been assigned by the DHCP server, do the following:
jack@opensolaris:~$ ifconfig -a lo0: flags=2001000849
mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 xnf0: flags=1004843 mtu 1500 index 2 inet 192.168.10.66 netmask ffffff00 broadcast 192.168.10.255 lo0: flags=2002000849 mtu 8252 index 1 inet6 ::1/128 xnf0: flags=2004841 mtu 1500 index 2 inet6 fe80::216:3eff:feaa:b9bc/10 - Since the installer is graphical, we’ll need to attach to the X server using vnc. On the solaris machine, you need to start the vnc server. You will be prompted for a password, which must be at least 6 characters in length:
jack@opensolaris:~$ mkdir ~/.vnc jack@opensolaris:~$ cp ~/.Xclients ~/.vnc/xstartup jack@opensolaris:~$ vncserver You will require a password to access your desktops. Password: password Verify: password
- On a machine with an X display, connect to a machine which can communicate with the host and establish a VNC session:
cjac@laptop:~$ echo $DISPLAY && ssh -AY dhcp-server :0.0 cjac@dhcp-server:~$ sudo apt-get install xvnc4viewer cjac@dhcp-server:~$ echo $DISPLAY && vncviewer 192.168.10.66:1 localhost:11.0 Password: password
- You should now have an X window with a view of the remote Solaris installer:
- Double-click on the "Install OpenSolaris" icon and proceed through the configuration. DO NOT reboot the system on completion, as we've got some Xen-specific mucking to do after the installer has completed.
- Return to your xm console shell and elevate your priviliges to root:
jack@opensolaris:~$ su - Password: opensolaris Jun 12 10:50:24 opensolaris su: 'su root' succeeded for jack on /dev/console Sun Microsystems Inc. SunOS 5.11 snv_111b November 2008 root@opensolaris:~#
- I slightly modified the script I found here to get a copy of the running kernel and boot archive:
root@opensolaris:~# cat > get_images.sh #/bin/bash dom0=$1 dompath=$2 unixfile=/platform/i86xpv/kernel/amd64/unix zdb rpool > /tmp/zdb.out # this takes 5+ minutes bootpath=`grep phys_path /tmp/zdb.out | awk -F\' '{print $2}'` dataset_id=`grep 'Dataset rpool/ROOT/opensolaris ' /tmp/zdb.out | awk -F, '{print $2}' | cut -d' ' -f 3` root=`pfexec beadm list -H | grep ';N*R;' | cut -d \; -f 1` mkdir /tmp/root pfexec beadm mount $root /tmp/root 2>/dev/null mount=`pfexec beadm list -H $root | cut -d \; -f 4` pfexec bootadm update-archive -R $mount echo "Kernel and ramdisk for $root should now be copied to $dom0:$dompath" echo "${mount}${unixfile} -> $dom0:$dompath/kernel.$root" echo "$mount/platform/i86pc/amd64/boot_archive -> $dom0:$dompath/ramdisk.$root" # pfexec beadm umount $root 2>/dev/null" echo "Kernel cmdline should be:" echo "$unixfile -B zfs-bootfs=rpool/$dataset_id,bootpath=$bootpath" ^D root@opensolaris:~# chmod u+x get_images.sh
- Run the script:
root@opensolaris:~# ./get_images.sh dom0 /boot/osol amd64
- Go get a cup of coffee or something. When you return, you should see something like this:
Kernel and ramdisk for opensolaris should now be copied to dom0:/boot/osol /tmp/root/platform/i86xpv/kernel/amd64/unix -> dom0:/boot/osol/kernel.opensolaris /tmp/root/platform/i86pc/amd64/boot_archive -> dom0:/boot/osol/ramdisk.opensolaris Kernel cmdline should be: /platform/i86xpv/kernel/amd64/unix -B zfs-bootfs=rpool/42,bootpath=/xpvd/xdf@0:a
- Copy the kernel and ramdisk to the dom0. Your dom0 is probably not on the same network segment as your dhcp server, but I'll assume that you can ssh from the dom0 to the dhcp server:
jack@opensolaris:~$ scp /tmp/root/platform/i86xpv/kernel/amd64/unix cjac@dhcp-server:/tmp/kernel.opensolaris jack@opensolaris:~$ scp /tmp/root/platform/i86pc/amd64/boot_archive cjac@dhcp-server:/tmp/ramdisk.opensolaris cjac@dom0:~$ sudo scp cjac@dhcp-server:/tmp/ramdisk.opensolaris /boot/osol/ cjac@dom0:~$ sudo scp cjac@dhcp-server:/tmp/kernel.opensolaris /boot/osol/
- shut the live domU down:
jack@opensolaris:~$ jack@opensolaris:~$ pfexec halt Jun 12 22:00:46 opensolaris halt: initiated by jack on /dev/console
- Create your final Xen domU config file
cjac@dom0:~$ sudo -s root@dom0:~# cat > /etc/xen/solaris0.cfg name = 'solaris0' vcpus = 1 memory = "512" vif = ['bridge=loc,mac=00:16:3e:aa:b9:bc'] disk = [ 'phy:/dev/vg0/solaris0-disk,0,w' ] kernel = '/boot/osol/kernel.opensolaris' ramdisk = '/boot/osol/ramdisk.opensolaris' extra = '/platform/i86xpv/kernel/amd64/unix -B zfs-bootfs=rpool/42,bootpath=/xpvd/xdf@0:a' on_shutdown = 'destroy' on_reboot = 'destroy' on_crash = 'destroy' ^D root@dom0:~# exit
- Instantiate the domU
cjac@dom0:~$ sudo xm create -c solaris0.cfg Using config file "/etc/xen/solaris0.cfg". Started domain solaris0 v3.2-1 chgset 'unavailable' SunOS Release 5.11 Version snv_111b 64-bit Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. Hostname: solaris0 Reading ZFS config: done. Mounting ZFS filesystems: (6/6) solaris0 console login:
6 responses to “Installing Open Solaris as a Xen domU”
Heh, that’s pretty cool. I haven’t touched xen yet (not a lot of work yet), but I have many long hours playing with Solaris. Neat.
It’s good to hear from you Dylan! Long time no see! I hope the Salmon Plan is going well. :)
Let me know if you want an account or a vm or something.
Your article got me up and running, thanks much!
glad to hear it!
CJ
Thanks so much for posting this!
I was struggling with getting NexentaStor Community Edition 3.1.1 running in XEN-4.0 with Debian Squeeze Dom0 until I found your post.
Performance during the install phase is still kind of awful. I’m passing Nexenta a linux software raid-1 set consisting of two partitions on my boot drives, and I think this may be the issue.
I’ll be passing a whole controller through as well (once I figure that out). My hope is that I can either
a) figure out why the rpool (syspool) performance seems so bad.
b) discover that the root volume performance doesn’t really matter for my virtual NAS appliance.
Either way, I’m leaps and bounds ahead of where I was prior to finding your blog!
Hey there Chris,
I’m glad I could help!
Cheers,
C.J.