Ganeti 2.4.x installation tutorial for CentOS/Scientific Linux ============================================================== Installing the base system and base components ++++++++++++++++++++++++++++++++++++++++++++++ /etc/hosts **Mandatory** on one node per cluster. 127.0.0.1 localhost # cluster name 192.168.1.1 gcluster # node names 192.168.1.11 node1.example.com node1 192.168.1.12 node2.example.com node2 # instance names 192.168.1.101 instance1.example.com instance1 192.168.1.102 instance2.example.com instance2 Installing The Hypervisor +++++++++++++++++++++++++ **Mandatory** on all nodes. .. admonition:: KVM on CentOS/Scientific Linux yum install kvm kvm-qemu-img python-virtinst .. admonition:: Xen on CentOS/Scientific Linux yum install xen KVM settings ~~~~~~~~~~~~ **Mandatory** on all nodes. .. admonition:: KVM on CentOS/Scientific Linux create bridge interface ex) br0 (eth0) edi /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE="eth0" BOOTPROTO="static" HWADDR="??:??:??:??:??:??" ONBOOT="yes" BRIDGE="br0" edit /etc/sysconfig/network-scripts/ifcfg-br0 DEVICE="br0" TYPE=Bridge BOOTPROTO="static" IPADDR="192.168.1.11" NETMASK="255.255.255.0" ONBOOT="yes" apply network configurations. /etc/init.d/network restart allow to bridge interface access. edit /etc/sysconfig/iptables *filter ... -A INPUT -j REJECT --reject-with icmp-host-prohibited ## FORWARD -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT COMMIT apply firewall rules. iptables-restore < /etc/sysconfig/iptables Xen settings ~~~~~~~~~~~~ **Mandatory** on all nodes. .. admonition:: Xen on CentOS/Scientific Linux chkconfig xend on chkconfig xendomains on chkconfig libvirtd off /etc/xen/xend-config.sxp (dom0-min-mem 0) (xend-relocation-server yes) (xend-relocation-port 8002) (xend-relocation-hosts-allow '') add dom0_mem to /etc/grub.conf title CentOS (2.6.18-194.32.1.el5xen) root (hd0,0) kernel /xen.gz-2.6.18-194.32.1.el5 dom0_mem=512M module /vmlinuz-2.6.18-194.32.1.el5xen ro root=/dev/VolGroup00/HostRoot module /initrd-2.6.18-194.32.1.el5xen.img You need to restart the Xen daemon for these settings to take effect: /etc/init.d/xend restart After installing either hypervisor, you need to reboot into your new system. On some distributions this might involve configuring GRUB appropriately, whereas others will configure it automatically when you install the respective kernels. reboot Selecting the instance kernel ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Mandatory** on all nodes. cd /boot ln -s vmlinuz-`uname -r` vmlinuz-2.6-xenU Installing DRBD +++++++++++++++ **Mandatory** on all nodes. .. admonition:: CentOS yum install drbd kmod-drbd-xen .. admonition:: Scientific Linux 6.x rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://elrepo.org/elrepo-release-6-3.el6.elrepo.noarch.rpm yum --enablerepo=elrepo install kmod-drbd83 .. admonition:: DRBD 8.3.0 or later create /etc/default/drbd ADD_MOD_PARAM="usermode_helper=/bin/true" Other required software +++++++++++++++++++++++ Use Extra Packages for Enterprise Linux (EPEL) repository ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ wget http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm rpm -ivh epel-release-5-4.noarch.rpm rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL sed -i "s/enabled = 1/enabled = 0/g" /etc/yum.repos.d/epel.repo rm epel-release-5-4.noarch.rpm -rf yum --enablerepo=epel install pyOpenSSL python-simplejson pyparsing python-inotify python-ctypes python-pycurl python-paramiko debootstrap .. admonition:: KVM yum --enablerepo=epel socat Configuring LVM +++++++++++++++ **Mandatory** on all nodes. The volume group is required to be at least 20GiB. If you haven't configured your LVM volume group at install time you need to do it before trying to initialize the Ganeti cluster. This is done by formatting the devices/partitions you want to use for it and then adding them to the relevant volume group:: pvcreate /dev/sda3 vgcreate vmvg /dev/sda3 or:: pvcreate /dev/sdb1 pvcreate /dev/sdc1 vgcreate vmvg /dev/sdb1 /dev/sdc1 If you want to add a device later you can do so with the *vgextend* command:: pvcreate /dev/sdd1 vgextend vmvg /dev/sdd1 Optional: it is recommended to configure LVM not to scan the DRBD devices for physical volumes. This can be accomplished by editing ``/etc/lvm/lvm.conf`` and adding the ``/dev/drbd[0-9]+`` regular expression to the ``filter`` variable, like this:: filter = ["r|/dev/cdrom|", "r|/dev/drbd[0-9]+|" ] Installing Ganeti +++++++++++++++++ .. admonition:: CentOS 6.x/Scientific Linux 6.x **Mandatory** on all nodes. wget http://jfut.integ.jp/linux/ganeti/noarch/ganeti-2.4.5-2.el6.noarch.rpm wget http://jfut.integ.jp/linux/ganeti/noarch/ganeti-instance-debootstrap-0.7-2.el6.noarch.rpm rpm -ivh ganeti-2.4.5-2.el6.noarch.rpm ganeti-instance-debootstrap-0.7-2.el6.noarch.rpm .. admonition:: CentOS 5.x/6.x/Scientific Linux 5.x **Mandatory** on all nodes. wget http://jfut.integ.jp/linux/ganeti/noarch/ganeti-2.4.5-2.el5.noarch.rpm wget http://jfut.integ.jp/linux/ganeti/noarch/ganeti-instance-debootstrap-0.7-2.el5.noarch.rpm rpm -ivh ganeti-2.4.5-2.el5.noarch.rpm ganeti-instance-debootstrap-0.7-2.el5.noarch.rpm Upgrade notes +++++++++++++ **Mandatory** on all nodes. /etc/init.d/ganeti stop tar czf /var/lib/ganeti-$(date +%FT%T).tar.gz -C /var/lib ganeti Install new Ganeti version on all nodes. **Mandatory** on master node. /usr/lib/ganeti/tools/cfgupgrade --verbose --dry-run /usr/lib/ganeti/tools/cfgupgrade --verbose This script upgrade the configuration files(/var/lib/ganeti). /etc/init.d/ganeti start gnt-cluster redist-conf /etc/init.d/ganeti restart gnt-cluster verify Initializing the cluster ++++++++++++++++++++++++ **Mandatory** on one node per cluster. gnt-cluster init --vg-name --master-netdev --nic-parameters link= .. admonition:: KVM gnt-cluster init --vg-name vmvg --master-netdev --enabled-hypervisors kvm --nic-parameters link= gcluster ex) gnt-cluster init --vg-name vmvg --master-netdev eth0 --enabled-hypervisors kvm --nic-parameters link=br0 gcluster .. admonition:: Xen gnt-cluster init --vg-name vmvg --master-netdev --nic-parameters link= gcluster ex) gnt-cluster init --vg-name vmvg --master-netdev eth0 --nic-parameters link=xenbr0 gcluster Enable use_bootloader for using VM's boot loader. .. admonition:: KVM on CentOS/Scientific Linux gnt-cluster modify --hypervisor-parameters kvm:kernel_path= .. admonition:: Xen on CentOS/Scientific Linux gnt-cluster modify --hypervisor-parameters xen-pvm:use_bootloader=True Joining the nodes to the cluster ++++++++++++++++++++++++++++++++ **Mandatory** for all the other nodes. gnt-node add gnt-node add node2 Setting up and managing virtual instances ----------------------------------------- Setting up virtual instances ++++++++++++++++++++++++++++ Setting up Debian lenny gnt-instance add -t drbd -n node1:node2 -o debootstrap --disk 0:size=1G -B memory=256 instance1 Setting up CentOS/Scientific Linux I recommend to use Ganeti Instance Image. http://code.osuosl.org/projects/ganeti-image/ ++++++++++++++++++ Jun Futagawa jfut (_at_) integ.jp