How I do vm management using Puppet, KVM, and GlusterFS on RHEL/CentOS

A GlusterFS user from IRC asked me about my puppet management of KVM in RHEL/CentOS and how it works. I started to write this post two weeks ago and had to stop because although it works great, I figured that wasn’t the answer he was looking for. I looked at the puppet module that I knew I’d hacked on from Carla Souza and after 2 hours of trying to figure out what I had done, gave up and went to bed. I finally was able to look at it again and it wasn’t quite as hacked up as I thought I remembered. I’ve since forked hers and added my changes.

Carla made a really nice module. It’s able to create the vm in several ways and ensure it’s up and running. There were only a few difficiencies with regard to EL that I had to address, and some missing features from newer versions of libvirt. I also added a define to wrap around the class instance so I could more easily set all my defaults.

My node only has this little snippet to define a VM.

  virt::kvm { "blog":
    desc      => "blog.edwyse.com",
    eth1_addr => "10.253.0.233",
    ensure    => running,
 }

That’s all wrapped in a define that sets all my standard settings,

define virt::kvm($desc,$eth1_addr,$memory = 1024, $targetmem = 524288, $disk_size = 6, $ensure = running) {
  $hostname = $name

  virt { "$hostname":
    desc        =>  "$desc",
    memory      =>  $memory,
    cpus        =>  4,
    graphics    =>  "disable",
    clocksync   =>  "UTC",
    kickstart   =>  "http://puppet/centos.ks.php?hostname=$hostname&eth1_addr=$eth1_addr ksdevice=eth0 console=ttyS0,115200",
    boot_location => "/mnt/gluster/centos/6/os/x86_64",
    virt_path   =>  "/mnt/gluster/vmimages/$hostname.img",
    disk_size   =>  $disk_size,
    disk_cache  =>  "writethrough",
    os_variant  =>  "virtio26",
    virt_type   =>  "kvm",
    interfaces  =>  ["br0", "br1"],
    autoboot    =>  true,
    ensure      =>  $ensure,
    on_poweroff =>  "destroy",
    on_reboot   =>  "restart",
    on_crash    =>  "restart",
  }

  exec { "/usr/bin/virsh setmem $hostname $targetmem": 
  unless  => "/usr/bin/virsh dommemstat $hostname | /bin/grep $targetmem",
  returns => [0,1] 
  }
}

As you can see there, I mount GlusterFS volumes in /mnt/gluster. My images are in a volume called vmimages, and I have a CentOS mirror on another volume. I have two bridged network connections, br0 and br1 which map to eth0 and eth1 in the VM. eth1 is defined statically, eth0 has a dhcp server assign it’s address.

Once the image is created and the instance boots for the first time, it reads it’s kickstart file from the web server on the puppet master.

<?php 

$hostname = $_GET['hostname'];
$eth1_addr = $_GET['eth1_addr'];

echo "
install
firewall --enabled --ssh
url --url=http://puppet/centos/6/os/x86_64
rootpw --iscrypted \$1\$xxxxx
user --name=myuser --iscrypted --password='\$1\$yyyyy''
network --device eth0 --bootproto dhcp --hostname $hostname --noipv6 --onboot=on
network --device eth1 --bootproto static --ip $eth1_addr --netmask 255.255.255.0 --onboot=on
auth  --useshadow  --enablemd5
text
keyboard us
lang en_US
selinux --disabled
skipx
logging --level=info
reboot
timezone --utc America/Los_Angeles
bootloader --location=mbr --append='console=ttyS0'
zerombr
clearpart --all --initlabel
part /boot --asprimary --fstype='ext4' --size=256
part /var/log --fstype='ext4' --size=512
part / --fstype='ext4' --grow --size=1
part swap --recommended
firstboot --disable
repo --name='Extra Packages for Enterprise Linux 6 - x86_64' --baseurl='http://download.fedoraproject.org/pub/epel/6/x86_64'
repo --name='Puppet for EL 6 - x86_64' --baseurl='http://yum.puppetlabs.com/el/6/products/x86_64'
services --enabled puppet,network
services --disabled cups,smartd,autofs,bluetooth,ip6tables,smartd,yum-updatesd,avahi-daemon,NetworkManager
%packages
@core
--nobase
puppet
facter
vim-enhanced
exim
-sendmail
-yum-updatesd
-cups
-NetworkManager
%post
(
echo 'S0:12345:respawn:/sbin/agetty ttyS0 115200' >> /etc/inittab
echo 'ttyS0' >> /etc/securetty
echo 'IPV6INIT=no' >> /etc/sysconfig/network
echo 'DHCP_HOSTNAME=$hostname' >> /etc/sysconfig/network
sed -i '/^NM_CONTROLLED/d' /etc/sysconfig/network-scripts/ifcfg-eth*
sed -i '/^DHCP_HOSTNAME/d' /etc/sysconfig/network-scripts/ifcfg-eth*
echo 'install ipv6 /bin/true' >> /etc/modprobe.d/noipv6.conf
) 1>/root/post_install.log 2>&1
"
?>

This creates an install that runs puppet as soon as it reboots, finalizing all it’s configuration.

There are several things that could be improved, like it would be nice to create an exported resource with the VMs on any one node and if that VM is moved to another node’s manifest, it could migrate itself. This is also, obviously, not the best method for everyone, but hopefully this is helpful.