Vagrant and Libvirt

Getting Started

I create a lot of disposable VMs and finally took the leap to using Vagrant to try to streamline some of the infrastructure work. My server setup, however is pretty complicated and uses libvirt with some passthrough devices, etc, etc. Anyway, using Virtualbox wasn’t an option unfortunately. This post is more for me than for anyone else. I wanted to jot down my experiences so that I can review in the future if necessary.

I started with a working libvirt setup on an 8 core 24 GB AMD machine running Fedora 22 that uses a bridged device ‘br0’ to connect all my VMs to the network. More on this later.

Much of my info came from https://groups.google.com/forum/#!topic/vagrant-up/lysSs5DHAYA

yum install vagrant vagrant-libvirt

I also needed:
libvirt-devel
ruby-devel
gcc-c++

and the following plugins:
vagrant plugin install vagrant-libvirt

some issue I had was fixed with downgrading fog-libvirt
vagrant plugin install –plugin-version 0.0.3 fog-libvirt

# other great options:
vagrant plugin install vagrant-rekey-ssh
vagrant plugin install sahara
vagrant plugin install vagrant-mutate
vagrant plugin install vagrant-triggers

#Add some useful functions found @https://github.com/maxamillion/vagrant-libvirt-rpm/blob/master/.bashrc_vagrant.sh
wget https://gist.githubusercontent.com/purpleidea/8071962/raw/ee27c56e66aafdcb9fd9760f123e7eda51a6a51e/.bashrc_vagrant.sh
echo ‘. ~/.bashrc_vagrant.sh’ >> ~/.bashrc
. ~/.bashrc

Once I had the basics running I wanted to create a base box with RHEL6.7 to suit my specific needs:

To begin I started with a manual minimal install of RHEL6.7 from the minimal ISO using the libvirt GUI. Root password was set to ‘vagrant’ during install. Once this was complete I added the following:

yum update
yum install rsync wget nc traceroute telnet
useradd vagrant
echo “vagrant” | passwd –stdin vagrant
mkdir ~vagrant/.ssh
chmod 700 ~vagrant/.ssh
wget –no-check-certificate https://raw.githubusercontent.com/mitchellh/vagrant/master/keys/vagrant.pub -O /home/vagrant/.ssh/authorized_keys
chmod 600 ~vagrant/.ssh/authorized_keys
chown -R vagrant:vagrant ~vagrant/.ssh

Modify sudoers file and comment out “Defaults requiretty”
Add “vagrant ALL=(ALL) NOPASSWD:ALL” to end of sudoers file

# Definitely the MOST important part if you want networking to work correctly on RHEL6
# Leave only eth0 config – the rest will be managed by vagrant
# Removing udev rules is required if you want this to work. Just removing the rules from the existing files did not work for me.
rm -f /etc/udev/rules.d/70-persistent-net.rules
rm -f /etc/sysconfig/network-scripts/ifcfg-eth{0,1,2,3}

# Make sure there are no references to a specific HDADDR or UUID in the ifcfg script
create /etc/sysconfig/network-scripts/ifcfg-eth0 with the following contents:
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=dhcp

#shutdown RHEL box
poweroff

#Copy the qcow2 to a new folder under the name box.img

mkdir ~/newbox
cd ~/newbox
cp /var/lib/libvirt/images/image_file_of_vm.qcow2 ~/newbox/box.img

#Create Vagrantfile in ~/newbox

Vagrant.configure("2") do |config|
config.vm.box = "vagrant-centos66-base"
end

#Create metadata.json in ~/newbox

{
"provider": "libvirt",
"format": "qcow2",
"virtual_size": 10
}

tar cvzf centos67.box ./metadata.json ./Vagrantfile ./box.img

vagrant box add base-centos67 centos67.box

# you don’t want to use ~/newbox as your rsync directory, make a new one.
mkdir ~/vagrant
cd ~/vagrant
vagrant init base-centos66
vagrant up
vagrant destroy

# IMPORTANT: always refresh virsh pool after a destroy, clears up a lot of potential problems.
virsh pool-refresh default

# Remove a libvirt box (example: testbox)
vagrant box remove base-centos67
rm /var/lib/libvirt/images/base-centos67_vagrant_box_image_0.img

# IMPORTANT: always refresh virsh pool after a destroy, clears up a lot of potential problems.
virsh pool-refresh default
See: https://github.com/pradels/vagrant-libvirt/issues/85

Networking out of the box did not provide outbound access. The internal bridge that vagrant creates and uses allows VM to VM communication but not outbound. This may be due to my lack of understanding but I wasn’t able to get it working yesterday. Instead I added a “public_network” to my Vagrantfile and used my ‘br0’ bridge. This gives me an eth1 interface in the VM that is bridged to my local network and gets an address via DHCP from my network.

#Useful Vagrantfile info: Setting up a publicly accessible server using kvm interface br0
Vagrant.configure(“2”) do |config|

# Assumes you are using dhcp,
Vagrant.configure(“2”) do |config|
config.vm.define :default do |default|
# Every Vagrant development environment requires a box. You can search for
# boxes at https://atlas.hashicorp.com/search.
config.vm.box = “base-centos67”

# Create a public network, which generally matched to bridged network.
# Bridged networks make the machine appear as another physical device on
# your network.
config.vm.network “public_network”, :dev => ‘br0’, :type => ‘bridge’
config.vm.synced_folder ‘./’, ‘/vagrant’, type: ‘rsync’
end
end

Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmailby feather