Xen bridges and bonding

From MyWiki

NOTE: the below configuration files are valid RHEL and probably RedHat derived distributions. It would be different way of doing the same thing on, say, Debian (you'd need to use /etc/network/interfaces file to do all the configuration), but the idea is the same.

The idea was to make use of bonding and VLAN support in my Xen configuration. So, the bonding will allow me to increase availability of my network connection and will carry multiple VLANs traffic inside.

Here is the xend configuration file:

[root@dom0-04 ~]# cat /etc/xen/xend-config.sxp
(xend-unix-server yes)
(xend-unix-path /var/lib/xend/xend-socket)
(xend-relocation-hosts-allow '^localhost$ ^localhost\\.localdomain$')
(network-script /bin/true)
(dom0-min-mem 1024)
(dom0-cpus 0)

I should probably change the line (dom0-cpus 0) to (dom0-cpus 1) later to allow more CPU time to dom0. Anyway, the key line above is:

(network-script /bin/true)

I don't want Xen to use its default network-bridge, as I will craft bridges myself.

To make use of bonding, you need to enslave your ethX devices. Here's the way I did it (you do the same way for all ethX devices you want to bond together):

[root@dom0-04 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
MASTER=bond0
SLAVE=yes

Then I created bond0 device itself:

[root@dom0-04 ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
BOOTPROTO=static

Notice, that I didn't assign IP address to it. It's a good thing to keep your dom0 network seporate and safe. So, I'd advise you to bond and bridge over interfaces that you don't use to access dom0 itself. Dedicate a separate interface (or a set of interfaces bonded together) to manage your dom0.

So, back to our configuration. I want to make use of VLAN 107 for my domU. So, I need to configure my Cisco first to allow for bonding and multiple VLANs inside. Cisco calls bonding EtherChannel and the thing you need to configure called trunk. Here is a sample one:

interface GigabitEthernet1/1
 description VLAN Trunk
 switchport
 switchport trunk native vlan 2
 switchport trunk allowed vlan 1,2,100-200
 switchport mode trunk
 no ip address
 logging event link-status
 load-interval 60
 udld port aggressive

Now, let's configure a bridge on top of our bond0. Here is sample of bridge to VLAN 107. I called the device br107:

[root@dom0-04 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br107
DEVICE=br107
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
DELAY=0
STP=off

And finally, we need to make the bridge device available to domU. The following line in domU's configuration file will provide the device to our domU:

vif = [ 'bridge=br107' ]

Inside domU it will show up as eth0.

That's pretty much it.