Thursday, July 4, 2013

Linux OVS & VxLAN based virtual networks

OVS ( is the Openflow switch implementation in Linux. It implements various OF versions including 1.3.   OVS has support to realize virtual networks using VLAN and GRE for a long time.  In recent past,   OVS was enhanced to support overlay based virtual networks.  In this post, I give some commands that can be used to realize virtual networks using VxLAN.

For more information about VxLAN,  please see this tutorial.

Recently, there was a very good development in OVS on overlays.  It is no longer required to have as many 'vports' as number of compute servers to realize a virtual network across multiple compute servers.  OVS now implements the concept of flow based overlay protocol values selection.  Due to this, one VxLAN port is good enough in OVS OF switch irrespective number of remote compute nodes and irrespective of number of virtual networks.

OVS introduced new extensions (an action and set of OXM fields that can be set using set_field action)  to Openflow protocol where OF controller specifies the flow with tunnel/overlay specific information.

VxLAN protocol layer adds overlay header and it needs following information - Source IP address and Destination IP address of outer IP header,  source port and destination ports of UDP header and VNI for VxLAN header.  OVS provides facilities for Openflow controller to set the source IP, destination IP and VNI using set_field action.  OVS introduced following NXM fields

NXM_NX_TUN_ID :  To specify VNI (VxLAN Network Identifier).
NXM_NX_TUN_IPV4_SRC :  To specify source IP of the outer IP header.
NXM_NX_TUN_IPV4_DST :  To specify the destination IP of the outer IP header.

VxLAN protocol layer knows the UDP destination port from the 'vport'.   ovs-vsctl command can be used to create VxLAN ports.  ovs-vsctl command can be used to create many VxLAN ports on the same VNI with different destination port on each one of them.  VxLAN protocol layer gets rest of information required to frame outer IP, UDP headers by itself and with the help of Linux TCP/IP stack.

Similarly,  VxLAN protocol layer informs the OVS OF switches by filling up above fields after decapsulating the packets.  Due to this,  Openflow controller can use above fields as match fields.

Essentially, OVS provided mechansims to set the tunnel field values for outgoing packets in the Openflow flows and also provided mechanisms to use these tunnel fields as match fields in OF tables for incoming packets.

Following commands can be used to create VxLAN ports using 'ovs-vsctl' without explicitly mentioning the tunnel destination and tunnel ID, letting Openflow controller to specify these field values in OF flows.

Creation of VxLAN port with default UDP service port:

  ovs-vsctl add-port br-tun vxlan0 -- set Interface vxlan0   type=vxlan  options:remote_ip=flow options:key=flow

Above command is used to create VxLAN port 'vxlan0' on OF switch 'br-tun' and specifying this port to get the tunnel ID (VNI)  and tunnel remote IP from the OF flow.  "key=flow" is meant to get the tunnel ID from the flow and "remote_ip=flow" is meant to get the tunnel destination IP address from the flow.

Small variation of above command to create the VxLAN port with different UDP destination port, 5000.

ovs-vsctl add-port br-tun vxlan1 -- set Interface vxlan1 type=vxlan options:remote_ip=flow options:key=flow options:dst_port=5000

OVS provides a mechanism to create Openflow flows without having to have external Openflow controller.  'ovs-ofctl' is the machanism provided by OVS to do this.

Following command can be used to create the
ovs-ofctl add-flow br-tun "in_port=LOCAL actions=set_tunnel:1,set_field:>tun_dst,output:1"  (OR)
ovs-ofctl add-flow br-tun "in_port=LOCAL actions=set_field:>tun_dst, set_field:1->tun_id, output:1
"set_tunnel" is used to specify the VNI.  "set_field" to specify the tunnel destination.

Other commands of interest are:

To see the openflow port numbers:
        ovs-ofctl show br-tun
To dump flows:
        ovs-ofctl dump-flows br-tun

1 comment:

Anonymous said...

Hi i am trying to follow the instructions but getting the following message:

ovs-ofctl :tun_dst is not allowed to set.

Can you please help?