VxLAN & Openflow Controller Role

 

Background

In one of previous posts here,  I argued that there is no need for Openflow controller to realize virtual networks.  In the same post,  I also mentioned that Openflow controller is required if intelligent traffic redirection is required to realize features such as network service chaining and advanced traffic visualization.  These advanced features require to control the traffic path on per flow basis (5-tuple based connections) across physical/virtual appliances.

VxLAN, by default, tries to discover the remote endpoint  using discovery mechanism. This discovery mechanism involves sending the packet using  multicast VxLAN encapsulation.   Compute nodes or end points that owns the DMAC address of the inner packet expects to consume the packet.  Using learning mechanisms,  each VxLAN end point creates VTEP entries.  Once VTEPs are learnt,  packets are sent only to the intended endpoint.  Please see this VxLAN tutorial to understand the VxLAN functionality. 

As discussed in the post about OVS & VxLAN based networks, OVS supports flow based overlay parameters such as remote endpoint address,  VNI etc.. 

Use cases requiring control from Openflow controller

One use case as briefly touched upon is the flow-control (FC) which is a some central entity across Openflow virtual switches controlling the traffic path, possibly on per connection basis. FC functionality running on top of OF controller, gets the packet-in (miss packet) from the OVS,  decides the next destination (an appliance implementing a network service) and then programs the flow in OF switch with appropriate overlay parameters.

Another use case of OF controller, in realizing VxLAN based virtual networking support, is send/receive of multicast & broadcast packets.  VxLAN specifications support multicast transport  for transporting inner mutlticast and broadcast packets.  As discussed above,  FC functionality is expected to get hold of any inner flows whether they are unicast based, multicast based or broadcast based.   Also,  I hear that many times some network operators don't allow multicast packets in their network infrastructure, but at the same time, they don't like to stop VMs using Multicast based protocols. OF Controller  can provide a feature to duplicate the broadcast and multicast inner packets as many times as number of possible destinations of the virtual network and send them over VxLAN using unicast VxLAN encapsulation.

Support available in Cloud Orchestration tools (Openstack)

Fortunately, in cloud computing world, Openstack already maintains inventory information such as

• Physical servers and their IP addresses.
• VMs on each physical server and MAC addresses of the VM Ethernet Ports.
• Virtual networks (VLAN based, VxLAN based or other overlay based).
• Physical servers that are participating in a given virtual network.
• Ports on OVS that are connected to the VMs and which virtual network they belong to.

Openstack provides APIs to get  hold of above information.  Also. Openstack has ability to notify interested parties when the repository changes.  Openflow controller can have a service called 'Cloud Resource Discovery' whose functionality is to keep the cloud resource repository in OF controller and make it available to OF controller applications such as FC. 

• When it comes up, it can go and read above information and keeps the repository with it. Also, it can register to get the notifications.  
• Upon any notification,  update local repository information. 

Note that there could be multiple OF controllers working in a cluster to take care of the load among them.  Since Openstack is one entity across OF controllers, responsibility of ensuring that the cloud repository is consistent across OF controllers is with the Openstack entity.

Details of OF Controller Role

Now, let us discuss the capabilities required in OF controllers to realize VxLAN based virtual networks to take of two functions - Enable FC & Work with Multicast/Broadcast packets.  In this,  I am assuming that OF controller is being used along side with Openstack. I am also assuming that VMs are the ones which are generating the traffic destined for another VM.  OVS is used to implement OF virtual switches and VxLAN overlays.  Essentially, OVS provides virtual switch where VMs are connected to on its north side and VxLAN ports which are connected to it on south side.

OF Controller predominantly need to have three main components;

• Cloud Resource Discovery (CRD) component
• Flow Control (FC) component
• OF Controller Transport (OCT) component (such as Opendaylight,  Freescale Openflow Controller Transport,  NOX Controller etc..).

Cloud Discovery component: 

Discovers following from Openstack

•  Virtual Networks configured
•  Physical Servers (OF Capable switches)
•  OF Logical switches within each OF Capable switch (For example, in Openstack world,  there are always to OF logical switch br-int and br-tun).
•  Names of ports attached to the OF logical switches
• Qualification of Ports (Network Port,  Port that is connected to VM  or patch port and the virtual network it corresponds to).
• Local VLAN IDs used in br-int of each OF Capable switch and their mapping to VxLAN network and dices.
• VMs - MAC Addresses and the corresponding physical servers.

Repositories:

OF Controller components typically maintain the repository, which is either populated by  CRD.  Many OCT do have repositories.  For example,  Freescale OCT has capability to store via its DPRM module

• OF Capable switches
• OF logical switches in each OF capable switch
• Containers (Domains)
• OF Logical switches that belong to domains.
• Port names associated with OF logical switch.
• Table names associated with each domain.
• It also has ability to put various attributes to each of above elements for extensibility.

Additional repositories are required : 

• Virtual Networks.  For each virtual network
• Fields:
• Virtual Network Name
• Virtual Network Description.
• Type of virtual network
• In case of VLAN:
• VLAN ID

• In case of VxLAN
• VNI

• Attributes
• To store the Openstack UUID.
• List of references to OF Capable switches.
• In each reference  node
• Array of OF Logical switch references. (Example:  br-int, br-tun)
• For br-int logical switch
• List of references of Ports that are connected towards VMs.
• For each reference,  also  store the reference to VM.
• List of references of ports that are connected to br-tun.
• For  br-tun logical switch
• List of references of ports that are connected to br-int
• List of references of ports that are connected to network
• List of references to OF VMs
• List of references to Ports that are 
• Views:
• Based on name
• Based on Openstack UUID

•  VM Repository - Each record consists of
• Fields:
• VM Name
• VM Description
• Attributes;
• Openstack VM UUID
• Kind of VM -  Normal Application VM or  NS VM.
• List of references of ports connected towards this VM  on br-int (MAC address is supposed to be part of the Port repository in DPRM).  Fore each reference
• VLAN associated with it (in br-int)
• Reference to Virtual Network.
• Reference to physical server (OF Capable switch)
• Views
• Based on VM name
• Based on VM UUID

•  Cell Repository
•  Availability Zone repository
•  Host Aggregate  repository

Existing repositories would be updated with following:

•  Port repository in each OF logical switch: Each port entry is updated with using attributes
•  Port Qualification attribute
• VM Port or Network Port or Patch Port.
• MAC Address attribute 
• MAC Address (in case this is connected to VM port)
• VM Reference attribute:
• Reference to VM record.
•  Virtual Network Reference attribute
• Reference to Virtual network record.
• OF Switch Repository :  I am not sure whether this is useful anywhere, but it is good to have following attributes 
• VM Reference attribute
• List of references to VMs that are being hosted by this switch.
• VN References attribute:
• List of references to VNs for which this switch is part of.
• Availability Zone Reference attribute
• Reference to  availability Zone record
• Host Aggregate Zone Reference Attribute
• Reference Host Aggregate Zone record.
• Cell Reference Attribute
• Reference to Cell record 

Flow Control

This component is the one which creates flows based on the information available in the repositories.  Assuming the flow granularity is at the connection level,  first packet from a VM of any connection results into a packet-miss (packet-in) in br-int.  OF controller Transport receives it from the OF switch and gives it over to the FC.  FC application knows the DPID of the switch,  in_port of the packet. From this information,  port repository is checked and from the port repository it finds the virtual network information such as local VLAN ID used in br-int and VxLAN VNI. It also finds out the remote IP address of the overlay endpoint based on VM MAC (DMAC of the packet).

Assuming that VMs communicating are in different compute nodes,  there are total 8 flows the FC module would need to create to let rest of packets of the connection go through between two VMs.
Any connection contains client-to-server side and server-to-client side.  There are two OF logical switches in each compute node.  Since, there are two compute nodes, total flows would be 2 (Due to Client to Server AND Server to Client) * 2 (Due to two Openflow switches) * 2 (because of two compute nodes).

Example:
Let us assume that compute nodes, node1 & node2, have outIP1  and outIP2 IP addresses.
Let us also assume that VM1 on Node1 is making HTTP connection (TCP, source port 30000,  destination port 80) with VM2 on Node2.  VM1 IP address is inIP1 and VM2 IP address InIP2.   Let us also assume that VxLAN  uses port number 5000.

When VxLAN based network is created,  let us assume that OVS agent created local VLAN 100 on node1 and VLAN 101 on Node2 for VxLAN network whose VNI is 2000.

FC establishes the following flows:

• Node1
• BR-INT:
• Client to Server flow
• Match fields
• Input Port ID:  < Port ID to which VM is connected to>
• Source IP;  inIP1
• Destination IP:  inIP2
• Protocol:  TCP
• source port : 30000
• destination port: 80
• Actions:
• Add VLAN tag 100 to the packet.
• Output port :  Patch Port that is connected between br-int and br-tun.
• Server to Client flow
• Match fields:
• Input Port ID: 
• VLAN ID: 100
• Source IP: inIP2
• Destination IP: inIP1
• Protocol:  TCP
• Source port: 80
• Destination Port: 30000
• Actions
• Remove VLAN tag 100
• Output port
•  
• BR-TUN
• Client to Server flow
• Match fields:
• Input Port ID:  Patch Port 
• VLAN ID:  100
• Source IP;  inIP1
• Destination IP:  inIP2
• Protocol:  TCP
• source port : 30000
• destination port: 80
• Actions
• Set Field:
• Tunnel ID: 2000
• Tunnel IP : outIP2
• Remove VLAN tag 100
• Output Port: 
• Server to Client flow:
• Match fields:
• Input Port ID: 
• Tunnel ID:  2000
• Source IP: inIP2
• Destination IP: inIP1
• Protocol:  TCP
• Source port: 80
• Destination Port: 30000
• Actions
•  Add VLAN tag 100
• Output port: < br-tun end of patch port pair>
• Node 2:
• BR-INT:
• Server to Client flow:
• Match fields
• Input Port ID:  < Port ID to which VM is connected to>
• Source IP;  inIP2
• Destination IP:  inIP1
• Protocol:  TCP
• source port :80
• destination port: 30000
• Actions:
• Add VLAN tag 101 to the packet.
• Output port :  .
• Client to Server flow
• Match fields
• Input Port ID:
• VLAN ID: 101
• Source IP: inIP1
• Destination IP: inIP2
• Protocol:  TCP
• source port: 30000
• destination port: 80
• Actions:
• Remove VLAN tag 101
• Output port: 
• BR-TUN :
• Client to Server flow:
• Match fields:
• Input Port ID: 
• Tunnel ID:  2000
• Source IP;  inIP1
• Destination IP:  inIP2
• Protocol:  TCP
• source port : 30000
• destination port: 80
•  Actions
• Add VLAN tag 101
• Output port:
• Server to Client flow:
• Match fields:
• Input port ID:
• VLAN ID: 101
• Source IP;  inIP2
• Destination IP:  inIP1
• Protocol:  TCP
• source port :80
• destination port: 30000
• Actions:
• Set field:
• Tunnel ID: 2000
• Remote IP :  outIP1
•  Remove vlan 101.
•  Output port:

In case of broadcast and multicast inner packets,  FC could use OF 1.3 groups with 'ALL' type with as many buckets as number of destinations.  Each bucket must have action with separate set of 'set-field' actions.

Assume that FC sees a Multicast flow from VM1 in Node1 (outIP1) with DIP: inMIP, UDP protocol and Destination Port : 2222 and Source port 40000.  Assuming that it should go to compute nodes Node2 (outIP2), Node3 (outIP3) and Node4 (outIP4) as there are VMs that are interested in these multicast packets. Then FC would generate following flows in Node1 and Node2.  Flows in Node3 and Node4 look similar.

• Node 1:
• br-int
• Match fields
• Input port: 
• Source IP :  inIP1
• Destination IP:  inMIP
• Protocol: UDP
• Source port: 40000
• Destination POrt:  2222
• Actions
• Add VLAN tag : 100
• Output port: 
•  br-tun
• Group object (G1):
• Type: ALL
• Bucket 1: 
• Set-fields:
• Tunnel ID: 2000
• Remote IP: outIP2
• Remove VLAN tag 100
• Output port:
• Bucket 2
• Set-fields:
• Tunnel ID: 2000
• Remote IP: outIP3
• Remove VLAN tag 100
• Output port:
• Bucket 3
• Set-fields:
• Tunnel ID: 2000
• Remote IP: outIP4
• Remove VLAN tag 100
• Output port:
• Match fields:
• Input port :
• VLAN ID: 100
• Source IP :  inIP1
• Destination IP:  inMIP
• Protocol: UDP
• Source port: 40000
• Destination POrt:  2222
• Actions
• Group Object: G1
•  Node 2
• br-tun
• Match fields:
• Input port:
• Tunnel ID: 2000
• Source IP :  inIP1
• Destination IP:  inMIP
• Protocol: UDP
• Source port: 40000
• Destination Port:  2222
• Actions:
• Push VLAN tag 101
• Output port:
• br-int
• Match fields:
• Input port:
• VLAN ID: 101
• Source IP :  inIP1
• Destination IP:  inMIP
• Protocol: UDP
• Source port: 40000
• Destination Port:  2222
• Actions
• Remove VLAN tag 101
• Output port:

FC would use repository information to create above flows.  Hence it is important to have the repository information arranged in a good data structure to get information upon packet-miss.

Evey flow that is created by FC in OF logical switches, FC maintains the flows locally too.  This is to enable cases where OF switches evict the flows.  When there is a packet-miss due to this,  FC would push the locally available flow into the OF switch.  That is, whenever there is packet-in,  FC first needs to check its local run time flow store before creating new flows by referring to repository.

Few more considerations that FC and CRD components that need to take care of are;

• VM Movement:  When VM is moved,  flows created in the OVS OF switches also should be moved accordingly.  CRD component is expected to listen for VM movement events from Openstack and internally update the repositories.  FC component, in turn, should update the OF flows accordingly - Removing flows from old compute node and put them in new compute node.

Linux OVS & VxLAN based virtual networks

OVS (www.openvswitch.org) is the Openflow switch implementation in Linux. It implements various OF versions including 1.3.   OVS has support to realize virtual networks using VLAN and GRE for a long time.  In recent past,   OVS was enhanced to support overlay based virtual networks.  In this post, I give some commands that can be used to realize virtual networks using VxLAN.

For more information about VxLAN,  please see this tutorial.

Recently, there was a very good development in OVS on overlays.  It is no longer required to have as many 'vports' as number of compute servers to realize a virtual network across multiple compute servers.  OVS now implements the concept of flow based overlay protocol values selection.  Due to this, one VxLAN port is good enough in OVS OF switch irrespective number of remote compute nodes and irrespective of number of virtual networks.

OVS introduced new extensions (an action and set of OXM fields that can be set using set_field action)  to Openflow protocol where OF controller specifies the flow with tunnel/overlay specific information.

VxLAN protocol layer adds overlay header and it needs following information - Source IP address and Destination IP address of outer IP header,  source port and destination ports of UDP header and VNI for VxLAN header.  OVS provides facilities for Openflow controller to set the source IP, destination IP and VNI using set_field action.  OVS introduced following NXM fields

NXM_NX_TUN_ID :  To specify VNI (VxLAN Network Identifier).
NXM_NX_TUN_IPV4_SRC :  To specify source IP of the outer IP header.
NXM_NX_TUN_IPV4_DST :  To specify the destination IP of the outer IP header.

VxLAN protocol layer knows the UDP destination port from the 'vport'.   ovs-vsctl command can be used to create VxLAN ports.  ovs-vsctl command can be used to create many VxLAN ports on the same VNI with different destination port on each one of them.  VxLAN protocol layer gets rest of information required to frame outer IP, UDP headers by itself and with the help of Linux TCP/IP stack.

Similarly,  VxLAN protocol layer informs the OVS OF switches by filling up above fields after decapsulating the packets.  Due to this,  Openflow controller can use above fields as match fields.

Essentially, OVS provided mechansims to set the tunnel field values for outgoing packets in the Openflow flows and also provided mechanisms to use these tunnel fields as match fields in OF tables for incoming packets.

Following commands can be used to create VxLAN ports using 'ovs-vsctl' without explicitly mentioning the tunnel destination and tunnel ID, letting Openflow controller to specify these field values in OF flows.

Creation of VxLAN port with default UDP service port:

  ovs-vsctl add-port br-tun vxlan0 -- set Interface vxlan0   type=vxlan  options:remote_ip=flow options:key=flow

Above command is used to create VxLAN port 'vxlan0' on OF switch 'br-tun' and specifying this port to get the tunnel ID (VNI)  and tunnel remote IP from the OF flow.  "key=flow" is meant to get the tunnel ID from the flow and "remote_ip=flow" is meant to get the tunnel destination IP address from the flow.

Small variation of above command to create the VxLAN port with different UDP destination port, 5000.

ovs-vsctl add-port br-tun vxlan1 -- set Interface vxlan1 type=vxlan options:remote_ip=flow options:key=flow options:dst_port=5000

OVS provides a mechanism to create Openflow flows without having to have external Openflow controller.  'ovs-ofctl' is the machanism provided by OVS to do this.

Following command can be used to create the
ovs-ofctl add-flow br-tun "in_port=LOCAL actions=set_tunnel:1,set_field:172.16.2.3->tun_dst,output:1"  (OR)
ovs-ofctl add-flow br-tun "in_port=LOCAL actions=set_field:172.16.2.3->tun_dst, set_field:1->tun_id, output:1

"set_tunnel" is used to specify the VNI.  "set_field" to specify the tunnel destination.

Other commands of interest are:

To see the openflow port numbers:
        ovs-ofctl show br-tun
To dump flows:
        ovs-ofctl dump-flows br-tun

L4 Switching & IPSec Requirements & Openflow extensions

 

L4 Switching - Introduction

L4 switching typically involves connection tracking,  NAT and common attack checks.  Stateful inspection firewalls,  NAT, URL filtering  and SLB (Server Load Balancing) are some of the middle-box functions that take advantage of L4 switching.  These middle-box functions inspect first few packets of every connection (TCP/UDP/ICMP etc..) and offload the connection processing to some fast path entity to do L4 switching.  Normally both fastpath and normal path functionality reside in the same box/device.

Openflow, promoted by industry big names,  is one of the protocols that separates control plane from data plane where control planes of multiple switches implemented in one logical central controller and leaving the data plane alone at devices, which are programmable to work in a specified way,  thereby making devices simple.

Middle-box functions described above can also be separated - Normal path (service plane) and fast path (L4 switchiing).  Normal path works on first packet or at the most first few packets and fast path works on the connection for rest of the packets in the connection. By implementing normal path (Service plane) in centralized logical controllers and leaving the fastpath (L4 switching) at the physical/logical device level,  similar benefits of CP/DP separation can be achieved.  Benefits include

• Programmable devices where device personality can be changed by controller applications.  Now,  Openflow switch can be made as L2 switch, L3 switch and/or L4 switch.
• By removing major software logic (in this case, it is service plane) from device to the central location, cost of ownership goes down for end customers.
• By centralizing the service plane,  operation efficiency can be improved significantly
• Ease-of software upgrades
• Configure/Manage from a central location.
• Granular flow control.
• Visibility across all devices.
• Comprehensive traffic visualization

L4 Switching - Use cases:

• Branch office connectivity with corporate head quarters :   Instead of having firewall, URL filtering and Policy control on every branch office device,  it can be centralized at corporate office. First few packets of every connection could go to main office.  Main office controller decides on the connection. If the connection is allowed,  it lets the openflow switch in the branch office to forward rest of the traffic on that connection.  This method only requires simple openflow switches in the branch office and all intelligence can be centralized at one location, that is, at main office. Thus, it reduces the need for skilled administrator at every branch office.
• Server Load Balancing in Data Centers:   Today data-centers have expensive server load balancers to distribute the incoming connections to multiple servers.  SLB devices are big boxes today for two reasons -  Normal path processing that selects the best server for every new connection and  fastpath packet processing are combined into one box/device.   By offloading packet processing to inexpensive openflow switches,  SLB devices only need to worry about normal path processing, which could be done in lesser expensive boxes and even using commodity PC hardware.  
• Managed Service providers providing URL filtering capability to home & SME offices :  Many homes today use PC based software to do URL filtering  to provide safe internet experience to kids.  SME administrators require URL filtering service to increase the productivity of their employees by preventing them to access recreational sites and also to prevent any malware contamination.  Again, instead of deploying URL filtering service at customer office premises,   service providers like to host this service centrally and manage centrally across multiple of their customers for operational efficiency.  Openflow controller implementing URL filtering service can get hold of packets until URL is fetched, find the category of URL, apply policy and decide on whether to continue the connection.  If the connection is to  be continued, then it can program the openflow switch in customer premises to forward rest of the packets.  
• Though URL filtering service is one use case given,  this concept of centralizing the intelligence at one central place and programming the openflow switches for fast path is equally applicable for "Session Border Controller" and "Application Detection using DPI" and other services. Whichever service needs to inspect first few packets of the connection is candidate for Service Plane/L4 Switching  separation using Openflow.

L4 Switching - Processing

L4 switching or fastpath entity are used to do connection level processing. Any connection consists of two flows -  Client-to-Server (C-S) flow and Server-to-Client (S-C) flow.  

Connections are typically 5-tuple based.  

Functionality of L4 switching:

• Connection Management :  
• Acts on the connection Add/Remove/Query/Modify requests from the service plane.
• Inactivity timeout :  Activity is observed on the connection. If no activity for some time (programmed during the connection entry creation),   connection entry is removed and notification is sent to the normal path.
• TCP Termination : L4 switching entity can remove the connection entry if it observes the both endpoints of TCP connections exchange FINs and ACKed. It can also remove the entry if TCP packet with RST is seen.
• Periodic notifications of the state collected to Service plane.
•  Packet processing logic involves
• Pre-flow lookup actions typically performed:
• Packet integrity checks :  Checksum verification of both IP and transport headers,  Ensuring that the length field values are consistent with packet sizes etc..
• IP Reassembly:  As the flows are identified by 5-tuples, it is required that packets are reassembled to determine the flow across fragments. 
• IP Reassembly related attack checks and remediation
• Ping-of-Death attack checks (IP fragment Overrun) - Checking for full IP packet never exceeds 64K.
• Normalization of the IP fragments to remove overlaps to protect from teardrop related vulnerabilities.
• Too many overlap fragments - Remove the IP reassembly context when too many overlaps observed.
• Too many IP fragments  & Very small IP fragment size:  Drop the reassembly context and associated IP fragments.
• Incomplete IP fragments resulting Denial of Service attacks - Limit the number of reassembly contexts on per IP address pair,  IP address etc..
• TCP Syn flood protection using TCP Syn Cookie mechanism
• Flow & Connection match:  Find the flow entry and corresponding connection entry.
• Post lookup actions
• Network Address Translation :  Translation of SIP, DIP, SP, DP.
• Sequence number translation in TCP connections :  This is normally needed when the service plane updates the TCP payload that decreases or increases the size or when the TCP syn cookie processing is applied on the TCP connection.
• Delta Sequence number updates in TCP connections :  This is normally required to ensure that right sequence number translation is applied, especially for retransmitted TCP packets.
• Sequence number attack checks :  Ensuring that sequence numbers of the packets going in one direction of the connection are within some particular range. This is to ensure that any attacker injecting the traffic is not honored.  This check is mainly required to stop TCP RST packets that are generated and sent by attackers.  As RST packet terminates the connection and thereby creating DoS attack, it is required that this check is done.
• Forwarding action :  To send the packet out by referring to routing tables and ARP tables.

IPSec - Introduction

IPSec is used to secure the IP packets. IP packets are encrypted to avoid leakage of data on the wire and also authenticated to ensure that the packets are indeed sent by the trusted party.  It runs in two modes - Tunnel mode and transport mode.   IP packets are encapsulated in another IP packet in tunnel mode.  No additional IP header in involved in transport mode. IPSec applies the security on per packet basis - Hence it is datagram oriented protocol.  Multiple encryption and authentication algorithms can be used to secure the packets.   Encryption keys and authentication keys are established on per tunnel basis by the Internet Key Exchange Protocol (IKE).   Essentially,  IPSec protocol suite contains IKE protocol and IPSec-PP (Packet processing).  

Traditionally,  implementations use proprietary mechanism between IKE and IPSec-PP as both of them sit in the same box.  New way of networking (SDN) centralizes control plane with distributed data paths.  IPSec is one good candidate for this.  When the control plane (IKE) is separated from the data path(IPSec-PP),   then there is a need for some standardization for communication between IKE and IPSec-PP.  Openflow is thought to be one of the protocols to separate out control plane and data plane.   But Openflow as defined in OF 1.3.x specification did not keep IPSec CP-DP separation in mind.  This blog post tries to provide extensions required in OF specification to enable IPsec CP-DP separation.

Openflow

ONF (Open Networking Foundataion) is standardizing Openflow protocol.  Controllers using OF protocol programs the OF switches with flows and instructions/actions to be performed on packets.  Openflow 1.3+ specification supports multiple tables which simplifies the controller programming as each table in the switch can be used for a purpose.

Contrary to what everybody says,  Openflow protocol is not really friendly let alone to SP/FP separation, but also to L3 CP/DP separation.  The way I look at OF today is that it is good for L2 CP/DP separation and also good for traffic steering kind of use cases, but it is not sufficient for L3 and L4 switching.  This post tries to give some of the features that are required in OF specifications to support L4 switching in particular and L3 switching to some extent.

Extensions Required to realize L4 switching & IPSec CP-DP separation

Generic extensions:

Table selection on packet-out message:

Problem Description
Current OF 1.3 specification does not give provision for controllers to start the processing of packet-out message from a specific table in the switch.  Currently,  controller can send the packet-out message with set of actions.    These actions are expected to be executed in the switch.  Typically 'OUTPUT' action is expected to be specified by the controller in the action list.  This action sends the packets out on the port given in "OUTPUT" action.  One facility is provided to run through OF pipeline though.  If the port in 'OUTPUT' action is specified to be a reserved port OFPP_TABLE,  then the packet-out message execution starts from table 0.

Normally controllers takes advantage of multiple tables in the OF switch by dedicating tables to purposes.  For example, when L4 switching  with L3 forwarding are used, multiple tables are used - A table to classify packets,  A table for Traffic Policing entries,  A table for PBR and few tables for routing tables,  a table for L4 connections and a table for next hop entries and a table for traffic shaping rules.   Some tables are populated pro-actively by the OF controller , hence miss packets are not expected by controllers.  Entries in some tables are populated re-actively.  Tables such as 'L4 connections',  'Next hop' entries are created reactively by the controllers.   Processing of the miss packets (packet-in) can result into a flow being created in the table which was the cause of miss packet. Then controllers likes to send back the packet to the OF switch and would like OF switch to start processing the packet as OF switch does when there is a matching entry. Controller programming does not get complicated  if that was possible.  As indicated before,  due to limitations of the specification,  controller can't ask switch to start from a specific table.   Due to lack of this feature in the switches,   controllers are forced to figure out the actions that would need to be performed and then program the action list in the packet-out message.  In our view,  that is quite complex task for the controller programmers.  As I understand,  many controller applications simply using packet-in message to create the fow, but drop the packet-in message.  This is not good at all.  Think of SYN packet getting lost.  It would take some time for client to retransmit the TCP SYN packet and that delay results into very bad user experience.

Also, in few cases,  applications (CP protocols such as OSPF, BGP)  generate packets that needs to be sent out on the data network.  Controllers typically sit in management network, not on the data network.  Hence these packets are to be sent to the data network via OF switches.  This can only be achieved by sending these packets as packet-out messages to the switch.  Yet times, these packets need to be exposed to only part of the table pipeline.  In these cases, it would be required to have ability for controllers to start the packet processing in the switch for these packets from a specific table given by controller.

Solution

Freescale Extension :

struct ofp_action_fsl_experimenter_goto_table {
      uint16_t type; /* OFPAT_EXPERIMENTER. */
      uint16_t len; /* Length is a multiple of 8. */
      uint32_t experimenter; /** FSL Experimenter ID **/
      uint16_t fsl_action_type; /** OFPAT_FSL_GOTO_TABLE **/
      uint16_t table_id;
}
OFP_ASSERT(sizeof(struct ofp_action_fsl_experimenter_goto_table == 8);

table_id :  When switch encounters this action,  switch starts processing the packet from the table specified by 'table_id'.

Though this action is normally present in the action_list of e packet-out messages, this action can be used in flow_mod actions too.  When there is GOTO_TABLE instruction and GOTO_TABLE action,  then the GOTO_TABLE action takes precedence and GOTO_TABLE instruction is ignored.

Notice that 'table_id' size is 2 bytes.  This was intentional.  We believe that OF specifications in future would need to support more than 256 tables.

L2 Network Virtualization & Is there a role for Openflow controllers?

 

Introduction

 

Current Method of Network Virtualization

IaaS (Cloud Service Providers) providers do provide network isolation among their tenants. Even Enterprise private cloud operators are increasingly expected to provide network isolation among tenants - tenants being departments,  divisions,  test networks, lab networks etc..  This allows tenants to have their own IP addressing space and possibly overlapping with other tenants' address space.

Currently VLANs are used by network operators to create tenant specific networks.  Some of the issues related to VLAN are:

• VLAN IDs are limited to 4K.  If tenants require 4 networks each on average,   only 1K customers can be satisfied on a physical network.  Network operators are forced to create additional physical networks when more tenants sign up.
• Performance bottlenecks associated with the VLANs :  Even though many physical switches support 4K VLANs,  many physical switches don't provide line rate performance when the number of VLAN IDs go beyond certain limit ( some switches don't work well beyond 256 VLANs)
• VLAN based networks have operational headaches -  VLAN based network isolation requires  all L2 switches are configured when a new VLAN is created or an existing VLAN is deleted.  Though many L2 switch vendors provide central console to work with their brand L2 switches,  it is operational  difficulty when switches from multiple vendors are present.
• Loop convergence time is very high
• Extending VLANs across data center sites or extending VLANs to customer premise has operational issues with respect to interoperable protocols,  Out-of-band understanding among network operators is required to avoid VLAN ID collisions.

To avoid issues associated with  capabilities  of  L2 switches,  networks having switches from multiple vendors  and limitations associated with VLANs,  increasingly overlays are used to virtualize physical networks to create multiple logical networks.

Overlay based Network Virtualization

Any L2 network requires the preservation of L2 packet from source to destination.  Any broadcast packet should go to all network nodes attached to the L2 network.  All multicast packets should go to network nodes that are willing to receive multicast packets of groups of their choice.

Overlay based network virtualization provides above functionality by overlaying the Ethernet packets using outer IP packet - Essentially tunneling Ethernet packetsfrom one place to another.

VxLAN, NVGRE are two of the most popular overlay protocols that are being standardized.  Please see my blog post on VxLAN here.

VxLAN provides 24 bits of VNI (Virtual Network Identifier).  In theory around 16M virtual networks can be created.  Assuming that each tenant may have 4 networks on average,  in theory, 4M tenants can be supported by CSP using one physical network.  That is, there is no bottleneck with respect to identifier space. 

Openstack

Openstack is one of the popular open source cloud orchestration tools.  It is becoming formidable alternative to VMWare vCenter and VCD.   Many operators are using Openstack and  KVM hypervisor as a secondary source of cloud virtualization in their networks.  Reliability of Openstack came long way and many vendors are providing support for a fee. Due to these changes,  adoption of openstack+KVM is going up as a primary source of virtualization. Openstack mainly has four components -  'Nova' for VM management across multiple physical servers,  'Cinder' for storage management,  'Quantum' for network topology management and 'Horizon' to provide front end user experience to operators (administrators and tenants).

Quantum consists of set of plugins -  Core plugin and multiple extension plugins.  Quantum defines API for plugins and let various vendors to create backend for the plugins.  Quantum core plugin API defines the management API of virtual networks - Virtual networks can be created using VLAN,  GRE and being upgraded to support VxLAN too.

Quantum allows operators to create virtual networks.  As part of VM provisioning,  Openstack NOVA provides facility for operators to choose the virtual networks on which this VM needs to be placed on. 

When 'Nova scheduler' chooses a physical server to place the VM, it asks the quantum to provide MAC address, IP address and other information to be assigned to VM using 'create_port'  API.  Nova asks quantum as many times as number of virtual networks that VM belongs to.   Quanutm provides required information to NOVA back.  As part of this call,  Quantum comes to know about the physical server and the virtual networks that needs to be extended to the physical server.  It, then, informs the quantum agent (that sits in host Linux of each physical server) the virtual networks it needs to create.   Agent on the physical server gets more information on virtual networks from quantum and then create the needed resources.   Agent uses OVS (Open Virtual Switch ) package that is present in each physical server to do the job.  Please see some description of OVS below.   Quantum agent in each physical server creates two openflow bridges - integration bridge (br-int) and tunnel  bridge (br-tun).  Agent also connects south side of br-int to north side of br-tun  using loopback port pair.   Virtual network port creation and association to br-tun is done by quantum agent for every new virtual network or when the virtual network is deleted.  North side of br-int towards VMs is handed by libvirtd and associated drivers as part of VM management. See below.

Nova talks to 'nova-compute' package in the physical server to bring up/down VMs.  'Nova-compute' in the physical server uses 'libvirtd' package to bring up VMs,  create ports and associate them with openflow switches using OVS package.  Brief description of some of the work, libvirtd does with the help of OVS driver are:

• Creates a Linux bridge for each port that is associated with the VM.
• North side of this bridge is associated with VM Ethernet port (using tun/tap technology).
• Configures ebtables to provide isolation among the VMs.
• South side of this bridge is associated with Openflow integration bridge (br-int).  This is achieved by creating loopback port pair with one port attached to Linux bridge and another port associated with the Openflow switch, br-int.

Openvswitch (OVS)

OVS is openflow based switch implementation.  It is now part of Linux distribution.  Traditionally Linux bridges are used to provide virtual network functionality in KVM based host Linux.  In Linux 3.x kernels,  OVS has taken that responsibility and Linux bridge is used for the purposes of enabling 'ebtables'.
 

OVS provides set of utilities :  ovs-vsctl and ovs-ofctl.   "ovs-vsctl" utility is used by OVS quantum agent in physical servers to create openflow datapath entities (br-int, br-tun),   initialize Openflow tables and add both north and south bound ports to the br-int and br-tun.   "ovs-ofctl" is command line to create openflow flow entries in the openflow tables of br-int and br-tun.  It is used by OVS quantum agent to create default flow entries to enable typical L2 switching (802.1D) functionality.  Since OVS is openflow based,  external openflow controllers can manipulate the traffic forwarding by creating flows in br-int and br-tun.  Note that external controllers are required only to add 'redirect' functionality AND virtual switching functionality can be achieved even without external openflow controller. 

Just to outline various components in the physical server:

• OVS  package - Creates Openflow switches,  Openflow ports and associate them to various switches and ofcourse provides ability for external controllers to control the traffic to/from VMs to external physical networks.
• Quantum OVS Agent :  Communicates with the Quantum plugin in Openstack tool to get to know the virtual networks and configure OVS to realize those networks in the physical server.
• OVS Driver in Libvirtd :  Allows connecting VMs to virtual networks and configures 'ebtables' to provide isolation among VMs.

Current VLAN based Network Virtualization solution

Openstack and OVS together can create VLAN based networks.  L2 switching is happening with no external openflow controller.   OVS Quantum agent with the help of plugin knows the VMs, their vports and corresponding network ports. Using this information,  agent associates the VLAN ID to each vport connected to the VMs.  This information is used by OVS to know which VLAN to use when packets come from VMs.  Also,  agent creates one rule to do the LEARNING for packets coming in from the network. 

Overlay based Virtual Networks

Companies like Nicira and bigswitch networks are promoting overlay based virtual networks. OVS in each compute node (Edges of the physical network) is used as starting point of overlays.  All L2 and L3 switches connecting compute nodes are only used for transporting the tunneled packets.  They don't need to participate in the virtual networks.   Since OVS in compute nodes is encapsulating and decapsulating inner ethernet packets into/from another IP packet,  in-between switches transfer the packets using outer IP header addresses and outer MAC headers.  Essentially,  overlay tunnels start and end at compute nodes.  With this,  network operators can configure the switches in L3 mode instead of problematic L2 mode.  May be, in future, one might not see any L2 switches in the data center networks.

Typical packet flow would be something like this:

- A VM sends a packet and it lands on the OVS in the host Linux.

- OVS applies actions based on the matching flows in br-int and packet is sent to br-tun.
- OVS applies actions based on the matching flows in br-tun and packet is sent out on the port (overlay port)

- OVS sends the packet to overlay protocol layer.

- Overlay protocol layer encapsulates the packet and sends out the packet.

In reverse direction,  packet flow would look like this:

- Overlay protocol layer gets hold of the incoming packet.

- Decapsulates the packets and exposes the packet with right port to the OVS br-tun.
- After applying any actions on the packet using matching OF flows in br-tun,  packet is sent to br-int.
- OVS applies the actions on the matching flows and figures out the destination port (one-to-one mapping with VM port)

- OVS sends the inner packet to the VM.

Note that:

- Inner packet is only seen by OVS and VM.

- Physical switches only see encapsulated packet.

VxLAN based Overlay networks using Openstack

OVS and VxLAN:

There are many open source implementations of VxLAN in OVS and integration of this with openstack.   Some details about one VxLAN implementation in OVS.

• Creates as many  vports in OVS as number of VxLAN networks in the compute node.  Note that,  even though  there could be large number of VxLAN based overlay networks,  only networks to which local VMs belong are created in OVS as vports.  For example,  If there are VMs corresponding to two overlay networks,  then two vports are created. 
• VxLAN implementation depends on VTEP entries to find out the remote tunnel endpoint address for a destination MAC address of the packet received from the VMs.  IP address is used as DIP of the outer IP header.
• If there is no matching VTEP entries,  Multicast learning happens as per VxLAN.
• VTEP entries can be created manually too.   A separate command line utility is provided to create VTEP entries to vports.  
• Since Openstack has knowledge of VMs and physical servers they are hosting,  Openstack with the help of quantum agent in each compute node can create VTEP entries pro-actively.

Commercial Products

Openstack and OVS provide fantastic facilities to manage virtual networks using VLAN and overlay protocols.  Some commercial products seem to be doing following:

• Provide their own Quantum Plugin in the openstack.
• This plugin communicates with their central controller (OFCP/OVSDB and Openflow controllers).
• Central controller is used to communicate with OVS in physical servers to manage virtual networks and flows.

Essentially,  these commercial products are adding one more controller layer between Quantum in openstack and physical servers.

My views:

In my view it is not necessary.  Openstack, OVS,  OVS plugin, OVS agent, and  OVS libvirtd driver are becoming mature and there is no need for one more layer of abstraction.  It is a matter of time where these open source components would be feature rich, reliable and supported by vendors such as redhat.   With OVS being part of Linux distributions and with ubuntu providing all of above components,  operators are better of sticking with these components instead of going for proprietary software.

Since OVS is openflow based,  there could be Openflow controller to add value with respect to traffic steeering and traffic flow redirections.  It should provide value, but one should make sure that default configuration is good enough to realize virtual networks without need for openflow controller.

In summary, I believe that Openflow controllers are not required to manage virtual networks in physical servers, but are required to add value added services such as traffic steering,  traffic visualization etc..

IP Fragmentation versus TCP segmentation

Ethernet Controllers are increasingly becoming more intelligent with every generation of NICs.  Intel and Broadcom have added many features in Ethernet NIC chips in recent past.  Multicore SoC vendors are adding large number of features into Ethernet IO hardware blocks.

TCP GRO (Generic Receive Offload - It used to be called Large Receive offload too) and GSO  (Generic Segmentation Offload and it is used to be called Transport Segmentation Offload)  are two new features (in addition to FCoE offloads) one can see from Intel NICs and many Multicore SoCs.  These two features are  good for any TCP termination applications on the host processors/cores.  These two features reduces the number of packets traversing the host TCP/IP stack. 

TCP GRO works across multiple TCP flows where it aggregates multiple consecutive TCP segments (based on TCP sequence number) of a flow into one or few TCP packets in the hardware itself, there by sending very few packets to the host processor.  Due to this,  TCP/IP stack sees  fewer inbound packets.  Since the packet overhead is significant in TCP/IP stacks, lesser packets uses lesser number of CPU cycles, thereby leaving more CPU cycles for applications, essentially increasing the performance of overall system.

TCP GSO intention is similar to TCP GRO,but for outbound packets.  TCP layer typically segments the packets based on  MSS value. The MSS value is typically determined from PMTU (Path MTU) value.  Since TCP and IP headers take 40 bytes of data,  MSS is typically ( PMTU -  40 ) bytes.  If PMTU is 1500 bytes, then the result MSS value is 1460. When the application tries to send large amount of data,  then the data is segmented into multiple TCP packets where each TCP payload carries up to 1460 bytes.  TCP GSO feature in the hardware eliminates the need for TCP layer to do the segmentation and thereby reduces the number of packets that traverse between TCP layer and to the hardware NIC.  TCP GSO feature in the hardware typically expect the MSS value along with the packet and it does everything necessary internally to segment and send the segments out.

Ethernet Controllers are increasingly providing support for IP level fragmentation and reassembly.  Main reason is being  increasing popularity of tunnels.

With increasing usage of tunnels (IPsec, GRE, IP-in-IP,  Mobile IP, GTP-U and futuristic VXLAN and LISP), the packet size is going up.  Though these tunnel protocol specifications provides guidelines to avoid fragmentation using DF bit and PMTU discovery,  it does not happen in reality.  There are very few deployments where DF (Don't Fragment bit) , which is required for PMTU discovery, is used.   As far as I know,  almost all IPv4 deployments fragment the packets during tunneling.  Some deployments configure network devices to do red-side fragmentation (fragmentation before tunneling so that the tunneled packets appear whole IP packet) and some deployments go for black-side fragmentation (fragmentation after tunneling is done).   On receive direction, reassembly happens either before detunneling or after detunneling. 

It used to be the case where fragmented packets are given lesser priority by service providers during network congestion.  With high throughput connectivity and increasing customer base for networks, service providers are competing for the business by providing very good reliability and high throughput connectivity. Due to popularity of tunnels,  service providers are also realizing that dropping fragmented packets may result in bad experience to their customers.  It appears that service providers are not treating the fragmented packets in a step-motherly fashion anymore.

IP fragmentation and TCP segmentation offload methods can be used to reduce the number of packets traversing the TCP/IP stack in the host.  Next question that comes to mind is how to tune the TCP/IP stack to use these features and how to divide the work  between these two HW features. 

First thing to tune in the TCP/IP stack is to remove the MSS dependency on PMTU.  As described above, today MSS is calculated based on PMTU value. Due to this, IP fragmentation is not used by TCP stack for outbound TCP traffic. 

TCP Segmentation adds the both TCP and IP header to each segment.  That is, for every 1460 bytes, there would be overhead of 20 bytes of IP header and 20 bytes of TCP header.  In case of IP fragmentation,  each fragment would have its own IP header (20 bytes of overhead).  Since TCP segmentation has more overheads,  one can say IP fragmentation is better.  Here, MSS can be set to a bigger value such as 16K and let IP layer fragment the packet if the MTU value is less than 16K.   This is certainly a good argument and it works fine in networks where the reliability is good.  Where the reliability is not good,  if one fragment gets dropped, TCP layer needs to send entire 16K bytes in retransmission.  If TCP had done the segmentation, it would only need to send fewer bytes. 

There are advantages and disadvantages with both approaches. 

With increased reliability of networks and with no special treatment on fragmented traffic by service providers,  IP fragmentation is not a bad thing to do.  And ofcourse, one should worry about retransmissions too. 

I hear few tunings based on the deployments.  Warehouse data center deployments where the TCP client and servers in a controlled environment are tuning MSS to 32K and more with 9K (jumbo frame) of MTU.  I think that , for 1500 bytes MTU,  going with 8K of MSS may work good.

SAMLv2 mini-tutorial and some good resources

Why SAML  (Security Assertion Markup Language)?

Single Sign-On:  Many organizations have multiple intranet servers with web front end.  Though all servers interact with common authentication database such as LDAP, SQL databases,  each server used to take authentication credentials from employee.  That is, if employee logs into server1 and then goes to server2 employees are expected to provide credentials again to the server2.  It may sound okay if the user is deliberately going to different servers.  But,  this is not good experience if there are hyperlinks from one server pages to other servers.  Single Sign-On solves this issue. Once the user logs into a server successfully,  the user does not need to provide his/her credentials when he/she accesses other servers.  Single authentication with single sign-on is becoming common requirement in organizations.

Cloud Services:  Organizations are increasingly adopting Cloud Services for cost reasons.  Cloud Services are provided by third party companies.  For example, salesforce.com provides Cloud service for CRM.  Taleo provides cloud service for talent acquisition systems.  Companies now have intranet servers,  some cloud services from different cloud service vendors.  Employees accessing the cloud service should not be asked to create accounts in the cloud service.  It is expected that cloud service provider uses the its subscriber organizations authentication database to validate the users signing in.  Many companies would not like to provide the access to their authentication database to cloud services directly.  Also,  many companies would not like their employees to provide their organization login credentials to cloud service for the fear of compromising the passwords, especially, senior and financial executives.  This requires the need for cloud services to redirect the employees to login using companies web servers and use single sign on mechanisms to allow the users to access cloud services.

SAMLv2 (version 2 is the latest version) facilitates the single sign-on not only across intranet servers of the company, but also across services provided by cloud providers.


How does SAMLv2 work?

There are two participants in SAML world -  Service Provider (SP) and Identity Provider (IDP).  Service provider is referred to represent the intranet servers and cloud servers.  Service provider provides some services to the authenticated users.  Identify provider refers to the system which takes login credentials from the user and authenticates the user using local database or LDAP database or any other authentication database.

When the user tries to access the resources in service providers,  SAMLv2 component of Service provider (SP)  first checks whether the user is already authenticated. If so, it allows the access to its resources. If the user is not authenticated, then rather than providing login screen and taking credentials, it instructs the user browser (by sending HTTP Redirect response with new location header containing the identity provider URL) to redirect the request to the identity provider.   Ofcourse SAMLv2 component of the SP needs to be configured with the IDP Provider URL beforehand by administrator.  SAMLv2 component of SP generates SAML request (in XML form) and 'Relay State".  It then adds these two to the location URL of the HTTP redirect response.   Browser then redirects the request as per location header to the IDP.   Now, IDP has the SAML request and Relaystate information. 

IDP first checks whether the user is already authenticated with it.  If not,  it sends the login page to the user as part of HTTP response.  Once the credentials are validated based on the authentication database,  SAMLv2 component of IDP generates the SAML response with the result of authentication,  principal name and  attests the response using private key of its certificate by signing the SAML response.  It also can add some more information about the user via attribute statements.  It then sends both SAML response and relay state to the browser as response to the last HTTP request it got (which is typically the credentials request).  IDP normally makes a HTTP page with embedded POST request  with  URL  which is SP URL (which it got from the SAMP request), SAML response and relaystate (which it got before) and javascript which is used by browser to send the POST request.   Browser gets this response and due to javascript it automatically posts the request to SP.  This POST request contains the SAML response and RelayState.

Now SAMLv2 component of SP ensures that SAML response is valid by verifying the digital signature.  Note that the public key (certificate) of IDP is expected to be configured in the SP beforehand by the administrator.  Then it checks the subject name of principal and result of authentication.  If result of authentication indicates 'success',  then it sends HTTP redirect response to the browser with original URL (from relaystate) as location header.  This makes the browser go the original URL to get hold of response.

Now onwards,  user would be able to access the pages in the SP.  If the user goes to another SP of the company,  same process as above would be repeated.  Since the user is already authenticated with the IDP,  IDP does not take login credentials again.  Since user does not see all the transactions that are happening underneath,  he/she gets single sign-on experience.

By keeping the IDP in Enterprise premises,  one can be sure that passwords are not given to any SPs whether they are intranet SPs or Cloud Service SPs.

SAMLv2 - Other SSO use cases:

The use case described above is called 'SP initiated SSO'. It is called SP initiated because user is trying to access the SP first.   Other use case in SSO is 'IdP initiated SSO' .   User access the IdP first.  IdP authenticates the user and then user is presented with links to different SPs that user can access.  When the user clicks on these special links, SAMLv2 component on the IDP generates the HTTP response with HTML page having POST binding to the SAMLv2 component of SP (It is called Assertion Consumer Service URL).  SAMLv2 response is provided in the HTML page as one of the post fields.  It adds 'RelayState' parameter with the URL that the user is supposed to be shown on the SP.   Note that 'RelayState" in this case is unsolicited.  There is no guarantee that the ACS in the SP will honor this parameter and also there is no guarantee that all ACSes treat the relayState as URL.   But many systems expect the URL in the relayState (including Google Apps Cloud Service) and hence sending URL is not a bad thing.

Above two cases 'SP Initiated SSO' and 'IdP initiated SSO' are mechanisms to access the resource by single authentication.  'Logout' is another scenario.  When user logs out (whether on a SP or on IdP),  the user should be logged out on all SPs.   This is called "Single Logout Profile". IdP is supposed to maintain all the SP information for which IdP was contacted for all users.  When the user logs out on IDP,  for all SPs in the user login session, it sends a logout request to 'Logout Service' component of SP.    If user logs out on SP and indicates his willingness to logout on all SPs, then the SP in addition to destroying the user authentication context on the SP also redirects the user browser to IDP. As part of redirection, it also frames the logout request too to the IDP.  IDP then destroys the context in the authentication context.  If the user indicated that he would be like to be logged out from all SPs, then the IdP would send the logout requests over SOAP to all other SP in the user authentication context.

For more detailed information about SAMLv2, start with SAMLv2 Technical overview and SAMLv2 specifications. You can find them here:

http://www.oasis-open.org/committees/download.php/27819/sstc-saml-tech-overview-2.0-cd-02.pdf
http://saml.xml.org/saml-specifications

IdP Proxy use case

So far the use cases described above talk about SP and IdP interaction via browser or directly.   SAML does not prohibit IdP proxy use case where IdP proxy work as IdP for SPs and as SP for IdPs.   Please see some links on this topic:

https://spaces.internet2.edu/display/GS/SAMLIdPProxy

Open Source SAMLv2 SP,  IdP and IdP Proxy:

I find that OpenAM (fork of Sun OpenSSO) as one of the popular open source SSO authentication framework.  You can find more information here:

http://forgerock.com/openam.html

IdP Proxy configuration is detailed here:

https://wikis.forgerock.org/confluence/display/openam/SAMLv2+IDP+Proxy+Part+1.+Setting+up+a+simple+Proxy+scenario

https://wikis.forgerock.org/confluence/display/openam/SAMLv2+IDP+Proxy+Part+2.+Using+an+IDP+Finder+and+LOAs

One more good site I found which details out IDP proxy and source code in Python:

http://www.swami.se/english/startpage/products/idpproxysocial2saml.131.html


Some good links, though not related IDP Proxy:

Google Apps as Service Provider and OpenAM as IDP:  https://wikis.forgerock.org/confluence/display/openam/Integrate+With+Google+Apps

OpenAM as Service provider and Windows ADFS as IDP:  https://wikis.forgerock.org/confluence/display/openam/OpenAM+and+ADFS2+configuration

Need for Pattern Matching Accelerators in UTM devices

Network security term typically refers to Threat prevention and Security on the wire.

Threat protection is normally achieved with multiple security technologies.  Basic protection is achieved from firewall technology.  IDS/IPS (Intrusion Detection/Prevention System),  Anti-Virus, Web application firewalls are some of the security technologies that are increasingly being used to protect networks (Network devices, Servers and Client machines).  Application Detection is another technology that is increasingly being used along with firewall to stop/allow traffic that can't be identified using ports in TCP/UDP headers, but requiring deep packet inspection.

Other than firewall,  all technologies listed above require deep packet and deep data inspection.  IDS/IPS technology adopts multiple techniques to identify the attack traffic. One of the techniques is to match the traffic data with known attack patterns.  Application detection also relays on pattern matching on the data as one of the techniques to detect the applications.  Anti Virus technology too depends on some pattern matching to detect viruses.

In almost all technologies above,  patterns get added to the deployed systems on continuous basis by device vendors as more attacks are discovered.  For example,  IPS devices, nowadays have around10,000 patterns (signatures) to detect the known attacks.  It keeps increasing every year.  Additionally, Some of these patterns are checked on every packet that goes through IPS.  This adds to number of CPU cycles requires to do IPS protection.

Many software algorithms are used to speed up the pattern matching performance.  Some of the techniques inlcude:

• DFA (Deterministic Finite State Automata)
• Bloom filters - Filters formed from the hashes of patterns can be used on the traffic to determine whether further analysis is required.
• PCRE algorithms to search for patterns of regular expression type.

IPS and other technologies also use techniques to reduce the number of patterns to be matched using protocol level intelligence and classifying the patterns in multiple buckets (protocol basis,  port basis,  even on the basis of application protocol stages such as URL based pattern database,  HTTP Request header,  Response header pattern databases etc..).

Due to above techniques,  some device vendors  think that there is no need for pattern matching hardware accelerators.  There is a reason for that too as some early developments of snort (popular open source IDS/IPS software) did not find much performance improvement with hardware accelerators.  But I believe HW accelerators are required for following reasons.

Performance Determinism:  IPS, Anti Virus,  Web application firewall and application detection technologies depend on the regular signature updates. Hardware deployed in the fields might have X number of signatures a day of purchase and they might go up to 2X or 3X over the years.  Performance determinism is expected by CSOs.  To maintain performance levels,  CPUs should be avoided in doing pattern matching.  Hardware accelerators specialized in pattern matching help in maintaining performance levels even with increasing number of signatures.

Protection from CPU hogging attacks:  With software based pattern matching, it is possible to hog the CPUs by crafting the packets with each data that matches a patterns multiple times.  Consider that there is a signature rule which tries to match a pattern "abc123def" and if there is 1Mbytes of data is sent with all the data having "abc123def" repeated,  then the CPU would take forever as it matches every packet multiple times.  CPU will not only spend time in matching the patterns, but also spends significant number of cycles in doing  further analysis.   Hardware accelerators normally designed such a way that the performance does not go down even if there are multiple matches.

Next question would be the what capabilities of hardware accelerators one should look for to mitigate performance issues  - One associated with explosive growth of attack patterns (signatures) and avoid CPU hogging by deliberate attempts by attackers.  I believe one should look for following capabilities.

• Accelerators should be programmable with decent number of patterns.
• Accelerators should be able to perform well even with large number of patterns.
• Accelerators should be able to perform well even if there are large number of matches.
• Accelerators should be able to perform pattern matches based on context information such as 'relative offset', 'Depth of the data to look' while doing pattern matching.  This will reduce the number of results being returned by the accelerator.  Smaller the number of results to software, lesser the post processing.
• Accelerators should be able to return results only when multiple patterns match on the data.  This also is required to reduce the number of results. 

In summary,  pattern matching hardware accelerators are required to reduce the CPU hogs either due to increase in signatures or intelligently crafted data by attackers. I feel that end customers should buy the UTM/IPS devices that take advantage of these accelerators to ensure that devices can be used at least for few years (future proof).

 

Locator and Identifier Seperation Protocol (LISP) - One more tunnel protocol

In 2012, I think that there would be focus on two technologies in network infrastructure market - SDN and  LISP (Locator and Identifier Separation Protocol).  LISP work is going on for few years and it seems to be talked about quite often in recent past.

Why LISP?

The reasons for LISP is detailed very well in the RFC 4984.  Some points of RFC 4984 are worth noting down and I am mentioned them here.

Multihoming

Internet presence is now part of business model of many organizations. Hence high availability of connectivity to Internet is becoming very important to organizations.   High availability is being achieved by having multiple links to ISPs and also multiple links to different ISPs.   Multiple links are used for load balancing the traffic as well as for redundancy.

Customers (companies) get IP address block (subnet) from the ISPs  and this address block is used by organizations to assign IP addresses to the machines that needs to be reachable from external nteworks.  Since each ISP would assign different blocks,  critical machines are provided with multiple IP addresses - one from each ISP assigned block.  Operating systems and routing protocols running in the machines and routers would ensure that the right IP addresses of the active links are used.  Each machine operating system should have this intelligence so that connections from applications running on operating sytem's TCP/IP stack are assigned with active IP addresses.  Since multiple IP addresses are assigned to a  machine,  machine is termed as multihomed machine. This concept is called multihoming. 

Even though above scheme works in general, existing active connections would get terminated if the link associated with IP addresses of the connections go down.  This could result in lost voice calls,  termination of very important TCP/IP connections.  This is one problem with provider assigned (PA) IP addresses (also called Provider Aggregatable addresses).   There is no issue for new connections though as routing protocols propagate this information.  Note that service providers don't allow the packets having source IP address other than the IP address block assigned by them from their customer networks.  Due to this, packets belonging to  active connections can't be sent onto links of other  service providers.  This is  one of the challenges organizations have with multihoming.

Second challenge with multihoming is that the propagation of active routes and links to each machine.  All machines that can be reachable from external networks need to have routing protocols implemented.  As you all know, end nodes typically don't have routing protocols enabled to not increase the maintenance headache for IT department.

Third challenge is multihoming for inbound connections to the organizations. When a link is down, somehow remote systems should not be using the addresses associated with the down links.   Typically, this is achieved by having FQDN (full qualified domain name) to each internal server machine and updating the DNS Server  with the IP addresses of active links.  That is, DNS Server can't just be using static information.  DNS Servers should be informed of  changes as soon as possible.  Even though this can be done at local DNS Servers level,  many DNS resolvers in Internet might have cached this information and remote systems continue to use down IP addresses for some time.  This can be achieved by not sending DNS response with 0 TTL, but that would increase the load on the local DNS Servers.

Finally,  organizations would like certain type of traffic (both inbound and outbound connections) to use some links over other links for several reasons such as cost of the link,  time of day etc..  

Basically, traffic engineering for outbound and inbound connections have good number of challenges and the techniques to overcome these challenges have limitations as described above.

Provider Independent Addresses:

Finally,  to address the issues of Multihoming and Traffic Engineering,  RIR (Regional Internet Registrar) introduced a policy document allowing organizations to request provider independent addresses (PI addresses). Provider Independent addresses are expected to be routed by all service providers.  That is, packets coming from their customers with these IP addresses as source IP address is expected to be honored by the service providers.

Benefits of the PI addresses are obvious with above background,  but consolidated reasons are given below:

• No need for multihoming support in end nodes, hence no need for enabling routing protocols in the end nodes.
• Traffic Engineering is simple - No need for dynamically updating DNS Servers.
• Simple to move to new service providers by organizations.  No renumbering the machines every time service provider is changed.
• With acquisitions and mergers,  consolation of networks is simple.

With PA (Provider Assigned) addresses,  addresses are aggregatable.  Hence the routing entries used to be small in number in the routers.  With provider independent addresses, the routes can't be aggregatable and hence the routing table size increases dramatically. Routing table sizes of  DFZ (Default Free Zone) routers are going up dramatically due to PI addresses.  According to BGP Routing Analysis Report, the number of routes in the DFZ routers went from 5000 in year 2000 to 400,000 in year 2011.  With IPv6 popularity and more liberal assignment of PI addresses in IPv6, it will not be a surprise where the number of routes in DFZ routes going to millions in next few years.  Since the routing table is referred by DFZ & service provider  routers  for every packet that is coming in,  more routes in the table reduces the performance of the router and hence the performance of overall Internet.

LISP is mainly born to address the issue of the scaling in DFZ routers.


Basic concept of LISP:

LISP is trying to keep the advantages of Provider Independent Addresses to the organizations and keep the routing table to reasonable size by using aggregatable addresses.  To address this,  LISP proposes two addressing schemes - Identifier Address space and  Locator Address space.  Identifier address space is similar to  provider independent address space. Organization are expected to assign addresses from allocated space to individual network elements.  Locator space is also assigned to organization, but it is it provider aggregatable address space.  Hence, this space should not be used to assign addresses to all network elements.  This address space should be assigned to tunnel routers (LISP tunnel routers) only.   When the organization changes the service provider,  it should only need to worry about IP address assignment to LISP routers and no other change is expected.

LISP standards call Identifier address EID (Endpoint ID) and locator IP address RLOC (Router LOCator).

LISP router contains two functions - Ingress Traffic Router(ITR)  and Egress Traffic Router (ETR). Ingress and Egress terms are with respect to Endpoint network.  Since endpoint identifier space is not expected to be visible to the core network routers,  ITR encapsulates the traffic coming from the endpoint network with tunnels with LISP, UDP and IP headers and sends it out onto the Internet.  ETR is expected to decapsulate the traffic coming from the Internet and pass the internal packet to the endpoint network.   Typically, ITR and ETR are implemented in customer edge routers. Initially,  Enterprises might expect service providers to provide LISP service and eventually Enterprise routers will have this functionality. 

The IP addresses used in IP header of tunnel  are from the RLOC space.  Since RLOC space is provider aggregatable,  routing table size will not increase dramatically.  Please see the LISP draft for more information on the tunnel header formats.


How does it solve the issues/challenges discussed above?

Multihoming is no longer required in end nodes. But it is still required on LISP routers though - That is there would be requirement for multiple links from different providers for redundancy and traffic engineering.  Active connections will not be suffered if traffic is redirected to other links as endnodes work with EID space always and those IP addresses would continue to work, similar to provider independent addresses.  Outer IP header address of LISP tunnel would change when links go down and come back up.  That should be okay as these addresses are only used to get to the LISP ETR.

EID to RLOC mapping:

ITR needs to know the source IP and destination IP to be used for the tunnel header. ITR uses the destination IP (EID) of the packets coming in from the local network to determine the remote ETR RLOC IP address. It does this using mapping database.  Each ITR expected to maintain EID to RLOC cache.  If it does not find the matching entry in the cache, then it talks to mapping resolvers.  Mapping resolve servers uses the Mapping Database to figure out the destination ETR and lets the destination ETR to send actual EID to RLOC mapping to the requesting ITR.   Basically, Mapping resolves and mapping databases only used to find the ETR.  But ETR is the one which gives the EID to RLOC mapping to the ITRs.

ETRs are expected to register its RLOC with the mapping database for EID prefixes it controls.  This is done using MAP_REGISTER message.  ITRs send MAP_REQUEST message to get the EID to RLOC mapping to mapping resolvers.  Mapping resolvers use the map register database to know the RLOC of the ETR and translates the map request destination IP address with the RLOC to redirect the  MAP-REQUEST packet to the ETR.  ETR then replies using MAP-REPLY message with the actual RLOCs to be used by ITR.  One might ask why can't mapping resolvers itself sends the MAP-REPLY to the ITR.  ETR is given this opportunity to do inbound traffic engineering.   ETR can give different RLOC IP address for different type of traffic or use different link at different times etc..

Mapping database, mapping resolver servers and associated message formats are described in IETF draft LISP MAP Server interface.

In summary,  MAP resolvers and MAP database servers are used to index ETRs and ETRs are the ones which actually provides EID to RLOC mapping.

The challenge really is how the index database is implemented.  Note that this database can become big as all EID prefixes would be maintained in this database.  This database search needs to be fast and the database is updated by multiple ETRs.  Update database operation also need to be fast. Ofcourse serach operation needs to be very fast.  To take care of scalability issues, multiple database servers would need to be used.  It is also required to divide the database into multiple servers.  One proposal I see usage of DHT (Distributed Hash Table).

Please see following links:

Alternate network:  http://tools.ietf.org/html/draft-ietf-lisp-alt-1
There is a DHT alternative to this.

Summary:

Year 2012 would see LISP based routers.  Initial set of routers would have software implementation of LISP routing functionality.  Once the standards achieve certain level of  maturity,  one would see Ethernet controllers (standalone or Multicore based) vendors adopting this technology in hardware.

Table Centric Processing and Openflow

Software in  Embedded network appliances consists of multiple entities - Management Plane (MP), Control Plane (CP) and Data Plane (DP).  Some examples of network appliances include routers, switches,  load balancers,  security devices and WAN Optimization devices etc.. 

MP entity typically consists of management engines such as CLI,  GUI Engines and Persistent storage Engines.

CP is mainly consists of several sub entities for different functions. For example, if a device consists of firewall,  Ipsec VPN and routing functionality,  then  there could be three sub-entities.  Each sub-entity might consists of multiple protocols and modules.  Routing function might have multiple control plane protocols such as OSPF, BGP, IS-IS etc..  CP also typically consists of exception packet processing modules.  For example, if there is no flow processing context found for a given packet in DP,  DP gives the packet to CP to process it and lets the CP to create the flow context in DP.

DP entity consists of actual packet processing logic. DP entity is typically implemented in Network Processor Units,  ASIC.  It is increasingly being implemented in Multicore processor SoCs with some cores dedicated for CP and rest of cores for DP. 

In this post, I am going to concentrate on DP (also called as datapath). 

As indicated DP implements the "datapath" processing.  Based on the type of network device, datapath consists of 'routing',  'bridging',  'nat', 'firewall', 'ipsec' and 'dpi'  modules and more.  Some background on each of the datapath functionality is listed below. Then we will see how table driven processing model helps in implementing datapaths.

Routing

Routing module typically involves following processing, upon packet reception:

• Parse and check integrity of the packet
• L2 (Ethernet, VLAN etc..)  header parsing.
• Identification of packet type 
• If it is IP,  proceed further, otherwise do non-IP specific processing.
• Integrity check involves ensuring the IP checksum is valid and packet size is at least as big as the 'total length' in the IP header.
• Extraction of fields - Source IP address,  Destination IP address,  ToS fields from the IP header.
• IP Spoofing Check:  This processing is done to ensure that the packet came from the right interface (Ethernet port).  Normally this is done by doing route lookup on source IP address.  Interface returned from the route lookup operation is compared with the port on which this packet came in. If both are same, then IP spoofing check is treated successful and further packet processing happens.  Basically, this check ensures that packet with destination IP address of current packet source IP address would go on the interface on which current packet came in. This requires "RouteLookup" operation using packet source IP address on the routing table which is populated by the CP.
• Determination of outgoing interface and next hop gateway:   This processing step involves "route lookup" operation on the "Destination IP" of the packet and optionally on TOS value of the packet.  "Route Lookup" operation indicates whether the packet needs to be consumed by local device (that is DIP is one of the local IP addresses) or whether the packet needs to be forwarded.  If the packet needs to be sent to the next hop, then the gateway IP address (or destination IP itself if the host is in the same subnet of the router) and outbound interface would be returned.   If the packet is meant to the local device itself, then packet is handed over to the CP. Rest of the processing assumes that packet is forwarded to the next hop.
• TTL Decrement :  At this processing step,  DP decrements the TTL and does incremental checksum update to the IP header checksum field.  If the TTL becomes 0, then the packet gets dropped.
• Packet Fragmentation:  If the packet size is more than PMTU value of the route or MTU of the outbound interface, then packets gets fragmented at this step.
• MAC Address Determination:  If the outbound interface is ETHRENT type  then this processing step  finds out the MAC address for the gateway IP address determined in one of the previous steps.  It refers to the "Address resolution table" populated by the CP.  It uses IP address as input and gets the MAC address.  If there is no matching entry,  packet is sent to the CP for ARP resolution.  In case the outbound interface is Point-to-Point interface,  L2 header can be found from the interface table which is populated by the CP.
• Packet update with L2 header:  DP frames the Layer 2 Header (Ethernet, VLAN, PPP etc..) and prepends in the beginning of the packet right before the IP header. 
• Packet Out:  Packet is sent out the outbound interface determined as part of "routelookup" operation.

At each step,  appropriate statistics counters are incremented. There are two types of statistics - Normal and Error statistics.  These statistics are typically  maintained globally,  on per interface basis,  on per routing entry or per ARP table entry.

Upon analysis of above steps,

IP routing datapath refers to few tables:

• IP Routing Database Table, which is LPM (Longest Prefix Match), for route lookup operation.
• MAC Address resolution table, which is exact match table,  to find MAC address for a given IP address.
• Interface Table, which index table, to find the L2 header to be used in case of  Point-to-Point interfaces.  Interface Index is typically returned by the "route lookup" operation.

IP routing in some devices also implement "Policy Based Routing" (PBR).  PBR is an ACL type of table with each entry identified by several fields, including IP header fields (SIP, DIP, TOS),  Input interface and output interface and even on the transport header fields.  The output of each ACL rule is the routing table.  Basically, "route lookup" functionality involves two steps - One finding the matching ACL rule in PBR table. This gets the routing table to use.  Then this routing table is searched with IP address and TOS to get the routing information as part of second step.  ACL rules can have field values represented in subnets, ranges and single value.   ACL is typically ordered list.

Bridging:  Bridging switches the packets without any modification to the packet.  Bridging is typically done among the Ethernet interfaces.  Bridging typically involves following steps upon receiving the packet from one of the Ethernet interfaces that belong to a bridge.

• Ethernet  Header Parsing: In this step,  Ethernet header and VLAN headers are extracted from the packet.
• Bridge Instance determination:  There could be multiple bridge instances supported by datapath.  This step determines the bridge instances based on the interface on which packet came in.  A given Ethernet or VLAN interface belongs to only one bridge.  CP populates the Interface table with entries, with each entry identified "Interface Index" and having parameters such as Bridge Instance Index (Index to bridge table),   whether this interface is allows learning,  forwarding etc.. 
• Learning Table Update:   Each Bridge maintains the learning table. It is also called FDB (Forwarding Database).  This table aims to provide Interface information given the MAC address.  This table is normally used by bridging datapath to determine the outbound interface for received packets.  But this table is updated with entries based on the incoming packet.  Its source MAC address and the interface on which packet came in used to populate the learning table. Basically,  it learns the machines on a physical network attached to the Ethernet port.   As part of this processing step,  if there is no learning table entry, then new entry gets added.  Note that learning is done if the Ethernet interface status in thhe "interface table" indicates that this interface can be used to learn the machines. In some systems,  the population of the learning table is done by the CP.  Whenever DP finds that there is no entry that matches the SMAC, then it sends the packet to the CP and CP creates the table entry.
• Determination of outbound interface:  In this step,  DP does the lookup on the FDB with DMAC address key.  If there is a matching entry, it knows the outbound interface on which the packet needs to be sent.  In case of Multicast packet, it refers to the Multicast FDB to find out the set of interfaces on which the packet need to sent out.  In case of broadcast packet (DMAC = ff:ff:ff:ff:ff:ff), then all interfaces in the bridge is selected to send the packet out.  Note that Multicast FDB is populated by the CP. CP does this by interpreting the IGMP, MLD and PIM-SM packets.
• Packet Out:  In this step, packet is sent out.  In case of unicast packet,  if the interface is known, packet is sent out.  In case the outbound interface is unknown (there is no matching FDB entry), then packet is sent out on all ports except the incoming port.   Packet is duplicated multiple times to achieve this.  Multicast packets are also sent on all ports if there is no matching  entry. Otherwise, it sends the packet to interfaces as indicated by Multicast FDB.  Broadcast packets are always sent to all ports in the bridge. Note that, in all these cases, packet is never sent on the interface on which packet came in. Also,  datapath does not send the packet if the interface is blocked for sending packet out.  This information is known from the interface table.

Bridging module maintains following tables.

• Unicast FDB, which is exact match table. Match field is 'MAC address'.  One FDB is maintained for each bridge instance.
• Mutlicast FDB, which  is also exact match table.  Match field is 'Multicast MAC Address'. One Multicast FDB is maintained for each bridge instance.
• Interface Index Table - Which is indexed by the incoming interface identifier. Typically global to the entire datapath.
• Bridge Instance Table - Which is indexed by bridge instance. Typically global to the entire datapath.

NAT/Firewall Datapath: NAT and Firewalls typically require Session Processing.  Session Management part of NAT/firewalls is typically implemented in datapath.  Sessions are typically 5-tuple sessions (Source IP, Destination IP, Protocol, Source Port and Destination Port) in case of TCP and UDP.  A Session is initiated by client and terminated by server.  NAT/Firewall devices being in between client and server machines,  it sees both sides of the connection - Client to Server and Server to Client.  Hence, in these devices,  sessions consists of two flows - C-to-S and S-to-C flows (Client-to-Server and Server-to-Client flows).   5-tuple values of C-to-S and S-to-C flow are same in case of firewall session except that values are toggled between source and destination.  That is, source IP of the C-S flow is used as destination IP of the S-C flow and destination IP of the C-S flow is used as source iP of the S-C flow. Same is true with source and destination ports.  In case of NAT,  this may not be as simple. NAT functionality translates client SIP, DIP, SP and DP to new values.  Hence the NAT session manager flows would have two different values in 5-tuples, but both belong to one session.  In summary,  a session maintained by NAT/firewall devices is pointed by two different flows.

Session Processing involves following steps in datapath:

• Packet Parsing and Integrity of headers (L2 header, IPv4/IPv6 header, TCP/UDP/ICMP/SCTP headers) :  Similar step as described in "Routing" section. 
• IP Reassembly:  Non-first IP fragments will not have transport header. Since session requires all 5-tuples for matching sessions,  if reassembly is not done, non-inital fragments will not be get matched onto the right flows. As part of the IP reassembly, many checks are made to check the anomalies and as part of this check fragments might get dropped.  Once the full transport packet is reassembled,  further processing happens.  Some of the checks that are done as part of this processing step are:
• Ensures that initial and middle fragments are at least of some configured size - This is to figure out any deliberate fragmentation by sender to make the device spend large number of CPU cycles there by creating DoS condition.
• Ensures that the total IP packet size never exceeds 64K size.
• Taking care of overlapping IP fragments.
• Ensuring that the data in the overlapped IP data is same. If not, it is considered as deliberate attempt and this processing steps drops these packets.
• Transport Header integrity checks:  At this step, transport header integrity is ensured such as
• Ensures that the packet holds transport header.
• Some datapaths may also verify transport checksum.
• Extraction of fields from the parsed headers :  Mainly SIP, DIP, P,SP and DP are extracted from network and transport headers.
• Security Zone lookup :  Every incoming interface belongs to one of the security zones.  CP programs the table of entries with each entry having Interface ID and security Zone ID it belongs to.  Security Zone ID is used in flow lookup function.
• Flow Lookup to get hold of session:  As discussed before, a session consists of two flows.  Flows are typically arranged in hash lists.  When the session is created by CP,  CP is expected to put two flows in the hash list  with each flow pointing to its session context.  Sessions are arranged in an array (Index Table).  Reference to session (array index) is put in the flow.   Basically,  there is one hash list for flows and Index table for sessions.   5-tuples and inbound security Zone ID are used to match the flow.  If there is no matching flow found,  the packet is sent to the CP.  If the flow is found,  then it goes onto next processing step.
• Anomaly Checks:  Once the flow, associated session context and complementary flow is determined,  then this processing step does checks to find out any anomalies using packet header contents and state variables that are maintained in the session.  Few things that come to my mind are: 
• Ensuring TCP sequence number of the current packets within reasonable range (Example: 128K) from the previous packet sequence.  This is done typically to ensure that MITM did not generate the TCP packet, mainly TCP packet with RST bit is set.   TCP RST packet can drop the session not only in the device, but also on the receiving client/server machine.
• Packet Modification:  This step involves packet modification.  Packets belongs to sessions that are created by firewall CP may not undergo many changes. In case of NAT, there are several modifications are possible based on session state created by CP.  Some of them are:
• Source NAT and Destination NAT modifies the SIP and DIP of the packet.
• NAPT may modify both SP and DP of TCP and UDP packets.
• To ensure that modified packet has unique IPID,  IPID also gets translated with unique IPID.
• TCP MSS value might be modified with lesser MSS value to avoid any fragmentation in future steps.
• TCP sequence numbers also may get translated. This is typically happens when CP does some packet processing before such as SYN FLOOD protection using SYN-COOKIE mechanism or due to some ALG processing.
• Packet Routing and Packet-out:  These steps are similar to IP Routing module.

 Upon analysis of above steps, one can find that there are following tables in Session Processing.

• Interface to Security Zone mappping table, which is an index table.
• Flow hash table, which is exact match hash table.
• Flow to Session ID table, which in index table.

Observations & Suggestions for new version of Openflow specifications:

All datapaths listed above use tables heavily.  Contexts created in each table are used by actions being performed.  It is not clear whether Openflow specification group is targeting to cover any type of datapath.  If so, few things to notice and improve. 

Packet Reinjection :  Each datapath might have more than one table.  Typically a table miss results into packet going to the control plane (controller in case of Openflow).  Control plane once acts on the packet and create appropriate context in the table,  it would like the packet processing in the datapath  to start from where the miss occurred.  Some times,  the processing should start from the table where the miss occurred or some times CP might do the operations on the packet and hence the packet processing needs to start in the datapath from the next table as specified in the newly created context.  Protocol between Control plane and datapath (Openflow) needs to provide this flexibility for control plane.

Atomic Update and Linkage of flows: Control plane yet times need to create multiple contexts in tables atomically.  For example, NAT/firewall control plane would need to create two contexts for two flows in atomic fashion.  Commit based facility would help in achieving this.  I think it is required to enhance openflow protocol to enable this operation.

Parsing headers & Extraction of fields:  Since there could be different datapaths that need to be exercised on a packet, one time parsing of the packet may not be sufficient. Each datapath or even each table might be preceded by parsing & extraction unit. There are different fields which are required in different paths.  There could be tunnel headers which need to be parsed for some datapaths to get to inner packet.  Each datapath might have its own requirement on which fields to be used for further processing.  That is, some datapath might need to work on IP header of tunnel and some other datapath might have requirement to use inner IP packet fields.  Hence I believe it is good to have parsing & extraction unit for each table. In cases, where the datapaths don't require different parsing & extraction units,  controller would not configure associated parsing & extraction units.  Some hardware devices (Multicore SoCs) I am familiar with support multiple parsing & extraction unit instances.  So, this is not a new thing for hardware ASIC/Multicore-SoCs. 

Pre Table lookup Actions:  Many times, there is a requirement to ensure that packet is good.  Packet integrity checks are very important during packet processing. Having some actions defined on per table basis (not on per flow basis) is good way to do these kinds of checks.  There may be a requirement in future to do even some modification to the packet before further parsing/extraction & eventual table lookup.  Pre-Table action list is one way to provide these facilities.  One can argue that, it is possible to use another table before packet is processed at current table and have context specific action on that table.  Though it is good thought,  it might result into inefficient usage of table resources in hardware.

Flexible Metadata:  Metadata is used to communicate the information among different tables and associated actions. There are two types of metadata required in datapaths - Some part of metadata is used as lookup action in further tables and some metadata is required by actions. Openflow specification defined the metadata, but it is limited to one word.  I believe that this is insufficient. Fixing the metadata size in standard is also not good. Controller can define the maximum metadata size required and program the size information in the datapath.  Datapath can allocate additional space to store metadata on per packet basis using the size information programmed by the controller.  Since there could be multiple types of datapaths and multiple tables within each type of datapath,  controller may categorize metadata for different purposes. Each metadata field and field size can be referred later on by table for lookup and associated actions.  I believe that openflow should provide that kind of flexibility - Setup the size of metadata required one time in the datapath,  language to use different fields in metadata for table lookup,  language to define action which can use different fields in the metadata or set/reset some fields in the metadata.

Actions :  Each type of datapath has its own actions - For example firewall session management datapath actions are different from the type of actions routing or bridging require.  Hence, there would be good number of actions eventually in the datapath.  And more actions would be required in future.  In addition to defining more actions in the openflow or associated specifications,  it should also provide flexibility to add new actions without having to create new hardware.  That is, openflow might need to mandate that the datapath hardware or software should be able to take newer actions in future.  To enable this,  it is required to define some kind of language to define new actions.  Since datapaths may be implemented in hardware ASIC,  the language should not have too many constructs/machine-codes.  Simple language is what is needed.  I thought simplified LLVM can be one option.