LTE GTP User plane eNodeB - Tutorial

This particular tutorial is limited to explaining some concepts related to GTP-U between eNodeB and S1 Peer Gateway.  Also it is limited to giving information from the context of eNodeB.

GTP-U specifications are defined in 3GPP TS 29.281 at www.3gpp.org.

GTP-U in eNodeB tunnels IP packets received from PDCP layer to S1 gateway by tunneling them in GTP tunnel in uplink direction.  In downlink direction, eNodeB detunnels the packets from GTP tunnel and pass the packets onto the PDCP layer to send them to mobiles (UEs).

GTU-U is like any other tunnel protocol adds protocol specific headers and then finally adds the outer IP header to route the packets to peer -  GTP-U header, UDP header and outer IP header to the packet as part of tunneling.  As part of de-tunneling, it removes the GTP-U, UDP and outer IP headers.

As part of S1-AP and X2-AP protocols,  GTP tunnel contexts are created in GTP-U module. 

Identification of GTP-U tunnel in eNodeB:

• PDCP layer in eNodeB gets hold of IP packets that are sent by UE to the core network and passes them to the GTP layer.  In addition to providing the packets, it also provides C-RNTI (Cell Radio Network Temporary Identifier), Logical Channel ID and a downlink handover flag (handover stage) or uplink handover flag. One of the handover flags is set only when the corresponding DRB in PDCP was informed by the RRC in control plane to forward the packets to the target eNB as part of handover process.   Why is the handover stage flag required?  It is because, during handover time, there could be more two GTP tunnel context for the same bearer (C-RNTI and LCI combination) - One established by S1 protocol to core network and another one established by X2-AP module to the target eNodeB.  If X2 connection is not available, then S1-handover would be used for Intra-LTE handovers, in which case,  you would have two S1-AP tunnels.   This flag would help in choosing the right GTP-U tunnel context.   Using these three tuples (C-RNTI, LCI and handover flag), GTP-U tunnel context is identified within a sector.  Processing that should happen for downlink traffic during handover is described later.  I am not sure about this, but my impression is that C-RNTI is unique within a sector. Since on GTP-U software instance works on multiple sectors (typically 3),  sector ID needs to be one of the key values to identify the GTU-U tunnel in uplink direction.  If same eNodeB GTP-U is used by multiple operators,  then one more parameter (operator ID) will be another identification parameter required to identify GTP-U tunnel.  As of this writing, I did not get any information that same eNB transport hardware is shared by multiple operators.  If only one operator, then the operator ID is always assumed to be 0.  Finally, we have following parameters to identify the GTP-U tunnel in uplink direction.
• Operator ID (Virtual Instance ID)
• Sector ID
• C-RNTI
• LCI
• Uplink handover flag (If  uplink traffic is handed over to target eNB)  or downlink handover flag (if downlink traffic is handover to the the target eNB).  Note that in non-handover scenarios, both the flags would be set to 0. 
• In downlink side,  packets are given to the GTP -U layer after Ipsec decryption is done.  IPsec layer is expected to pass just the packet.  Packet contains GTP-U header and outer IP header.  GTP-U header contains TEID (Tunnel End Point ID).  Using Source and Destination IP addresses of outer IP header and TEID,  GTP-U tunnel context can be identified.  Key parameters to identify the GTP-U tunnel in the downlink direction are:
• Operator ID (Virtual instance ID)
• Source IP  (from outer IP header of GTP)
• Destination IP (from outer IP header of GTP).
• TEID : This is the ID which was sent to the SGW by the control plane software for SGW to send the packet to the right GTP-U tunnel.

GTP-U Operational concepts in eNodeB:

Working with Path MTU discovery procedures by UE and end points on core network side: 

GTP-U tunneling increases the packet size since it adds GTP-U header and outer IP header to the packet.  If the MTU of the outbound interface is less than the resulting packet size, then packets would need to be fragmented on the sender side and they need to get reassembled before handing over the packets to the received GTP-U layer.  Fragmentation and reassembly not only adds CPU cycles, but also increases the amount of bandwidth that would be used in the backhaul network.  Also, if one fragment is dropped for whatever reason,  the entire set of fragments corresponding to the packet would get dropped as part of reassembly process.  This can lead to more bandwidth on the wireless link too.  I have my own theory on thi which I explained later on.   Anyway to reduce the fragmentation,  PMTU discovery process is normally adopted by any network elements and UE and any other end points are no exception.  GTP-U layer in eNodeB should honor Path MTU discovery process even for IPv4 networks.GTP-U layer also can follow similar approach that are followed by other tunneling protocols.

• GTP-U module in eNodeB:
• GTP-U layer receives the IP packet with DF bit set.
• GTP-U layer knows the amount of header information it is going to add as part of tunneling.
• If the resulting data size is less or equal to  the PMTU value which is already stored in the GTP-U context,  then there will not be fragmentation.  It also need to ensure to put the DF bit in the outer IP header to ensure that it discovers new PMTU value if any.  As part of GTP-U context creation,  it can set its PMTU value from the interface MTU value.
• If the resulting data size is more than the PMTU value stored in the GTU-U context, it generates ICMP Destination unreachable message with the PMTU value  = PMTU value stored in the GTP-U context - Header information GTP-U layer adds.
• When GTP-U module receives the ICMP Destination Unreachable - Path MTU excededed message, it should find out the GTP-U context from the payload of ICMP error message. It should only interpret the ICMP error messages destined to it.
• Update the PMTU value stored in the GTP-U context if it is less than the value in ICMP error message.  This updated value would be used by new packets coming from UE at later time and can be used to generate the ICMP error message itself.

If IPSec packet processing is also going to happen, then it is better to keep the size of headers being added by IPsec too while doing above operations. 

Ok, now to my concerns: Wireless link bandwidth may be more precious compared to backhaul network bandwidth.  By doing PMTU discovery, TCP MSS value would be decreased.  That is more IP packets would be generated for a given amount of data when path MTU discovery process is run.  More packets is equal to more overhead on the wireless link.  So, I believe that PMTU discovery will not be used in IPv4 world.  Having said that, above logic is expected to be implemented by eNodeB.

IPv6 Support:

IPv6 is becoming popular in RAN too.  It is expected that Ipv6 is also supported.  Here rather than Ipv4 outer IP header, IPv6 IP header is used.  Note that internal packet can be IPv4 or IPv6.

HopCount/TTL Decrement:

TTL Decrement operation on the internal packets need not be done by the GTP-U layer.  But I think it should check for TTL and discard the packet if the value is 0.

MBMS (Multmedia Broadcast Multicast Service) - Section 4.2.6 of 3GPP-TS-29.281 :  I don't have much idea on how this works in the control plane.  My guess is that IGMP proxy mechanism is used in eNodeB.  Based on this information, EPC somehow creates the GTP tunnel with all eNBs that require multicast packets.  It appears that EPC sends the TEID (GTP Tunnel ID) via control plane protocols. GTP-U layer gets the request from the control plane protocols to create the GTP Tunnel with the given TEID. Note that in unicast, the receiver of the GTP packets would give the TEID to peer. But in multicast,  sender of the GTP packets give the TEID to the receiver.  I guess it is required this way to ensure that all eNB get the same TEID and only one copy need to be sent from the EPC to eNBs over backhaul network.

My understanding of the multicast packet processing is simple in eNB.  When GTP-U receives the packets and detunnels them (in downlink direction), it needs to inform PDCP that these packets are Multicast packets. L2 stack of the LTE might use this information to send the packets over to Multicast transport channel.  I am not sure whether there is any RAB (Radio Access Bearer) created for multicast packets.  I think it should. If that is the case, then there is no special flag is also required between GTP-U and PDCP.  Does anybody know? If so, somebody inform me which specification talks about this?

Handling of Sequence numbers -  Section 4.3.1 of 3GPP-TS-29.281:

It appears from the description of this section,  only requirement on eNB to ensure to pass the sequence number in the GTP header which comes along with the packet should be sent to the PDCP.  I don't know what PDCP does as it has its own sequence number.  Though this section talks about reordering of received packets, it did not talk about what should be done in case some packets got dropped on the way.  There is no mention of retries and acknowledgments. It makes me believe that this section is not applicable for eNBs.  I would love to hear from others on this.

This section also talked about sending message number (sequence number) along with request signaling messages if there are responses.  GTP Echo-Request is  path management request message to find out whether the peer is active and live.  Peer responds by sending Echo-Response.  Response is expected to have same message number for sender to match the response with request message.

Interfaces with other modules:

3GPP specifications goes into great details on interfaces among wireless network elements - eNB, SGW, MME, UE etc..  For right reasons, 3GPP specifications don't  describe the interfaces among different functions within each network element. It is left to the developers working on particular network elements. Vendors of these equipment may get  some software elements from different software vendors.  In those cases, it becomes easy for integrators if there are standard interfaces. I did not find any software API definition.  I tried to give my understanding of interfaces among different software modules that interact with GTP-U.  In following sections,  I have combined both GTP-U, GTP-U relay entity and UDP into one and I am calling that combination as GTP-U.

GTP-U interfaces with four other software elements:

• Control Plane (S1 and X2) - for setting up and tearing down  the GTP-U tunnels.
• Packet Processing :
• PDCP -  For sending packets to the UE in downlink direction and to receive packets from PDCP in uplink direction.
• IP Forwarding (with and without IPsec) -  to send and receive packets from the EPC or other eNodeBs in handover cases.
• Management plane - for initialization, configuration,  reading statistics, sending alerts/logs etc..

Initialization & Configuration:

• Init():  This function is typically needed to allocate memory for different contexts - GTP tunnel contexts and GTP Path Management contexts.  Note that there could be as many GTP tunnel contexts as number of Radio Access Bearers.  Each UE typically have three SRBs and at least three DRBs - Default, Voice dedicated and Data dedicated bearers.  GTP tunnels are established only for DRBs though.  There could be as many GTP path management contexts as number of S1 peers. X2 peers will not participate in echo request/response path. Parameters can include:
• Maximum number of Virtual Instances
• Maxinum number of sectors
• Maximum number of GTP Tunnel contexts across all Virtual instances and Zones.
• Maximum number of Path Management contexts.
• Configuration:
•  SetEchoConfiguration() :  Echo request/response messages are used to find out the liveness of the peer.  Sender retransmits the echo request to the peer for every X number of seconds (T3_RESPONSE timeout) for Y number (N3-REQUESTS) if it does not receive response.  If it does not receive response for y number times, then it GTP-U module is expected to send the alarm to the control plane.  Parameters
• T-RESPONSE timeout : Default: 60 seconds
• N3-REQUESTS : Default is 5.
• GetEchoConfiguration():  Returns the configuration that was made before.
• SetUDPSourcePortRange():   GTP-U is application on top of UDP.  Destination port of the GTP tunnel is standardized and it is 2152.  But the source port of each GTP-Tunnel to a given destination has to be unique.  Since UDP is also used by control plane, control plane may like to reserve a range for GTP-U. This function can be used to set that range.  COntrol plane is not expected to use ports in these port range for its purpose.  Parameters:
• Min source Port  : Default - 63000
• Max Source Port :  2^16 - 1
• Destination port:  Default 2152 (In case somebody wants to give some other port number during debugging time).
• GetUDPSourcePortRange():  To read the port range that was set using SetUDPSourcePortRange() function. 
• SetHousekeepingTimeOuts():  Parameters include
• Amount of time to wait after forwarding (for handover purposes) tunnels are created and before it gets indication to transfer the forwarded packets to the PDCP layer.  Valid at target eNB.
• Statistics:  There are no specific statistics defined by standards. I suggest to define the statistics for debugging.

Tunnel Establishment:

Tunnels are established by S1 and X2 protocols running in control plane. There are as many GTP tunnels as number of Radio Access Bearers.  It is good to visit the handover procedures before defining the interface for tunnels establishment as well as packet level API with PDCP.

Even though there are different types of handovers,  from GTP-U perspective they are same.  There are two types of handovers -  X2 based handovers and S1-based handovers.  if both source eNB and target eNB belongs to the same MME, then X2 based handover is used.  If there are different MMEs, then S1 handover would be used.  For more details about the handover, please see 3GPP TS 36.300 (Overall architecture) and 23.401.

Handover is applied for each RAB independently.   Typically only RABs that are setup using acknowledged mode do lossless handover. Unacknowledged mode RABs only do seemless handover. Control plane knows which RABs, in turn, which GTP-tunnels require handover and accordingly programs them in the user plane software in both PDCP and GTP-U layers. 

As part of lossless handover source eNB is expected to send unacknowledged downlink packets waiting at the  RLC layer and  new packets that are coming in from SGW.  Ofcourse, first GTP-U needs to send the unacknowledged PDCP packets and then the packets that come from the SGW.  In case of seemless handover only the new packets that are coming in from SGW are transferred to the target eNB.

Since PDCP packets already have sequence number associated with them by the PDCP layer, it is necessary that PDCP SN is sent along with the packet to the target eNB which is expected to send to the UE.   SGW sends the END-MARKET GTP packet to indicating the SGW will not be sending the packet to source eNB.  Target eNB is expected to send the packets with the PDCP SN first to the UE, then the packets which were sent by source eNB which don't have sequence number and then only it should start sending the packets which came from SGW to the target eNB.  Note that X2 or S1 protocol (based on whether X2 or S1 handover) create GTP tunnels as part 'Handover Request Ack'. It is expected that source eNB receives the GTP tunnel ID for sending the pending downlink packets. 

GTP-U also is used to send the uplink packets that were received out of sequence by eNB in PDCP layer.  These packets are also can be sent to the target eNB.  PDCP status transfer message that is sent to the UE as part of handover execution tells the UE the sequence numbers of the packets which were received out-of-order.  UE then accordingly only has to do selective retransmissions and thereby saving air bandwidth.  In any case, source eNB sends these out-of-order received packets to the target eNB with the sequence numbers.  The tunnel it uses to send these packets is new tunnel created for this purpose. This TEID of the tunnel is also sent along with Handover request Ack by the target eNB.

Normally each tunnel has both self TEID and peer TEID.  But in case of tunnels that are setup for forwarding the packets  have peer TEID with respect to source eNB and self TEID with respect to target eNB.

• GTPUCreateTunnel() :  This function is used to create normal tunnels as well as forwarding tunnels during handover.  Control plane creates as many tunnels as number of DRBs (Dedicated Resource Bearers).  Parameters include:
• CP Reference Identification:  Control plane can put its reference which is expected to be sent along with the indications.
• Virtual Instance ID : in case of multiple operators sharing the same eNB hardware.
• Identification parameters in uplink direction:
• Sector ID,  C-RNTI and Logical Channel ID.
• Tunnel Type :  Normal,  Forwarding-downlink-sourceNBside,  forwarding-downlink-targetNBside,  forwarding-uplink-sourceeNB and forwarding-uplink-targeteNB.
• Normal :  Indicates normal tunnel established between eNB and SGW.
• Forwarding-downlink-sourceeNB :  Indicates that this tunnel is being created at source eNB for forwarding pending downlink packets. GTP-U in source eNB would start expecting the PDCP layer to give all downlink packets which were not acknowledged followed by local end-marker  along with the last packet.  GTP-U knows from the end-marker that no more packets should be expected from the local PDCP layer. Even if there are no packets are to be forwarded,  GTP-U expects the PDCP layer to indicate end-marker.
• Forwarding-uplink-sourceeNB:  Indicates that this tunnel is being created at source eNB for forwarding the out-of-order uplink packets.  GTP-U in source enB would start expecting the PDCP layer to give all uplink packes that came in out-of-order followed by local end-marker along with the last packet.  Even if there are no packet to be forwarded,  GTP-U expects PDCP layer to give end-marker. 
• Forwarding-downlink-targeteNB: Indicates that this tunnel is being creates at target eNB for receiving the forwarded downlink packets from source eNB.
• Forwarding-uplink-targeteNB:  Indicates that this tunnel is being creates at target eNB for receiving the out-of-sequence uplink packets at the sourceeNB.
• Identification parameters in downlink direction:
• TEID Self (Only if tunnel type is normal,  forwarding-downlink-targeteNB and forwarding-uplink-targeteNB)
• local IP address (Could be IPv4 or IPv6)
• Peer IP address (Could be IPv4 or IPv6)
•  TEID Peer: Required to send the packets to the peer. Only required if tunnel type is Normal,  forwarding-downlink-sourceeNB and forwarding-uplink-sourceeNB.
• DSCP Marking:  Different RABs are created due to different QoS treatment required.  QoS treatment should be not only limited to MAC scheduler enforcement, but also to be enforced on uplink traffic before sending the packets out onto the backhaul network.  Normally the DSCP value of the IP packets coming from the UE would have right value. In case, they don't send the right value, then it can be set here.  QoS shaping module at the egress level (towards backhaul network) would use the DSCP value of the IP packet to provide differential treatment to the packets. In case of IPsec layer between GTP and QoS,  IPsec is normally configured to copy the DSCP value from inner IP header to IPsec outer IP header there by marked DSCP value is preserved all the way to the QoS module. 
• Active flag: Indicates whether this tunnel is active.  This flag is required due to 'Multiple Preparation' feature.  Source eNB may initiate handover procedure with multiple target eNBs, but only one will be activated eventually.
•  GTPUTargeteNBStartXmitOfForwardedPkts():  This functions will be used by control plane to initiate the transmission of forwarded packets. This function is called on target eNB.  When control plane (RRC) receives the 'RRCReconfigurationComplete' message from the UE by target eNB,  it is expected that this function is called  after the control plane programs the PDCP layer with the information it receives as part of SN-STATUS_TRANSFER.  GTP-U on targeteNB is expected to keep the forwarded packets  until this function is called. It is also expected that GTP-U layer of target eNB sends the packets with sequence number first before sending the forwarded non-seq packts and then the new packets which are coming to target eNB directly from SGW.   Note that when this function is called GTP-U layer of target eNB can send both DL and UL packets to the PDCP layer.  I guess UL packets would be used by PDCP layer for sending status transfer PDCP control message to the UE.  These packets will be eventually would be given back to the GTP layer in order to the GTP tunnel which is created with new SGW (Normal tunnel).  
• Parameters include:
• Identification parameters such as Virtual Instance ID, Sector ID,  C-RNTI, LCI and tunnel type.
• I think that there is no special function required in source eNB from control plane  to initiate handover.  Tunnel creation with appropriate tunnel-type itself is the indication that it can forward the packets. But PDCP user plane layer need to give the packets to the GTP-U layer. 
• GTPUChangeActiveStatus():  This function can be used by control plane to activate or deactivate an particular tunnel. This is required due to 'Multiple preparation' feature.  
• GTPU Do Hairpin Transfer:  This function is called by eNodeB control plane on original S1 tunnel to do the hairpin. This is expected to be done as part of processing HandoverComplete message from target eNodeB.  Once it is done,  GTP-U layer forwarding downlink packets over GTP tunnel established to the target eNB.
•  GTPUTerminateTunnel():  This function terminates the tunnel in the GTP-U.  Control plane (S1 and X2) call this function when it finds that this tunnel is no longer required. Parameters:
• Identification parameters such as Virtual Instance ID, Sector ID,  C-RNTI, LCI and tunnel type.
• GTPUGetTunnelInfo():  This function can be used by control plane to get the information and statistics of a given tunnel.  Parameters:
• Identification parameters such as Virtual Instance ID, Sector ID,  C-RNTI, LCI and tunnel type
• Output:  Statistics counters maintained on per tunnel basis and any state information.
• GTPUGetFirstTunnelInfo() and GTPUGetNextTunnelInfo():  Meant for walking through the tunnels.  Mainly for debugging and SNMP.
• xxxGTPUErrorIndication():  This function is expected to be called by GTP-U to indicate the peer provided error to the control plane (Error indication message received by the GTP-U).
• xxxGTPUPathErrorInd():  This function is expected to be called by GTP-U when it determines the peer is not alive.  As you might have seen, I did not define any function for control plane to initiate the path liveness checks.  Whenever the normal tunnel is created,  GTP-U is expected to create path management context to check the liveness of peer.  It is possible that there are more tunnels to the same peer. In that case, GTP-U should not be creating duplicate path management contexts.  When there are no tunnels to the peer,  path management context can be removed as there is no need to do liveness checks.  One way to do this in implementation is to have 'counter' (atomic) indicating the number of tunnels that have same peer.  This gets incremented whenever new tunnel is established for which path management context already exists.  It gets decremented when the tunnel is terminated.  When the count becomes 0, GTP-U can remove the context.

 Interfacing with PDCP:


GTP-U interfaces with PDCP to

• to send packets to the UE in downlink direction. 
• to get hold of packets from the PDCP in uplink direction 
• to get hold of packets which were not acknowledged in downlink direction. This is required in handover cases.  These packets would be sent by GTP-U to the target eNB.
• to get hold of packets which were received by PDCP from UE (uplink) out-of-order. This is required in handover cases. These packets would be sent by GTP-U to the target eNB.

   

• GTPUSendNormalPktToPDCP():  This function is used to send packets to the UE . Parameters may include
• Identification parameters such as Virtual Instance ID,  Sector ID, C-RNTI, LCI.
• packet in implementation specific buffer
• GTPUSendHOSeqPktToPDCP():  This function is used by GTP-U in target eNodeB. It is used to send the packets  with the sequence number. Parameters include:
• Identification parameters such as Virtual Instance ID,  Sector ID, C-RNTI, LCI.
• Direction of packet (Uplink or downlink) : Note that as part of handover both downlink and uplink packets are sent by source eNB.  Downlink packets are meant to be sent to the UE by PDCP layer.  Uplink packets are expected to be used by PDCP to reorder the packets with newly arrived uplink packets. Note that these uplink packets would be given to the GTP-U layer in sequence. Also note that these packets would be given to normal GTP-U tunnel which was established between target eNB and SGW.
• bValid PDCP SeqNum : Indicates whether the PDCP seq number is valid.
  
• PDCP seq number:   Sequence number is valid in handover scenarios.  This is normally done by the target eNB.  GTP-U implementation gets this from the extension header "PDCP PDU Number".
  
• packet in implementation specific buffer.
• Notes:  GTPUSendNormalPktToPDCP() and GTPUSendHOSeqPktToPDCP() could have been combined into one function. It is good to have two different functions for performance reasons. If combined both,  then number of arguments and preparation of arguments would take some CPU cycles which can be saved by having two different functions.
• GTPURecvNormalPktFromPDCP() :  This function is called by PDCP or some other glue layer to give packets to the GTP-U layer.  Parameters include:
• Identification parameters:  Virtual Instance ID, sector ID, C-RNTI, LCI.
• Note that there could be multiple GTP-U tunnel matching with these parameters. There would be only one tunnel to the SGW, but all other tunnels are X2 tunnels.  Since this function is called to send normal packets, it is expected that this information (normal) is used to select the right tunnel.
• Packet in implementation specific buffer.
• GTPURecvHOPktFromPDCP() :  This function is called by PDCP or some glue layer to give HO packets to the GTP-U layer.  This function is used by PDCP to give both UPLINK and DOWNLINK pending packets.   This function is called by source eNB.  Parameters include:
• Identification parameters:  
• Virtual Instance ID, Sector ID, C-RNTI, LCI
• Type of packet:  Downlink or uplink.
• Sequence number (PDCP sequence number): This will be sent to the peer using PDCP PDU number extension header.
• Some notes:  Due to 'Multiple Preparation' feature, there could be multiple forwarding tunnels in source eNB. But fortunately only one will be active.  It is expected that this 'active' information is used to match the right tunnel.

Some implementation notes:

• There would be two contexts that would need to be maintained in GTP-U - Tunnel contexts and path management contexts.   Tunnel contexts need to be arranged in two different hash tables - One for uplink and another for downlink.  
• Key parameters to the uplink hash table would be:
• Virtual instance ID,  Sector ID, C-RNTI, LCI
• Type (Normal, Handover)
• In case of handover,  Uplink or downlink,  Active/Inactive.
• Key parameters to the downlink hash table would be:
• IP type (IPv4, IPv6)
• Source IP and Destination IP from GTP-U packet.
• TEID from the GTP-U packet
• Some notes: It may be good to have two different hash tables - one for IPv4 and another for IPv6 for performance reasons.  Some thing to think about.

Performance considerations:

• Use RCUs during packet processing. Absolutely no locks during packet processing. Locks may be okay during tunnel establishment and termination.  That is why, RCUs are the best in multicore environments.  
• Multicore processors provide statistics accelerators. Use this functionality to increment/decrement the global counters and counters that may be updated across multiple cores.

Above scribbling is really my notes that was prepared while going through the GTP-U specifications.  I am new to 3GPP and it took me a solid one week to get handle. I hope it would be useful for you and hopefully jump start your study of GTP-U.

Linux Open source applications - Porting considerations for developers

You have your proprietary software in Linux and you have a need to integrate with open source applications. Let us see the items the developers need to keep in mind.

1. Selection of open source package :  Some cases, I find that many open source development projects on a given application.  Consider following to choose one project over another:
• Is the open source project actively maintained?
• Check the number of releases made so far.
• Latest release date
• Consistency of releases.
• Activity in the mailing list.
• Number of developers maintaining the project.
• Roadmap of features.
• Usage of this project in commercial or other open source projects.
• Code License
• Is this code GPL or BSD licensed? 
• Are you planning to add significant number of features to the open source application?  If you are, you better choose the BSD licensed code.
• If it is GPL code,  are the include/library files LGPL?  If not LGPL,  inclusion of the header file itself  in your code or linking with the library contaminates your code with GPL.
• If there is no BSD or LGPL code, then developers needs to be careful in using the package to ensure that their proprietary code is not contaminated with GPL.  If the code is modified, then developers have no choice other than making the code public. But integration of the open source package with rest of the software need to be ensured that the rest of the code is not contaminated.  More information on how to take care of this is described below.

Once the open source package is selected based on above criteria,  porting of the software to your platform and integration with rest of the software will be the next step.

Porting of the open source software to your platform:  Most of open source packages come with GNU configure script. For more details about configure script, please see here.  If you are compiling it for different target system, please ensure to provide right target type to the configure script.  Some times, configure script may not be written for your target type. Choose the ones closest and modify the configure script according to your requirement.   In addition also ensure to enable/disable features based on your requirement by passing the right arguments to the configure script. Based on the open source package, this may take a day or two. 

Integration with the Management system:  Many open source packages typically take configuration in a file.  When the package is started,  it reads from the configuration file and use it for its operations.   But any serious product provides user interface to the administrator such as CLI, Web GUI,  SNMP, NetConf or TR-069 etc using proprietary configuration & Management software.   The configuration of the open source package is expected to be done via this user interface. Typical activities involved in integrating with configuration & Management system are :

• Understanding the configuration of the open source package.
• Creating data model and associated GUI 
• Creation of backend logic to the configuration & Management system to convert data model elements to the open source package understandable format and store them in the file. Backend logic also should have specific way to inform the open source application daemon to read the file for the changed configuration to be effective.
• Many times, it is good to store the configuration data in your own format in addition to converting and storing that into the config file for open source daemon.  Storing it in your own format helps in 
• Reading the data model instance without reading the configuration file.
• Storing the user entered configuration along with rest of configuration of proprietary software.  This helps in import and export operation of complete device configuration. Otherwise, import & export operations involve many number of files which is little bit more complex.
• Helps in creation of configuration audit logs with right information easily (specifically when the some part of the configuration is modified - Note that it is expected that the modification operation log shows both old and new values.  New value will be known from user input, but old values need to be retrieved. Retrieval is easy and fast if the configuration is maintained in your own format in the memory).
• Helps in synchronizing the 'diff' configuration with the participant devices in high availability environments.  In HA environments, configuration created on master device is expected to be sent to all participant devices.  Since participant devices already have some configuration,  master device is expected to send only the 'diff'.  Again maintaining the configuration in your own format as rest of the configuration in memory would make your logic consistent.
• Basically use 'config' file of open source daemons as a way to communicate with the open source daemon. But maintain the configuration in your own format like you do for your proprietary software.
• If the open source package is GPL, then ensure that this backend logic does not include any files of open source package and don't use any GPL libraries to link with your backend logic. It is necessary to develop backend logic just by understanding the config file format. Otherwise, GPL contamination is possible and you would be forced to release your configuration & Management software public. 
• Showcasing the run time statistics in your configuration & management system:  Statistics counters are incremented by the open source daemon as part of its operation. Many open source daemons don't do much on statistics other than updating them. They expect somebody to get hold of the statistics. Configuration & Management system  is expected to show these statistics to the users via CLI/GUI/SNMP interfaces.  To get hold of statistics counters, some logic needs to be developed in open source daemon.  If the open source daemon has 'select/poll' kind of way of looking for events, then create a new socket (Unix sockets or loop back socket) and do appropriate binding.  Define some message header for clients to request specific information (command type).  Create the logic to wait on the socket (via poll) and act on based on command and send the response.  If the open source package is synchronous (that is no poll or select), then create a new thread which waits on the domain/loopback socket and act on the commands.  Client side of this mechanism will with your configuratin & management system. Basically your backend logic for statistics user interface will communicate with the open source daemon in synchronous fashion using domain/loop-back sockets.
• Integration with Logging/Alert software:  Every product has mechanism to show case the events that happened during its operation. Logging/Alert software normally format the logs and store them to SQL database or send them via syslog or email to management software. Many open source packages do have mechanism to log the events. But there is no standard unfortunately, that is each open source package does this differently.  This is one area of integration which is required to give uniform look and feel for administrators irrespective of origin of events.  One good thing is that many open source applications tunnel all the events via some fixed number of API functions.  These API functions need to be changed to send the events to your logging/alert system.  To avoid GPL contamination, ensure that the your logging/alert software is a daemon by itself.  Add whatever minimal software that is required to   bridge the open source logging API  to communicate with your logging/alerting system. Please note that this glue logic becomes open source and be prepared to release it when requested by others. 

Advice to Network Device testers - Simulate Capacity/Stress related faults

Capacity in network devices such as UTM is specified with respect to simultaneous connections in case of firewall, ALGs, intrusion Prevention functionality, tunnels in case of IPsec VPN, number of sessions in case of Anti Virus and Spam functions and many more related to different smaller functions. All the functions are not normally used at the same time. Even if all functions are used at the same time, all sessions may not be going through all functions. Due to this, network device vendors typically oversubscribe memory. That is, the memory needed for all functions for the specif ed capacity would be lot more than the memory available in the devices.

This could pose interesting problem in the field. If there is a deployment where multiple functions are used by large number of connections, there could be memory shortage and other resources shortage. This leads to error being returned when the resource is being allocated. If error detection, propagation and recovery is not taken care well by the software, this could lead to instability, leaks, crashes and lockups. It is tester job to ensure that these kinds of problems do not happen in the field. Typically testers simulate different conditions and ensure that system is stable. Yet times, it is not possible for testers to test all different combinations or simulate different conditions.

I believe testers should be able to simulate all possible combinations by simulating all kinds of error conditions. As part of it, testers should ask development team to provide facilities to inject the faults. In particular, testers should ask for facilities to inject faults for following.

• Memory allocation failures: Almost all functions in software would allocate memory either at the time of connection establishment or on packet basis or to queue the packets and control data etc... Testers should have ammunition to inject the memory fault for specific functions.
  
• Socket/File open failures
• Semaphore creation failures
• Thread/Tasklet creation failures
• Fault simulation of any other OS resource that gets allocated after software is completely initialized.

Testers should go at testing in methodical way:

• Keep list of all functions and OS resource allocations they do.
• For each one of them, create a test case.
• Before running the test case, configure to inject fault.
• Run the test and ensure that system works as expected.
• Run the test without fault and ensure that system is stable.
  

I believe that this kind of testing should happen for every release - feature or maintenance releases. If these tests are done manually, it takes very long time. My suggestion is to automate them.

RSYNC - Applicability for WAN Deduplication

The very old and good rsync can be key to WAN deduplication.

WAN Deduplication purpose is to reduce the amount of traffic on the WAN links by removing duplicate date. It is also called as DRE (Data Redundancy Elimination).

'rsync' is utility provided in Unix variants for a long time. It is mainly used to mirror the data across multiple servers and also is used to for backing up the data. It recursively goes through all files and directories and updates the data in the backup or mirror server. One feature that is interesting to WAN deduplication is its ability to do 'delta encoding'. 'rsync' has feature to send only 'differences' to the destination machine which inturn creates a new file from the existing file and the 'delta' information it gets from origin server. 'rsync' also can compress the data using 'zlib' and thereby saving even more WAN bandwidth.

To understand the algorithms used by 'rsync' for delta encoding, check this technical report. it uses mechanism called 'rolling checksum' (Alder32 algorithm) to figure out the differences between the file the mirror machine has and the file the origin sever has. Why is this 'rolling checksum' required? Note that when file gets updated by anybody in the origin server, the changes could be anywhere in the file. It can be in the beginning of the file, middle of the file or at the end of the file. 'Delta' generation should work and not duplicate any common data irrespective of placement of changes the original file has undergone. Mirror server breaks down the file it has into multiple chunks of some size (typically S = 1K), calculates both rolling checksum and MD5 checksum on the chunks. Then it sends them to the origin server. Origin Server does rolling checksum for chunks of size S. Since the file might have undergone changes, origin server creates rolling checksum for each byte offset. Note that mirror server generates rolling checksum for non-overlapping chunks. Since origin server generates large number checksums, rolling checksum algorithm needs to be very fast. Alder32 algorithm has property of generating checksum without going through the all bytes of the chunk and hence it is very fast. It can generate checksum incrementally. Then the received checksums are compared with the checksums generated locally by origin server. If any checksum matches, then it verifies by comparing with MD5 checksum. If MD5 checksum also matches, then origin server assumes that mirror server has this chunk. Once it finds out the all duplicate data, it only sends the matching block information and any new or modified data for non-matching chunks. For detailed information on this algorithm, please check this link.

Other utilities which are useful for WAN deduplication : 'rdiff'. 'rdiff' uses the rsync algorithm to generate delta file with the difference from old file and new file. Then this can be applied to old file at other machine to get the new file.

Firewall and NAT ALG Testing Recommendations

Overview:

Stateful inspection firewalls open temporary holes to allow data connections based on information it reads from the control connection. Some protocols such as FTP, SIP, RSTP, H.323, MGCP open a connection and exchange IP address, port information to peer end point for data transfer. Ports that are exchanged in control connections are not well known ports and they are ephemeral. Due to this, administrators can't configure firewall rules to allow these connections without allowing everything. Application Level Gateways (ALGs) are software modules within firewall interpret the protocol packets by extracting the ephemeral port information and open temporary holes to allow data connections to pass through the firewall between protocol end points. Since each protocol is different, multiple ALG modules are required - one for each protocol.

ALGs also do the address and port translation in the protocol data if firewall supports NAT functionality. If the IP address or ports are specified in ASCII form, there is a big possibility where the data length of the packet changes after translation. In case of TCP based ALG, this results into sequence number modifications in TCP header. Firewalls typically take care of maintaining the delta sequence numbers and modify further packets with this delta in both "Sequence number" and "Ack number" fields to ensure the integrity is maintained with client and sever end points of the connection. It is also important to note that ALGs modify different packets during the life of session and firewall software is expected to keep updating the delta sequence numbers appropriately. It is also to be noted that firewalls need to keep the history of delta numbers with respect to original sequence numbers to apply appropriate delta in case of retransmitted packets which are older.

To apply translation on the data, it is required that the ALG has complete PDU. In some protocols such as H.323 and SIP, this can be large. If there is congestion in the network, the end point does not send the PDU in one TCP packet and requires acknowledgment to send rest of PDU. Due to this, newer generation of firewalls send the acknowledgment to make the end point send rest of protocol data.

Many routers change the TCP MSS value of SYN and SYN+ACK packets of transit traffic to lower value whenever there is multimedia traffic to ensure that VOIP packets do not get stuck. As we all know, Voice traffic is delay sensitive and it should be transmitted as soon as possible. If routers has slow link then it takes significant time to transmit 1500 byte packets. If link bandwidth is 256kbps, it takes around 45msec to transmit 1500 byte packet. If WAN controller of the router chooses 1500 byte packet and if VOIP packet comes right after that, then VOIP packet may need to wait upto 45msec there by increasing the delay of real time traffic. By lowering the TCP MSS value, the size of TCP packets generated by end points can be controlled. Broadband routers setting the value of MSS value of transit TCP packets to 256 bytes are quite common. In these cases, the protocol data of complex protocols requiring ALG comes in many packets. Firewall and ALGs ensure to extract the relevant data for opening holes and translation even protocol data is coming in multiple TCP packets.

As discussed before, ALGs open temporary holes - pin holes. If ALGs are not implemented well, attackers can make control connection and send PDUs with data such a way that pin holes are created to access internal critical services. Also attacker can DoS the firewall by sending large number of PDUs which creates large number of pin holes there by causing service disruption to genuine users/connections.

ALG implementation can become very complex. Vulnerabilities increase with complexity. Buffer overflows, boundary conditions are typical problems associated with complex protocol implementation. That is one place validation should concentrate on.

Many protocols specifications (standards) don't specify maximum length of protocol messages - especially text based protocols such as SIP, HTTP etc.. Protocol implementations (end points and ALGs) typically assume the typical sizes while allocating buffers to buffer the data and don't allow the traffic if it exceeds this limit. Since these sizes are not universally adopted by different implementations, this could pose interoperability problems if ALG implementation assumption of size is different from end point implementations. This is one area validation should concentrate on. In my view ALG implementations should not assume any size restrictions for the PDUs which are not interpreted for its operation. For PDUs that are needed to be buffered, this size restriction should be as maximum as it can be. Some times the protocol messages is prepended with size information. In those cases, ALG implementations must allocate the buffer based on this size information. Validation should concentrate to ensure that ALG does not impose any problems in functionality.

As said before, ALGs also do the translations in the protocol data. ALG implementation should ensure that the translations happen for all three cases - Source NAT, Destination NAT and Source & Destination NAT. Many times validation Engineers concentrate on testing using one session. Many times problems related to NAT can't be found if only one session is used. Validation Engineers should test the ALG based firewall implementations with multiple sessions.


Recommendations:

As you can see, validation testing of ALGs is not as simple as running standard applications on both ends of firewall device. For example, running standrad FTP client and Server applications on two sides of firewall device and ensuring the file transfer succeeds is necessary, but not enough validation of FTP ALG. I recommend validation engineers to consider following for each ALG before certifying.

Functional testing:

• Test with standard applications: Make a list of popular applications. Ensure that different combination of applications as client and server succeed in following cases. Configure firewall to allow control connections (initial connections) only.
  
• Without NAT
  
• Source NAT : With clients behind internal network and servers in external network.
  
• with NAT IP address whose length in dotted decimal form is more than that of source IP address.
• with NAT IP address whose length in dotted decimal form is equal to length of source IP address.
• With NAT IP address whose length in dotted decimal form is less than that of source IP address.
• Destination NAT: With servers in internal network and clients in external network.
• DNAT IP address in dotted decimal is equal to length of Destination IP address in dotted decimal form.
• DNAT IP address is more in length than that of destination IP address in dotted decimal form.
• DNAT IP address is less in length than that of destination IP adderss in dotted decimal form.
• Source NAT and destination NAT together: With servers in internal network and clients in external network.
• Explore all options of applications and ensure that all options work with above NAT combination.
• Ensure that Private IP address (client IP address in case of SNAT, Destination IP address in case of DNAT) does not appear on the packets after translation. This requires capturing the packets and searching for IP address in both binary and dotted decimal form.
  
• Understand the protocol and get familiar with messages and fields and their lengths. If size is not mentioned in the protocol specification, get familiar with realistic maximum length of messages and fields. Test to ensure that ALG does not drop messages when messages with maximum sized fields are sent.
  
• Ensure that ALGs perform well even when there is temporary packet loss. FragRoute tool can be used to drop the packets. Ensure to test with all NAT combinations.
• Ensure that ALGs perform well when the TCP packet sizes are smaller than protocol messages. FragRoute tool can be used to change the TCP packet sizes. Ensure to test with all NAT combinations.
• Ensure that ALGs perform well when TCP packets with smaller size and reordered. FragRoute tool can be used to reorder the TCP segments. Ensure to test with all NAT combinations.

Negative Testing: This testing is required to ensure that systems don't crash when invalid packets are sent.

• Ensure that ALGs don't misbehave (crash or lockup) when protocol messages and fields of different lengths and values are sent. Make a note of all messages in the protocol specifications. Ensure to send messages of different length by writing your own client and server protocol or instrument the existing client and server implementation. I prefer later to reduce the effort required to do this kind of testing. It is relatively simple to generate messages and fields with different lengths and values for the first message of protocol. But for messages that are deep down the protocol require successful initial protocol phase. Hence I prefer going with instrumenting the existing open source client/server protocol implementations.
• Go through some of the common vulnerabilities found in client and server implementations by searching through the CERT repository. Since ALG is also interpreting the protocol messages, ensure that these kinds of vulnerabilities are not present in the ALG.
  

Stress testing: This is one important step to ensure that system under test can cope up with capacity specified. Also, it is important to ensure that system performs as per performance requirements.

• Use IXIA/SmartBits kind of tools to simulate large number of client and servers to ensure that system works as specified with respect to capacity.
• Use IXIA/SmartBits to test the connection rate and ensure it satisfies the specifications of the box.
• Use IXIA/SmartBits to test the throughput requirements.
• Use IXIA/SmartBits to test throughput and connection rate combination requirements.
• Repeat above test cases for 12 hours to ensure that the system is stable.

Asymmetric Routing & Proxy based appliance deployment

Though it is uncommon in small networks, asymmetric routing is not so uncommon in Medium to large networks.

Asymmetric routing is typically referred when the client to server packets take one path and server to client traffic take another path. It typically happens when there are multiple WAN links from Enterprise network to the ISP where the traffic from/to the ISP can come/go in any WAN link (packet level load balancing). It is also possible in Enterprise data centers where the machines are reachable from more than one router and servers are configured to send the packet to one router (via default gateway).

Proxy based appliances expects the full control of the connection traffic (both client to server and server to client traffic of any TCP or UDP session). If there is no asymmetric routing, these devices can be placed in line of traffic.

WCCPv2 (Web Caching Communication Protocol) can be used in cases where the routing is not symmetric. WCCP is widely implemented in routers and switches. If the routers where the asymmetric traffic is passing through, has WCCP capability, then this feature can be used to redirect the traffic to 'proxy based appliances'. WCCP feature allows routers to take traffic flow information from the WCCP client (proxy based appliances). Router then redirect the matching to the WCCP clients via GRE tunnel. WCCP also has a feature to return the traffic on GRE to the routers for further packet processing. The feature 'packet return' is optional, but in my view it is required for following reasons.

• Proxy appliances need not have sophisticated routing protocols.
• Router/switch which are redirecting the traffic can do processing on the packets in similar way whether or not the traffic is redirected, such as applying QoS policies, Firewall policies and any other processing that is configured on the router.
• Router load balancing features can be utilized on all traffic including redirected traffic. If proxy based appliance is doing its own forwarding, it may not have facility or visibility to balance the traffic across multiple WAN links.

One good thing about WCCP is that there is no change required in proxy applications in proxy appliances. WCCP is separate protocol which can be run as a separate daemon for control messages with the routers participating in the WCCP. The data traffic is sent on GRE. GRE implementation in Linux exposes it as a interface ('dev') similar to any link layer interface there by no change required in TCP/IP stack or applications running on TCP/IP stack. But, there is one thing that needs to be taken care if 'Packet return' method is chosen. Packets should be sent to the right redirected router. Note that the proxies do change the content and due to that the number of packets returned back would not be same as the number of packets which were redirected to it. Also, the content of packets also will not be same. Fortunately, WCCP and routers don't care about this.

How can the proxy appliances ensure to send the packets to the right router?

Due to asymmetric routing, for each connection there are two routers involved. One router sends the client to server traffic to Proxy appliance and second router passes server to client traffic to the appliance. Proxy appliance should ensure to the right traffic of the connection to the right router.

Proxies terminate the client initiated connections and make new connection to the server. Assuming that R1 router is sending the client traffic for a given TCP connection to the Proxy appliance and Router R2 is redirecting the traffic of TCP connection to the proxy appliance. That is, any traffic going to the client for this connection would be going through R2 and any traffic going to the actual server would be going via the router R1. So, proxy appliance when they come in picture should ensure the same behavior for the traffic. Proxy appliance is expected to keep the state and should give packets destined for client to R2 and destined to server to R1. This might require some changes to Linux Kernel TCP/IP stack or this can be implemented in IP Tables connection tracking module.

WCCPv2 protocol is documented at http://www.wrec.org/Drafts/draft-wilson-wrec-wccp-v2-00.txt.

For fear of this link vanishing in future, I have copied the text here.

INTERNET-DRAFT                                         M Cieslak
                     D Forster
                                                      G Tiwana
                                                      R Wilson
                                                      Cisco Systems
                                                      13 Jul 2000
                                                      Expires Jan 2001

                 Web Cache Coordination Protocol V2.0

Status of this Memo

  This document is an Internet-Draft and is in full conformance with all
  provisions of Section 10 of RFC2026.

  Internet-Drafts are working documents of the Internet Engineering Task
  Force (IETF), its areas, and its working groups. Note that other
  groups may also distribute working documents as Internet-Drafts.

  Internet-Drafts are draft documents valid for a maximum of six months
  and may be updated, replaced, or obsoleted by other documents at any
  time. It is inappropriate to use Internet-Drafts as reference material
  or to cite them other than as "work in progress".

  The list of current Internet-Drafts can be accessed at
  http://www.ietf.org/ietf/lid-abstracts.txt.

  The list of Internet-Draft Shadow Directories can be accessed at
  http://www.ietf.org/shadow.html.

1. Abstract

  This document describes version 2.0 of the Web Cache Coordination
  Protocol (WCCP). The WCCP V2.0 protocol specifies interactions between
  one or more routers and one or more web-caches. The purpose of the
  interaction is to establish and maintain the transparent redirection
  of selected types of traffic flowing through a group of routers. The
  selected traffic is redirected to a group of web-caches with the aim
  of optimising resource usage and lowering response times.

  The protocol does not specify any interaction between the web-caches
  within a group or between a web-cache and a web-server.

2. Definitions

  Assignment Method

     The method by which redirected packets are distributed between



                                                              [Page 1]

     web-caches.

  Designated Web-Cache.

     The web-cache in a web-cache farm responsible for dictating to the
     router or routers how redirected traffic should be distributed between
     the members of the farm.

  Forwarding Method

     The method by which redirected packets are transported from router to
     web-cache.

  Packet Return Method

     The method by which packets redirected to a web-cache are returned to
     a router for normal forwarding.

  Redirection Hash Table.

     A 256-bucket hash table maintained by the router or routers. This
     table maps the hash index derived from a packet to be redirected to
     the IP address of a destination web-cache.

  Service Group

     A group of one or more routers plus one or more web-caches working
     together in the redirection of traffic whose characteristics are part
     of the Service Group definition.

  Transparent Redirection.

     Transparent redirection is a technique used to deploy caching without
     the need for reconfiguration of clients or servers. It involves the
     interception and redirection of traffic to one or more web-caches by a
     router or switch transparently to the end points of the traffic flow.

  Usable Web-Cache.

     From the viewpoint of a router a web-cache is considered a usable
     member of a Service Group when it has sent that web-cache a
     WCCP2_I_SEE_YOU message and has received in response a WCCP2_HERE_I_AM
     message with a valid "Receive ID".

  Web-Cache Farm.

     One or more web-caches associated with a router or routers.




                                                              [Page 2]

3. Introduction

3.1 Protocol Overview

  WCCP V2.0 defines mechanisms to allow one or more routers enabled for
  transparent redirection to discover, verify, and advertise
  connectivity to one or more web-caches.

  Having established connectivity the routers and web-caches form
  Service Groups to handle the redirection of traffic whose
  characteristics are part of the Service Group definition.

  The protocol provides the means to negotiate the specific method
  used for load distribution among web-caches and also the method used
  to transport traffic between router and cache.

  A single web-cache within a Service Group is elected as the designated
  web-cache. It is the responsibility of the designated web-cache to
  provide routers with the data which determines how redirected traffic
  is distributed between the web-caches in the Service Group.

3.2 WCCP V2.0 enhancements

  WCCP V2.0 supports the following enhancements to the WCCP V1.0
  protocol.

  * Multi-Router Support.
  WCCP V2.0 allows a farm of web-caches to be attached to more than one
  router.

  * Multicast Support.
  WCCP V2.0 supports multicasting of protocol messages between
  web-caches and routers.

  * Improved Security.
  WCCP V2.0 provides optional authentication of protocol packets
  received by web-caches and routers.

  * Support for redirection of non-HTTP traffic.
  WCCP V2.0 supports the redirection of traffic other than HTTP traffic
  through the concept of Service Groups.

  * Packet return.
  WCCP V2.0 allows a web-cache to decline to service a redirected packet
  and to return it to a router to be forwarded. The method by which
  packets are returned to a router is negotiable.





                                                              [Page 3]

  * Alternative Hashing.
  WCCP V2.0 allows the designated web-cache to mark individual buckets
  in the Redirection Hash Table for a secondary hash. This allows the
  traffic being hashed to a particular bucket to be distributed across
  the members of a Service Group.

  * Multiple Forwarding Methods
  WCCP V2.0 allows individual web-caches to negotiate the method by
  which packets are forwarded to a web-cache from a router. Packets
  may now be forwarded unencapsulated using a Layer 2 destination
  address rewrite.

  * Multiple Assignment Methods
  WCCP V2.0 allows the designated web-cache to negotiate the method by which
  packets are distributed between the web-caches in a service group.
  Packets may now be assigned using a hashing scheme or a masking scheme.

  * Command and Status Information
  WCCP V2.0 includes a mechanism to allow a web-cache to pass a command
  to the routers in a Service Group. The same mechanism can be employed
  by the routers to pass status information to the web-caches in a
  Service Group.

4. Protocol Description

4.1 Joining a Service Group

  A web-cache joins and maintains its membership of a Service Group by
  transmitting a WCCP2_HERE_I_AM message to each router in the Group at
  HERE_I_AM_T (10) second intervals. This may be by unicast to each
  router or multicast to the configured Service Group multicast
  address. The Web Cache Info component in the WCCP2_HERE_I_AM message
  identifies the web-cache by IP address. The Service Info component of
  the WCCP2_HERE_I_AM message identifies and describes the Service Group in
  which the web-cache wishes to participate.

  A router responds to a WCCP2_HERE_I_AM message with a WCCP2_I_SEE_YOU
  message. If the WCCP2_HERE_I_AM message was unicast then the router will
  respond immediately with a unicast WCCP2_I_SEE_YOU message. If the
  WCCP2_HERE_I_AM message was multicast the router will respond via the
  scheduled multicast WCCP2_I_SEE_YOU message for the Service Group.

  A router responds to multicast web-cache members of a Service Group
  using a multicast WCCP2_I_SEE_YOU message transmitted at 9 second
  intervals with a 10% jitter.

  The Router Identity component in a WCCP2_I_SEE_YOU message includes a list
  of the web-caches to which the packet is addressed. A web-cache not



                                                              [Page 4]

  in the list should discard the WCCP2_I_SEE_YOU message.

4.2 Describing a Service Group

  The Service Info component of a WCCP2_HERE_I_AM message describes the
  Service Group in which a web-cache wishes to participate. A Service
  Group is identified by Service Type and Service ID. There are two
  types of Service Group:

  * Well Known Services
  * Dynamic Services.

  Well Known Services are known by both routers and web-caches and do
  not require a description other than a Service ID.

  In contrast Dynamic Services must be described to a router. A router
  may be configured to participate in a particular Dynamic Service
  Group, identified by Service ID, without any knowledge of the
  characteristics of the traffic associated with the Service Group. The
  traffic description is communicated to the router in the
  WCCP2_HERE_I_AM message of the first web-cache to join the Service
  Group. A web-cache describes a Dynamic Service using the Protocol,
  Service Flags and Port fields of the Service Info component. Once a
  Dynamic Service has been defined a router will discard any subsequent
  WCCP2_HERE_I_AM message which contains a conflicting description. A
  router will also discard a WCCP2_HERE_I_AM message which describes a
  Service Group for which the router has not been configured.

4.3 Establishing Two-Way Connectivity

  WCCP V2.0 uses a "Receive ID" to verify two-way connectivity between a
  router and a web-cache.  The Router Identity Info component of a
  WCCP2_I_SEE_YOU message contains a "Receive ID" field. This field is
  maintained separately for each Service Group and its value is
  incremented each time the router sends a WCCP2_I_SEE_YOU message to
  the Service Group.

  The "Receive ID" sent by a router is reflected back by a web-cache in
  the Web-Cache View Info component of a WCCP2_HERE_I_AM message. A
  router checks the value of the "Receive ID" in each WCCP2_HERE_I_AM
  message received from a Service Group member. If the value does not
  match the "Receive ID" in the last WCCP2_I_SEE_YOU message sent to
  that member the message is discarded.

  A router considers a web-cache to be a usable member of a Service
  Group only after it has sent that web-cache a WCCP2_I_SEE_YOU message
  and received a WCCP2_HERE_I_AM message with a valid "Receive ID" in
  response.



                                                              [Page 5]

4.4 Negotiating the Forwarding Method

  A web-cache and router may negotiate the method by which packets are
  forwarded to the web-cache by the router.

  This negotiation is per web-cache, per Service Group. Thus web-caches
  participating in the same Service Group may negotiate different
  forwarding methods with the Service Group routers.

  A router will advertise the supported forwarding methods for a Service
  Group using the optional Capabilities Info component of the
  WCCP2_I_SEE_YOU message. The absence of such an advertisement implies
  the router supports the default GRE encapsulation method only.

  A web-cache will inspect the forwarding method advertisement in the
  first WCCP2_I_SEE_YOU message received from a router for a particular
  Service Group. If the router does not advertise a method supported by
  the web-cache then the web-cache will abort its attempt to join the
  Service Group. Otherwise the web-cache will pick one method from those
  advertised by the router and specify that in the optional Capabilities
  Info component of its next WCCP2_HERE_I_AM message. Absence of a
  forwarding method advertisement in a WCCP2_HERE_I_AM message implies
  the cache is requesting the default GRE encapsulation method.

  A router will inspect the forwarding method selected by a web-cache in
  the WCCP2_HERE_I_AM message received in response to a WCCP2_I_SEE_YOU
  message. If the selected method is not supported by the router the
  router will ignore the WCCP2_HERE_I_AM message. If the forwarding
  method is supported the router will accept the web-cache as usable and
  add it to the Service Group.

4.5 Negotiating the Assignment Method

  A web-cache and router may negotiate the method by which packets are
  distributed between the web-caches in a Service Group.

  The negotiation is per Service. Thus web-caches participating in
  several Service Groups may negotiate a different assignment method for
  each Service Group.

  A router will advertise the supported assignment methods for a
  Service Group using the optional Capabilities Info component of the
  WCCP2_I_SEE_YOU message. The absence of such an advertisement implies
  the router supports the default Hash assignment method only.

  A web-cache will inspect the assignment method advertisement in the
  first WCCP2_I_SEE_YOU message received from a router for the Service
  Group. If the router does not advertise a method supported by the



                                                              [Page 6]

  web-cache then the web-cache will abort its attempt to join the
  Service Group. Otherwise the web-cache will pick one method from those
  advertised by the router and specify that in the optional Capabilities
  Info component of its next WCCP2_HERE_I_AM message. Absence of an
  assignment method advertisement in a WCCP2_HERE_I_AM message implies
  the cache is requesting the default Hash assignment method.

  A router will inspect the assignment method selected by a web-cache in
  the WCCP2_HERE_I_AM message received in response to a WCCP2_I_SEE_YOU
  message. If the selected method is not supported by the router the
  router will ignore the WCCP2_HERE_I_AM message. If the assignment
  method is supported the router will accept the web-cache as usable and
  add it to the Service Group.

4.5 Negotiating the Packet Return Method

  A web-cache and router may negotiate the method by which packets are
  returned from a web-cache to a router for normal forwarding.

  The negotiation is per Service. Thus web-caches participating in
  several Service Groups may negotiate a different packet return method
  for each Service Group.

  A router will advertise the supported packet return methods for a
  Service Group using the optional Capabilities Info component of the
  WCCP2_I_SEE_YOU message. The absence of such an advertisement implies
  the router supports the default GRE packet return method only.

  A web-cache will inspect the packet return method advertisement in the
  first WCCP2_I_SEE_YOU message received from a router for the Service
  Group. If the router does not advertise a method supported by the
  web-cache then the web-cache will abort its attempt to join the
  Service Group. Otherwise the web-cache will pick one method from those
  advertised by the router and specify that method in the optional
  Capabilities Info component of its next WCCP2_HERE_I_AM
  message. Absence of a packet return method advertisement in a
  WCCP2_HERE_I_AM message implies the cache is requesting the default
  GRE packet return method.

  A router will inspect the packet return method selected by a web-cache
  in the WCCP2_HERE_I_AM message received in response to a
  WCCP2_I_SEE_YOU message. If the selected method is not supported by
  the router the router will ignore the WCCP2_HERE_I_AM message. If the
  packet return method is supported the router will accept the web-cache
  as usable and add it to the Service Group.






                                                              [Page 7]

4.6 Advertising Views of the Service Group

  Each router advertises its view of a Service Group via the Router View
  Info component in the WCCP2_I_SEE_YOU message it sends to web-caches.
  This component includes a list of the useable web-caches in the
  Service Group as seen by the router and a list of the routers in the
  Service Group as reported in WCCP2_HERE_I_AM messages from
  web-caches. A change number in the component is incremented if the
  Service Group membership has changed since the last WCCP2_I_SEE_YOU
  message sent by the router.

  Each web-cache advertises its view of the Service Group via the Web
  Cache View Info component in the WCCP2_HERE_I_AM message it sends to
  routers in the Service Group. This component includes the list of
  routers that have sent the web-cache a WCCP2_I_SEE_YOU message and a
  list of web-caches learnt from the WCCP2_I_SEE_YOU messages. The Web
  Cache View Info component also includes a change number which is
  incremented each time Service Group membership information changes.

4.7 Security

  WCCP V2.0 provides a security component in each protocol message to
  allow simple authentication.  Two options are supported:

  * No Security (default)
  * MD5 password security

  MD5 password security requires that each router and web-cache wishing
  to join a Service Group be configured with the Service Group
  password. Each WCCP protocol packet sent by a router or web-cache for
  that Service Group will contain in its security component the MD5
  checksum of the WCCP protocol message (including the WCCP message
  header) and a Service Group password. Each web-cache or router in the
  Service Group will authenticate the security component in a received
  WCCP message immediately after validating the WCCP message header.
  Packets failing authentication will be discarded.

4.8 Distribution of Traffic Assignments

  WCCP V2.0 allows the traffic assignment method to be negotiated. There
  are two types of information to be communicated depending on the
  assignment method:

  * Hash Tables
  * Mask/Value Sets






                                                              [Page 8]

4.8.1 Hash Tables

  When using hash assignment each router uses a 256-bucket Redirection
  Hash Table to distribute traffic for a Service Group across the member
  web-caches. It is the responsibility of the Service Group's designated
  web-cache to assign each router's Redirection Hash Table.

  The designated web-cache uses a WCCP2_REDIRECT_ASSIGNMENT message to
  assign the routers' Redirection Hash Tables. This message is
  generated following a change in Service Group membership and is sent
  to the same set of addresses to which the web-cache sends WCCP2_HERE_I_AM
  messages. The designated web-cache will wait 1.5 HERE_I_AM_T
  seconds following a change before generating the message in order to
  allow the Service Group membership to stabilise.

  The Redirection Hash Tables can be conveyed in either an Assignment
  Info Component or an Alternate Assignment Component within a
  WCCP2_REDIRECT_ASSIGNMENT.  Both components contain an Assignment
  Key. This will be reflected back to the designated web-cache in
  subsequent WCCP2_I_SEE_YOU messages from the routers in the Service
  Group. A WCCP2_REDIRECT_ASSIGNMENT may be repeated after HERE_I_AM_T
  seconds if inspection of WCCP2_I_SEE_YOU messages indicates a router
  has not received an assignment.

  A router will flush its Redirection Hash Table if a
  WCCP2_REDIRECT_ASSIGNMENT is not received within 5 HERE_I_AM_T seconds
  of a Service Group membership change.  A router will flush its
  Redirection Hash Table if it receives a WCCP2_REDIRECT_ASSIGNMENT
  message in which it is not listed.

  The designated web-cache lists the web-caches to which traffic should
  be distributed in either an Assignment Info Component or an Alternate
  Assignment Component within a WCCP2_REDIRECT_ASSIGNMENT message. Only
  those web-caches seen by every router in the Service Group are
  included.

4.8.2 Mask/Value Sets

  When using mask assignment each router uses masks and a table of
  values to distribute traffic for a Service Group across the member
  web-caches. It is the responsibility of the Service Group's designated
  web-cache to assign each router's mask/value sets.

  The designated web-cache uses the Alternate Assignment Component in a
  WCCP2_REDIRECT_ASSIGNMENT message to assign the routers' mask/value
  set. This message is generated following a change in Service Group
  membership and is sent to the same set of addresses to which the
  web-cache sends WCCP2_HERE_I_AM messages. The designated web-cache



                                                              [Page 9]

  will wait 1.5 HERE_I_AM_T seconds following a change before generating
  the message in order to allow the Service Group membership to
  stabilise.

  The Alternate Assignment Info component of the
  WCCP2_REDIRECT_ASSIGNMENT contains an Assignment Key. This will be
  reflected back to the designated web-cache in subsequent
  WCCP2_I_SEE_YOU messages from the routers in the Service Group. A
  WCCP2_REDIRECT_ASSIGNMENT message may be repeated after HERE_I_AM_T
  seconds if inspection of WCCP2_I_SEE_YOU messages indicates a router
  has not received an assignment.

  A router will flush its mask/value set if a WCCP2_REDIRECT_ASSIGNMENT
  is not received within 5 HERE_I_AM_T seconds of a Service Group
  membership change. A router will flush its mask/value set if it
  receives a WCCP2_REDIRECT_ASSIGNMENT in which it is not listed.

  The designated web-cache lists the web-caches to which traffic should
  be distributed in the Alternate Assignment Info component of the
  WCCP2_REDIRECT_ASSIGNMENT message. Only those web-caches seen by every
  router in the Service Group are included.

4.9  Electing the Designated Web-cache

  Election of the designated web-cache will take place once a Service
  Group membership has stabilised following a change. The designated
  web-cache must be receiving a WCCP2_I_SEE_YOU message from every
  router in the Service Group.

  Election of the designated web-cache is not part of the WCCP
  protocol. However it is recommended that the web-cache with the lowest
  IP address is selected as designated web-cache for a Service Group.

4.10 Traffic Interception

  A router will check packets passing through it against its set of
  Service Group descriptions. The Service Group descriptions are
  checked in priority order. A packet which matches a Service Group
  description is a candidate for redirection to a web-cache in the
  Service Group.

  A router will not redirect a packet with a source IP address matching
  any web-cache in the Service Group.








                                                             [Page 10]

4.11 Traffic Redirection

4.11.1 Redirection with Hash Assignment

  Redirection with hash assignment is a two-stage process. In the first
  stage a primary key is formed from the packet (as defined by the
  Service Group description) and hashed to yield an index into the
  Redirection Hash Table.

  If the Redirection Hash Table entry contains an unflagged web-cache
  index then the packet is redirected to that web-cache. If the bucket
  is unassigned the packet is forwarded normally. If the bucket is
  flagged as requiring a secondary hash then a secondary key is formed
  (as defined by the Service Group description) and hashed to yield an
  index into the Redirection Hash Table. If the secondary entry contains
  a web-cache index then the packet is directed to that web-cache. If the
  entry is unassigned the packet is forwarded normally.

4.11.2 Redirection with Mask Assignment

  The first step in redirection using the mask assignment method is to
  perform a bitwise AND operation between the mask from the first
  mask/value set in the Service Group definition and the contents of the
  packet. The output of this operation is the set of fields in the packet
  which will be used for value matching. The selected fields from the
  packet are then compared against each entry in the list of values for
  that mask/value set. If a match is found the packet is redirected to
  the web-cache associated with the value entry. If no match is found
  the process is repeated for each mask/value set defined for the
  Service Group.  If, after trying all of the mask/value sets defined
  for the Service Group, no match is found, the packet is forwarded
  normally.

  Mask/value sets are processed in the order in which they are
  presented in the Alternate Assignment component. Value elements are
  compared in the order in which they appear in the mask/value set of which
  they are part.

4.12 Traffic Forwarding

  WCCP allows the negotiation of the forwarding method between router
  and web-cache (See Negotiating the Forwarding Method). The currently
  defined forwarding methods are:

  * GRE Encapsulated
  * Unencapsulated with L2 rewrite





                                                             [Page 11]

4.12.1 Forwarding with GRE Encapsulation

  Redirected packets are encapsulated in a new IP packet with a GRE [1]
  header followed by a four-octet Redirect header.

  The GRE encapsulation uses the simple four-octet GRE header with the
  two Flags and Version octets set to zero and a Protocol Type of
  0x883E.

  The Redirect header is as follows:

  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |D|A| Reserved  |  Service ID   |  Alt Bucket   |  Pri Bucket   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  D       Dynamic Service
          0: Well known service
          1: Dynamic service

  A       Alternative bucket used
          0: Primary bucket used
          1: Alternative bucket used

  Service ID

     Service Group identifier

  Alt Bucket

     Alternative bucket index used to redirect the packet. Only valid
     for hash assignment.

  Pri Bucket

     Primary bucket index used to redirect the packet. Only valid for hash
     assignment.

4.12.2 Forwarding with L2 Rewrite

  Redirected packets are not encapsulated. The router replaces the
  packet's destination MAC address with the MAC address of the target
  web-cache.

  This forwarding method requires that the target web-cache
  be directly-connected to the router at Layer 2. A router will not
  allow a web-cache which is not directly attached to negotiate this
  forwarding method.




                                                             [Page 12]

4.13 Packet Return

  WCCP V2.0 allows a web-cache to decline a redirected packet and return
  it to a router for normal forwarding as specified by the packet's
  destination IP address. The method by which packets are returned from
  router to cache is a matter for negotiation (see Negotiating the
  Packet Return Method).

  When a router receives a returned packet it must not attempt to
  redirect that packet back to a web-cache. Two methods are available to
  prevent any further redirection:

  * Interface Configuration
  * Encapsulation

  The interface configuration method requires that a router is
  configured to inhibit redirection of packets arriving over interfaces
  connected to web-caches. Redirection may be disabled for all packets
  arriving on an interface or for packets where the source MAC
  address is that of a web-cache. This mechanism is efficient but is
  topology dependant and thus may not always be suitable. In this case
  the packet return method in use is L2.

  The encapsulation method requires a web-cache to send returned packets
  to a router with encapsulation. Returned packets are encapsulated in a
  GRE packet [1] with a Protocol Type of 0x883E and contain the original
  Redirect Header or a null Redirect Header if none was present in the
  original redirected packet. The receiving router removes the GRE
  encapsulation from the packets and forwards them without attempting to
  redirect. The packet return method used in this case is GRE.

4.14 Querying Cache Time-Out

  If a router does not receive a WCCP2_HERE_I_AM message from a Service
  Group member for 2.5 * HERE_I_AM_T seconds it will query the member by
  unicasting a WCCP2_REMOVAL_QUERY message to it.  The target Service
  Group member should respond by sending a series of 3 identical
  WCCP2_HERE_I_AM messages, each separated by HERE_I_AM_T/10 seconds.

  If a router does not receive a WCCP2_HERE_I_AM message from a Service
  Group member for 3 * HERE_I_AM_T seconds it will consider the member
  to be unusable and remove it from the Service Group. The web-cache
  will no longer appear in the Router View Info component of the
  WCCP2_I_SEE_YOU message.

  The web-cache will be purged from the assignment data for the Service
  Group.




                                                             [Page 13]

4.15 Command and Status Information

  WCCP V2.0 includes a mechanism to allow web-caches to send commands to
  routers within a service group. The same mechanism can be used by the
  routers to provide status information to web-caches.

  The mechanism is implemented by the Command Extension component. This
  component is included in the WCCP2_HERE_I_AM message from a web-cache
  passing commands to routers in a Service Group.

  If a router needs to send status information to a web-cache it will
  include a command in the Command Extension component within its own
  WCCP2_I_SEE_YOU message. That command will indicate the type of status
  information being carried.

5. Protocol Messages

  Each WCCP protocol message is carried in a UDP packet with a
  destination port of 2048. There are four WCCP V2.0 messages:

  * Here I AM
  * I See You
  * Redirect Assign
  * Removal Query

5.1 'Here I Am' Message

  +--------------------------------------+
  |         WCCP Message Header          |
  +--------------------------------------+
  |       Security Info Component        |
  +--------------------------------------+
  |        Service Info Component        |
  +--------------------------------------+
  |  Web-Cache Identity Info Component   |
  +--------------------------------------+
  |    Web-Cache View Info Component     |
  +--------------------------------------+
  | Capability Info Component (optional) |
  +--------------------------------------+
  |Command Extension Component (optional)|
  +--------------------------------------+









                                                             [Page 14]

5.2 'I See You' Message

  +--------------------------------------+
  |         WCCP Message Header          |
  +--------------------------------------+
  |       Security Info Component        |
  +--------------------------------------+
  |        Service Info Component        |
  +--------------------------------------+
  |    Router Identity Info Component    |
  +--------------------------------------+
  |      Router View Info Component      |
  +--------------------------------------+
  |       Assignment Info Component      |
  |                OR                    |
  |        Assignment Map Component      |
  +--------------------------------------+
  | Capability Info Component (optional) |
  +--------------------------------------+
  |Command Extension Component (optional)|
  +--------------------------------------+

5.3 'Redirect Assign' Message

  +--------------------------------------+
  |         WCCP Message Header          |
  +--------------------------------------+
  |       Security Info Component        |
  +--------------------------------------+
  |        Service Info Component        |
  +--------------------------------------+
  |      Assignment Info Component       |
  |                OR                    |
  |    Alternate Assignment Component    |
  +--------------------------------------+

5.4 'Removal Query' Message

  +--------------------------------------+
  |         WCCP Message Header          |
  +--------------------------------------+
  |      Security Info Component         |
  +--------------------------------------+
  |       Service Info Component         |
  +--------------------------------------+
  |     Router Query Info Component      |
  +--------------------------------------+




                                                             [Page 15]

5.5 WCCP Message Header

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                              Type                             |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Version             |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_HERE_I_AM (10)
     WCCP2_I_SEE_YOU (11)
     WCCP2_REDIRECT_ASSIGN (12)
     WCCP2_REMOVAL_QUERY (13)

  Version

     0x200

  Length

     Length of the WCCP message not including the WCCP Message Header.


5.6 Message Components

  Each WCCP message comprises a WCCP Message Header followed by a number of
  message components. The defined components are:

  * Security Info
  * Service Info
  * Router Identity Info
  * Web-Cache Identify Info
  * Router View Info
  * Web-Cache View Info
  * Assignment Info
  * Router Query Info
  * Capabilities Info
  * Alternate Assignment
  * Assignment Map
  * Command Extension

  Components are padded to align on a four-octet boundary. Each
  component has a 4-octet header specifying the component type and
  length. Note that the length value does not include the 4-octet
  component header.



                                                             [Page 16]

5.6.1 Security Info Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Type              |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Security Option                       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Security Implementation                   |
  |                               .                               |
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_SECURITY_INFO (0)

  Length

     Length of the remainder of the component.

  Security Option

     WCCP2_NO_SECURITY (0)
     WCCP2_MD5_SECURITY (1)

  Security Implementation

     If Security Option has the value WCCP2_NO_SECURITY then this field is
     not present. If Security Option has the value WCCP2_MD5_SECURITY this
     is a 16-octet field containing the MD5 checksum of the WCCP message and
     the Service Group password. The maximum password length is 8 octets.

     Prior to calculating the MD5 checksum the password should be padded
     out to 8 octets with trailing zeros and the Security Implementation
     field of the Security Option set to zero. The MD5 checksum is calculated
     using the 8 octet padded password and the WCCP message (including the
     WCCP Message Header).











                                                             [Page 17]

5.6.2 Service Info Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Type              |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | Service Type  |  Service ID   |  Priority     |  Protocol     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Service Flags                         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Port 0              |         Port 1                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Port 6              |         Port 7                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_SERVICE_INFO (1)

  Length

     Length of the remainder of the component.

  Service Type

     WCCP2_SERVICE_STANDARD (0).
     Service is a well known service and is described by the Service ID.
     All fields other than Service ID must be zero.

     WCCP2_SERVICE_DYNAMIC (1).
     Service is defined by the Protocol, Service Flags and Port fields.

  Service ID

     Service number. A number in the range 0-255. For well known services
     numbers in the range 0-50 are reserved. The numbers currently defined
     for well known services are:

     0x00    HTTP







                                                             [Page 18]

  Priority

     Service priority. The lowest priority is 0, the highest is
     255. Packets for redirection are matched against Services in priority
     order, highest first. Well known services have a priority of 240.

  Protocol

     IP protocol identifier

  Service Flags

     0x0001  Source IP Hash
     0x0002  Destination IP Hash
     0x0004  Source Port Hash
     0x0008  Destination Port Hash
     0x0010  Ports Defined.
     0x0020  Ports Source.
     0x0100  Source IP Alternative Hash
     0x0200  Destination IP Alternative Hash
     0x0400  Source Port Alternative Hash
     0x0800  Destination Port Alternative Hash

     The primary hash flags (Source IP Hash, Destination IP Hash, Source
     Port Hash, Destination Port Hash) determine the key which will be
     hashed to yield the Redirection Hash Table primary bucket index. If
     only the Destination IP Hash flag is set then the packet destination
     IP address is used as the key. Otherwise if any of the primary hash
     flags are set then the key is constructed by XORing the appropriate
     fields from the packet with the key (which has an initial value of
     zero).

     The key is hashed using the following algorithm:

         ulong hash = key;
         hash ^= hash >> 16;
         hash ^= hash >> 8;
         return(hash & 0xFF);

     If alternative hashing has been enabled for the primary bucket (see
     Assignment Info Component) the alternate hash flags (Source IP
     Alternative Hash, Destination IP Alternative Hash, Source Port
     Alternative Hash, Destination Port Alternative Hash) determine the
     key which will be hashed to yield a secondary bucket index. The key
     is constructed by XORing the appropriate fields from the packet with
     a key (which has an initial value of zero).





                                                             [Page 19]

  Port 0-7

     Zero terminated list of UDP or TCP port identifiers. Packets will be
     matched against this set of ports if the Ports Defined flag is set. If
     the Ports Source flag is set the port information refers to a source
     port, if clear the port information refers to a destination port.













































                                                             [Page 20]

5.6.3 Router Identity Info Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |               Type            |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Router ID Element                       |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Sent To Address                        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                      Number Received From                     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Received From Address 0                   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Received From Address n                   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_ROUTER_ID_INFO (2)

  Length

     Length of the remainder of the component.

  Router ID Element

     Element containing the router's identifying IP address and Receive
     ID. The IP address must be a valid, reachable address for the router.

  Sent To Address

     IP address to which the target web-cache sent the WCCP2_HERE_I_AM
     message. When this component is present in a unicast WCCP2_I_SEE_YOU
     message it will contain the IP address that the target web-cache
     used. When present in a multicast WCCP2_I_SEE_YOU message it will
     contain the Service Group multicast address.

  Number Received From

     The number of web-caches to which this message is directed. When using
     multicast addressing it may be less than the number of caches which



                                                             [Page 21]

     actually see the message.

  Received From Address 0-n

     List of the IP addresses of web-caches to which this message is
     directed. When using multicast addressing it may be a subset of the
     caches which actually see the message.












































                                                             [Page 22]

5.6.4 Web-Cache Identity Info Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |              Type             |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                  Web-Cache Identity Element                   |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_WC_ID_INFO (3)

  Length

     Length of the remainder of the component.

  Web-Cache Identity Element

     Element containing the web-cache IP address and Redirection Hash Table
     mapping.




























                                                             [Page 23]

5.6.5 Router View Info Component

  This represents a router's view of the Service Group.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |              Type             |         Length                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Member Change Number                      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Assignment Key                         |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Number of Routers                      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                            Router 0                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                            Router n                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Number of Web-Caches                    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                    Web-Cache Identity Element 0               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                    Web-Cache Identity Element n               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_RTR_VIEW_INFO (4)

  Length

     Length of the remainder of the component.

  Member Change Number

     Incremented each time there is a change in Service Group membership.



                                                             [Page 24]

  Assignment Key

     Assignment Key element received in the last WCCP2_REDIRECT_ASSIGNMENT
     message. Used by the designated web-cache to verify that an assignment
     has been executed.

  Number of Routers

     Number of routers in the Service Group

  Router 0-n

     IP addresses of routers in the Service Group. This list is constructed
     from routers reported by web-caches via WCCP2_HERE_I_AM messages. Note
     that a router does not include itself in the list unless it has also
     been reported via a WCCP2_HERE_I_AM message.

  Number of Web-Caches

     Number of useable web-caches in the Service Group

  Web-Cache Identity Element 0-n

     Identity elements of useable web-caches in Service Group. This list
     contains web-caches that have sent the router a WCCP2_HERE_I_AM
     message with a valid "Received ID".

























                                                             [Page 25]

5.6.6 Web Cache View Info Component

  This represents a web-cache's view of the Service Group.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type                |      Length                   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                          Change Number                        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Number of Routers                      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Router ID Element 0                    |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Router ID Element n                    |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Number of Web-Caches                    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Web Cache address 0                     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Web Cache address n                     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_WC_VIEW_INFO (5)

  Length

     Length of the remainder of the component.

  Change Number

     Incremented each time there is a change in the view.

  Number of Routers

     Number of routers in the Service Group




                                                             [Page 26]

  Router ID Element 0-n

     List of elements containing the identifying IP address for each router
     in the Service Group and the last "Received ID" from each.

  Number of Web-Caches

     Number of web-caches in the Service Group

  Web Cache address 0-n

     List of web-cache IP addresses learnt from WCCP2_I_SEE_YOU messages.







































                                                             [Page 27]

5.6.7 Assignment Info Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |              Type             |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Assignment Key                        |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Number of Routers                       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                    Router Assignment Element 0                |
  |                              .                                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                              .                                |
  |                              .                                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                    Router Assignment Element n                |
  |                              .                                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Number of Web-Caches                      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Web-Cache 0                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                              .                                |
  |                              .                                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Web-Cache n                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |    Bucket 0   |  Bucket 1     |   Bucket 2    |   Bucket 3    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |    Bucket 252 |  Bucket 253   |   Bucket 254  |   Bucket 255  |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_REDIRECT_ASSIGNMENT (6)

  Length

     Length of the remainder of the component.






                                                             [Page 28]

  Assignment Key

     The designated web-cache expects this element to be returned by a router
     in subsequent WCCP2_I_SEE_YOU messages.

  Number of Routers

     Number of routers reachable by the designated web-cache.

  Router Assignment Element 0-n

     Elements containing the router IP address, "Receive ID" and "Change
     Number" for each router.

  Number of Web-Caches

     Number of useable web-caches in the Service Group seen by all routers.

  Web Cache 0-n

     List of the IP addresses of useable web-caches in Service Group. The
     position of a web-cache identifier in this list is the web-cache
     index. The first entry in the list has an index of zero.

  Bucket 0-255

     Contents of the Redirection Hash Table. The content of each bucket is a
     web-cache index value in the range 0-31. If set the A flag indicates
     that alternative hashing should be used for this web-cache. The value
     0xFF indicates no web-cache has been assigned to the bucket.

      0 1 2 3 4 5 6 7
     +-+-+-+-+-+-+-+-+
     | Index       |A|
     +-+-+-+-+-+-+-+-+
















                                                             [Page 29]

5.6.8 Router Query Info Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Type              |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                           Router ID                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                           Receive ID                          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Sent To IP Address                        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Target IP Address                         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_QUERY_INFO (7)

  Length

     Length of the remainder of the component.

  Router ID

     Router IP address. The same address advertised in a WCCP2_I_SEE_YOU
     message.

  Receive ID

     Receive ID expected by the router.

  Sent To IP Address

     IP address to which the web-cache sent its last WCCP2_HERE_I_AM
     message. This will not be the Router ID if the web-cache is
     multicasting its WCCP2_HERE_I_AM messages.

  Target IP Address

     IP address of web-cache being queried.









                                                             [Page 30]

5.6.9 Capabilities Info Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Type              |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Capability Element 0                      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Capability Element n                      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_CAPABILITY_INFO (8)

  Length

     Length of the remainder of the component.

  Capability Element

     Element in Type-Length-Value format (TLV) describing a router or
     web-cache capability.
























                                                             [Page 31]

5.6.10 Alternate Assignment Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |              Type             |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |       Assignment Type         |     Assignment Length         |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Assignment Body                         |
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_ALT_ASSIGNMENT (13)

  Length

     Length of the remainder of the component.

  Assignment Type

     Currently defined values:

     WCCP2_HASH_ASSIGNMENT           (0x00)
     WCCP2_MASK_ASSIGNMENT           (0x01)

  Assignment Length

     Length of Assignment Body

  Assignment Body

     The format of Assignment Body depends upon the value of Assignment Type.

     Assignment Type = WCCP2_HASH_ASSIGNMENT

     In this case the body of the message is identical to the Assignment
     Info Component with the Type and Length fields omitted.










                                                             [Page 32]

     Assignment Type = WCCP2_MASK_ASSIGNMENT

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                         Assignment Key                        |
     |                               .                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                       Number of Routers                       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Router Assignment Element 0                |
     |                              .                                |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                              .                                |
     |                              .                                |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Router Assignment Element n                |
     |                              .                                |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             Number of Mask/Value Set Elements (m)             |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                    Mask/Value Set Element 0                   |
     |                              .                                |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                               .                               |
     |                               .                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                   Mask/Value Set Element m                    |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     Assignment Key

        The designated web-cache expects this element to be returned by a
        router in subsequent WCCP2_I_SEE_YOU messages.

     Number of Routers

        Number of routers reachable by the designated web-cache.

     Router Assignment Element 0-n

        Element containing the router IP address, Receive ID and Change
        Number for each router.

     Number of Mask/Value Set Elements (m)

        Number of Mask/Value Set elements in this message




                                                             [Page 33]

     Mask/Value Set Element 0-m

        A list of the Mask/Value Element Sets for the Service Group
















































                                                             [Page 34]

5.6.11 Assignment Map Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type                |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Number of Mask/Value Set Elements (n)             |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                    Mask/Value Set Element 0                   |
  |                              .                                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                   Mask/Value Set Element n                    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_ASSIGN_MAP (14)

  Length

     Length of the remainder of the component.

  Number of Mask/Value Set Elements (n)

     Number of Mask/Value Set elements in the message

  Mask/Value Set Element 0-n

     A list of the Mask/Value Element Sets for the Service Group


















                                                             [Page 35]

5.6.12 Command Extension Component

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |           Type                |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |       Command Type            |      Command Length           !
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Command Data                           |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     WCCP2_COMMAND_EXTENSION (15)

  Length

     Length of the remainder of the component.

  Command Type

     The command specifier.

  Command Length

     The length of the Command Data field of this command

  The defined Command Types are:

  Command Type:   WCCP2_COMMAND_TYPE_SHUTDOWN (01)
  Command Length: 4
  Command Data:   Web-cache IP address
  Description:    This command is used by a web-cache to indicate to
                  the routers in a Service Group that it is shutting
                  down and should no longer receive any redirected traffic.


  Command Type:   WCCP2_COMMAND_TYPE_SHUTDOWN_RESPONSE (02)
  Command Length: 4
  Command Data:   Web-cache IP address.
  Description:    This command is used by a router to acknowledge
                  receipt of a SHUTDOWN command received from the web-cache
                  identified by the IP address in the Command Data field.




                                                             [Page 36]

5.7 Information Elements

5.7.1 Router ID Element

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Router ID                             |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Receive ID                            |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Router ID

     Router's identifying IP address. This must be a valid IP address by
     which the router is reachable.

  Receive ID

     Defined per Service Group. Incremented each time the router sends a WCCP
     protocol message including a Router Identity Info component. Will never be
     zero.

5.7.2 Web-Cache Identity Element

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         WC Address                            |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |        Hash Revision          |U|        Reserved             |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                          Bucket Block 0                       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                               .                               |
  |                               .                               |
  |                               .                               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                          Bucket Block 7                       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |    Assignment Weight          |           Status              |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  WC Address

     Web-Cache IP address





                                                             [Page 37]

  Hash Revision

     0x00

  U

     If set indicates that the web cache does not have an assignment in the
     Redirection Hash Table and that Bucket Block data is historical.
     Historical data may be used by the designated web-cache to re-assign
     the same bucket set to a web-cache that left and subsequently
     rejoined a Service Group.

  Bucket Block 0-7

     256-bit vector. A set bit indicates the corresponding Redirection
     Hash Table bucket is assigned to this web-cache.

  Assignment Weight

     Hash weight. May be used to indicate to the designated web-cache how new
     assignments should be made.

  Status

     Hash status. May be used to indicate to the designated web-cache how new
     assignments should be made.

5.7.3 Assignment Key Element

  This element identifies a particular assignment.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                       Key IP Address                          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                      Key Change Number                        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Key IP Address

     Designated web-cache IP address

  Key Change Number

     Incremented if a change has occurred.





                                                             [Page 38]

5.7.4 Router Assignment Element

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Router ID                             |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Receive ID                            |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Change Number                          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Router ID

     Router's identifying IP address. It must be a valid address by which
     the router is reachable.

  Receive ID

     Last Receive ID received from the router identified by Router
     ID. A router will ignore an assignment if Receive ID is invalid.

  Change Number

     Last Member Change Number received from the router identified by
     Router ID. A router will ignore an assignment if Change Number is
     invalid.

5.7.5 Capability Element

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |             Type              |          Length               |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                             Value                             |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Type

     Currently defined types are:

     WCCP2_FORWARDING_METHOD     0x01
     WCCP2_ASSIGNMENT_METHOD     0x02
     WCCP2_PACKET_RETURN_METHOD  0x03






                                                             [Page 39]

  Length

     Length of Capability element Value

  Value

     The length and format of the value field is dependant on the capability type.

     Type = WCCP2_FORWARDING_METHOD

     A 32-bit bitmask indicating supported/selected forwarding methods.
     Currently defined values are:

     WCCP2_FORWARDING_METHOD_GRE     0x00000001
     WCCP2_FORWARDING_METHOD_L2      0x00000002

     Type = WCCP2_ASSIGNMENT_METHOD

     A 32-bit bitmask indicating supported/selected assignment methods.
     Currently defined values are:

     WCCP2_ASSIGNMENT_METHOD_HASH    0x00000001
     WCCP2_ASSIGNEMNT_METHOD_MASK    0x00000002

     Type = WCCP2_PACKET_RETURN_METHOD

     A 32-bit bitmask indicating supported/selected packet return methods.
     Currently defined values are:

     WCCP2_PACKET_RETURN_METHOD_GRE  0x00000001
     WCCP2_PACKET_RETURN_METHOD_L2   0x00000002

5.7.6 Mask/Value Set Element

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                         Mask Element                          |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                   Number of Value Elements (n)                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Value Element 0                        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                              .                                |
  |                              .                                |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                        Value Element n                        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+



                                                             [Page 40]

  Mask Element

     Mask element for this set.

  Number of Value Elements (n)

     The number of value elements in this set.

  Value Element 0-n

     The list of value elements for this set.

5.7.7 Mask Element

  Note that in all of the mask fields of this element a zero means
  "Don't care".

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Source Address Mask                       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                   Destination Address Mask                    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |      Source Port Mask         |   Destination Port Mask       |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Source Address Mask

     The 32 bit mask to be applied to the source IP address of the packet.

  Destination Address Mask

     The 32 bit mask to be applied to the destination IP address of the packet.

  Source Port Mask

     The 16 bit mask to be applied to the TCP/UDP source port field of the packet.

  Destination Port Mask

     The 16 bit mask to be applied to the TCP/UDP destination port field of the packet.









                                                             [Page 41]

5.7.8 Value Element

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                     Source Address Value                      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                  Destination Address Value                    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |      Source Port Value        |   Destination Port Value      |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |                   Web Cache IP Address                        |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Source Address Value

     The value to match against the source IP address of the packet after
     masking.

  Destination Address Value

     The value to match against the destination IP address of the packet after
     masking.

  Source Port Value

     The value to match against the TCP/UDP source port number of the
     packet after masking.

  Destination Port Value

     The value to match against the TCP/UDP destination port number of the
     packet after masking.

  Web-cache IP address

     The IP address of the web-cache to which packets matching this value
     element should be sent.













                                                             [Page 42]

6. Security Considerations

  WCCP V2 provides a mechanism for message authentication. It is
  described in section 4.7 of this document. The authentication
  mechanism relies on a password known to all routers and web-caches in
  a Service Group. The password is part of the Service Group
  configuration and is used to compute message checksums which can be
  verified by other members of the group. Should the password become
  known to a host attempting to disrupt the operation of a Service Group
  it would be possible for that host to spoof WCCP messages and appear
  as either a router or web-cache in the Service Group.

  To pose as a router in a Service Group a host would advertise its
  presence to the members of the group in I_SEE_YOU messages.  If
  accepted as part of the Service Group the host would receive the
  configuration for the group in a HERE_I_AM message from the designated
  web-cache. This situation would not pose any threat to the operation
  of the Service Group because the host would not be performing any
  packet redirection and all packets would flow normally.

  To pose as a web-cache within a Service Group a host would advertise
  its presence in HERE_I_AM messages. Acceptance of the host as part of
  the Service Group would be decided by the designated cache and may be
  subject to additional security checks not specified by WCCP. Should
  the host become part of the Service Group it would be assigned a
  proportion of the traffic redirected by the routers in the Service
  Group. Assuming that the host drops any redirected packets the net
  effect to clients would be that some attempts to retrieve content via
  the Service Group routers would fail.


7. References

  [1] Hanks, Li, Farinacci & Traina, "Generic Routing Encapsulation
  (GRE)", RFC 1701, October 1994


8. Authors' Addresses

  Martin Cieslak
  Cisco Systems
  170 Tasman Drive
  San Jose, CA 95143

  David Forster
  Cisco Systems
  170 Tasman Drive
  San Jose, CA 95143



                                                             [Page 43]

Gurumukh Tiwana
  Cisco Systems
  170 Tasman Drive
  San Jose, CA 95143

  Rob Wilson
  Cisco Systems
  170 Tasman Drive
  San Jose, CA 95143

  email: robewils@cisco.com

  Expires January 2001






































                                                             [Page 44]