Web Application firewalls, IPS & Network Anti Virus - Fixing the performance issues

Security professionals know that the intrusion and malware detection is now beyond looking at stream of packets.  Detection require

• SSL Decryption - Many client side attacks are increasingly hidden in HTTPS connections.( Check this out )
• Extracting data from the packets (Example:  HTML, Javascript,  Different types of files to detect attacks embedded in the data)  ( See this )
• Decoding the data (Such as UTF-8, UTF-16, De-compression etc..)
• Emulation of data if the data is script (such as Javascript) to counter evasion techniques used by attackers.
• Comparing with known signatures or codelets OR doing some kind of heuristics

This kind of analysis is not possible with stream based firewalls, IPS and AV.  These require collection of data.  They all require proxies. If some IPS/AV vendor says that they do detection without reassembling data and collecting the data, then as end user you will not be wrong to say that they either miss lot of intrusions or give too many false positives.

Computational power to do above is very high.  It is not surprising to see just less than 10Mbps of IPS, AV combined performance in devices which give 1Gbps of firewall, Ipsec throughput.  I hear stories of customer disappointments when they turn on IPS and/or AV functionality in security devices.

Network security analysts advising companies to enable full functionality even for traffic originated from trusted networks.  It should not be surprising anybody as trusted network boundary is reducing  due to mobility of machines in trusted network. That is, machines are moving from trusted to untrusted and vice versa. Examples :  laptops, ipads etc..   These  machines may get infected when they are in untrusted network and may get infect other machines in trusted network when they are brought into corporate networks. That is the reason, now full protection is being enabled on the security devices. 

HTTP is singlemost protocol that occupies majority of network bandwidth in many organizations.  HTTP is also interactive protocol. Any performance issue also  impact the user experience.   Solving HTTP performance problem not only improves user experience, but also would increase the performance of overall system.

Techniques that can be used to improve the performance of HTTP Anti-malware and IPS analysis are given below. End users might look for following features.

• Avoid doing duplicate IPS and Anti-Malware checks :  It is very common tha same resource is requested by same/multiple users in the orgnaization via HTTP.  Nework device once AV and IPS check is done on the resource should avoid doing the check again.  This requires caching of AV and IPS analysis and using it when the same resource is requested at later time.   Ofcourse, it should have life time so that it checks for AV/IPS if the content of the resource is changed.  Life time can be equal to the Expiry time of the resource which comes along with the HTTP response headers.  If possible,  this system also can do caching of the response which avoids even going to origin server, there by saving the WAN bandwidth too.  I believe that AV/IPS devices would have HTTP Caching moving forward.
• Auto blacklisting of URIs :  Malware may be served with dynamic content. In which case, above mechanism of caching does not work.  More often, Malware is served using the same URI.  If the data downloaded from a URI contains the malware, that URI can be blacklisted if malware is detected multiple times.   If the request comes to the same URI at later time,  request can be denied without even senidng the request to the origin server.  Always make sure that the newer blacklisted entries are honored by the device.
• TCP and SSL offload:  Proxies can benefit greatly if some other entity such as intelligent PCI-e takes care of TCP/IP stack and SSL offload.   
• Implement proxies as per my earlier post.
• Usage of Multicore processors and distributing the load across multiple cores.  Selection of Multicore processor depends on several factors such as cost, number of cores (performance), acceleration features etc..  But here I am only covering the features.  Features that would help in processing are :
• Processing power - Higher the processing power, better the performance would be.
• Cache Size matters:  Unlike typical firewall/Ipsec processing, amount of code that gets executed in doing AV/IPS analysis is lot higher. Higher sized L1 and L2/L3 caches would store more instructions and goes to DDR less often.  Cache for storing data is also important.
• Acceleration hardware -
• Compression/Decompression Accelerator:  To take care of decomperssing the compressed files coming in the HTTP response.
• SIMD (Single instruction Multiple Data) based hardware to do acceleration of
• Memory /String operations - Copy, Set
• Checksum, CRC operations
• HTML and URL decoding operations.
• and many more...

Hope it helps..