Detecting the Malware & Intrusions inside SSL/TLS Connections - Tips

I can't stress any more on the need for detecting and preventing attacks on client applications (Desktop/Laptop/Smart phones).  Network security devices certainly have  ways to detect the attacks on HTTP clear connections, but not when attacks are present in the HTTPS connections. Attackers may relay on this and might host malware on HTTPS and invite innocent users to browse these sites through social networking mechanisms.

To detect the attacks on HTTPS and other SSL/TLS connections, it is required that 'Network Security Devices' act as good Man-in-the-middle and get hold of clear data by decrypting it. Typically servers authenticate themselves to the clients using X.509 certificates and client authenticate to the servers using 'user name'/'password' combination.   Network Security Devices protecting servers can do this easily as administrator can upload server certificate to the network security device which in turn can terminate SSL connection on behalf of servers and analyze the clear data.  But when network security devices protect the client machines, server certificates are not in the control of administrator.  On top of that, there are large number of servers (Servers in Internet) that get accessed by client machines.

Since 'Network Security Devices' need to get the clear data,  only choice to terminate the SSL connections even if they are going towards servers in Internet.  So, one way network security devices (NSD) can do this is by creating certificate dynamically  signed by local CA certificate (self signed certificate).  There are three main considerations these NSDs should ensure.

• User experience at the browser should not suffer on per page/site basis.
• Performance of NSD should not be overly degraded.
• Security of the network should not be compromised.

Let us revisit how these requirements are met after discussing the approach first.

Flow of the Connection:

• NSDs should act as SSL/TLS Proxies.
• NSD terminates TCP Connection made by client.
• NSD makes TCP connection to the Server.
• NSD should also proceed with  the SSL Connection with the Server (Server authenticates with the client via X.509 certificate).
•  NSD must have all known CAs configured at factory default time. Also, it should have facilities to upload new CA certificates, deleted existing CA certificates.
• It is good to understand popular CAs from the browser default repository and ensure that those CA certificates are present in NSD  for authenticating original servers
• If authentication is not successful during authentication phase (There could be multiple reasons : No matching CA certificate,  Certificate was expired,  Certificate was revoked etc.. ), then it should send this information to the client with  with exact reason and all possible information in the certificate extension (vendor) and it can also send this information to the browser (if it is HTTPS). This information can be communicated with the administrator so that he can take any remedy that is possible such as populating with new CA certficate(s).  Whether server authentication is successful or not,  following step of dynamic certificate creation is required.
•  NSD is expected to have pre-configured certificates for its authentication with the clients. NSD is also expected to have self-signed CA certificate.
• NSD changes the pre-configured certificate with information from the received certificate from server in previous step.  Information such as 'Subject name',  'Extensions',  'Serial Number',  'Validity' would need to get copied from server certificate.   If this certificate is being created to send 'Server authentication failure', then  ' Subject name' only is copied from the server certificate as 'Serial number' and 'validity' may be the actual culprits for server side authentication failure.
• Issuer name should be the subject name of 'Self signed CA certificate'.
• NSD signs the certificate with 'Self Signed Certificate's private key.
• Put it in the cache. If the cache is full, remove the currently un-used old entry.
• Note that above operations need not be done, if the cache entry is found matching with the 'Subject name', 'Serial Number', 'Validity', 'Issuer name' of received server certificate.
• Now NSD terminates SSL Connection on the client side.
• Browser may object to this authentication first time as it does not recognize the 'self signed CA' certificate. Administrator of the organization is expected to give instructions to employees of organization to store this self-signed CA certificate in browsers. Once it is accepted,  browser will not complain again.
• Once SSL Connection is established on both server and client side,  clear traffic will be seen by the NSD and it can do every thing it can do to detect and protect client machines the sane way it does with clear connections.

 Now let us revisit on how the main considerations are taken care?

User experience at the browser should not suffer on per page/site basis :  Only thing the end user needs to do in the browser is to accept the 'Self Signed Certificate' provided by the NSD one time.  Since NSD is ensuring to keep the Subject name intact,  browser does not keep alerting the user for every site he/she visits.


Performance of NSD should not be overly degraded :  NSD is not generating the certificate/private key pair for every SSL site that is being visited by users.  It is expected to have some X number of certificates/private key pairs with random subject name,  Serial number.  On per connection basis, these are changed with values from the server certificate received as part of SSL connection  and then the signing processing occurs.  By avoided generation public/private-key pairs,  performance does not go down dramatically.  Also, this new certificate/private-key pair is cached.  Only when there is no space in the cache,  there may be possibility of this pair getting removed.  But immediate HTTPS connections to the same server would not undergo above process.


Security of the network should not be compromised : There could be concerns on security.  But it is not all that bad.  NSD itself is authenticating the server with trusted CA certificates. It also provides the information of CA certificate (Mainly subject name, issuer name, Serial number) in the dynamically generated certificate as part of extension so that clients can look at this information when he/she views the received certificate in browsers.  Clear data is visible within NSD device, but not on the wire. SO there should not be any worry about informational disclosure.

Also, administrators normally would be provided with white list configuration where administrator can configure two kinds of white lists - White list of client IP addresses,  White list of destination IP addresses (administrator can configure domain names).  If the connection is coming from the IP addresses which is in the white list, then NSD is not expected to apply MITM mechanism, that is SSL Connection should not be terminated. Similary, if the the destination IP falls in the white list of destination IP addresses, then also SSL connection should not be terminated.  This facility provides additional tool for administrators to satisfy their end users on doing SSL MITM proxying selectively.

If you are a buyer or already have a security product,  ask your vendor whether the device can protect you from attacks that are hidden in SSL connections.