Following the recent security vulnerability in the OpenSSL TLS implementation (see CVE-2014-0160) server administrators have been scrambling to protect vulnerable services.
Fortunately any virtual server or pool using Stingray Traffic Manager's built-in support for TLS (the ssl_decrypt or ssl_encrypt settings) are not vulnerable, see this Knowledge Base article for the status of all Riverbed products.
One write-up of the vulnerability asks;
Can I detect if someone has exploited this against me?
and;
Can IDS/IPS detect or block this attack?
This article shows how STM's TrafficScript can operate on connections to log possible attempts at exploitation of this vulnerability, and drop only those connections that may be malicious.
A simple TrafficScript rule to drop TLS connections using heartbeat messages
To get started, the following simple TrafficScript rule can be used as a request rule drop to all incoming heartbeat records (make sure that the rule runs for every request made on the connection by checking "Every" and not "Once" in the administrative UI);
# Get the TLS record header
$record_header = request.get( 5 );
# Check if this record is a heartbeat record.
$content_type = lang.ord( string.left( $record_header, 1 ) );
if( $content_type == 24 ) {
connection.discard(); # Terminate the connection
}
# Not a heartbeat record, so skip to the next record.
$record_length = ( lang.ord( string.substring( $record_header, 3, 3 ) ) << 8 ) |
lang.ord( string.substring( $record_header, 4, 4 ) );
request.endsAt( 5 + $length );
To test our homemade Intrusion Detection/Prevention System (IDS/IPS) we will first need to arrange for the traffic to pass through the traffic manager on its way to a TLS server. For example purposes, I have setup a TLS server using the openssl command line utility:
openssl s_server -accept 34848 -cert file.cert -key file.key -www -msg
This instructs an openssl test server to reply to incoming TLS/HTTP requests on port 34848 with an HTML page.
Browsing to https://localhost:34848/ on same machine (and ignoring any warnings about the invalid certificates, depending on how you created file.cert) leads to the output like;
read from 0x1ae8c50 [0x1aee340] (11 bytes => 11 (0xB)) 0000 - 16 03 01 02 00 01 00 01-fc 03 03 ........... read from 0x1ae8c50 [0x1aee34e] (506 bytes => 506 (0x1FA)) : write to 0x1ae8c50 [0x1af7e30] (66 bytes => 66 (0x42)) 0000 - 16 03 03 00 3d 02 00 00-39 03 03 53 4e 60 26 00 ....=...9..SN`&. : :
I've specifically picked out the beginning of two TLS records, where each TLS record's bytes are formatted like so;
struct {
ContentType type;
ProtocolVersion version;
uint16 length;
select (SecurityParameters.cipher_type) {
case stream: GenericStreamCipher;
case block: GenericBlockCipher;
case aead: GenericAEADCipher;
} fragment;
} TLSCiphertext;
Both the client's message and the server's response above have their first byte as 0x16 - ContentType "handshake", the fourth and fifth bytes give the length of the data following the record header: for the ClientHello handshake message the client sent this is 0x200 (512) bytes. The TLS heartbeat records we are interested in have ContentType of 24 (or 0x18).
To pass the request and response through STM I configure;
a pool "my TLS service" in traffic manager, adding your test server as its node, and
a virtual server "my IDS" that uses "my TLS service" as its default pool.
The virtual server has its protocol set to "SSL (HTTPS)" so that it will pass-through the TLS traffic as it arrives.
Make sure the TrafficScript above has been added to to the Rules Catalog, and that it has been configured as a request rule on this virtual server.
Directing your browser to the newly created virtual server should behave just as before, except any connection where TLS heartbeat messages are exchanged will be terminated. You may want to verify your newly protected system by using one of the many tools available; here are two examples:
First, from Filippo Valsorda, a web based tool where you simply type in the URL for virtual server (if accessible from the internet):
https://filippo.io/Heartbleed/
Second, from Steffen Ullrich (and others), a perl script that may be more suitable for testing a test deployment:
https://github.com/noxxi/p5-scripts/blob/master/check-ssl-heartbleed.pl
You should take care to only perform vulnerability testing on systems where you have permission to do so.
A comprehensive TrafficScript rule with a customizable response to TLS heartbeat messages
The above TrafficScript rule is simple and effective at preventing both heartbleed attack requests to a server and compromising responses from a client; it is, however, quite a blunt instrument. It doesn't consider the case where the applications legitimately use TLS heartbeat messages for regular operation, and it will result in the traffic manager buffering the whole record before forwarding it on to the server (which may or may not impact performance, depending on the traffic your deployment observes).
A more complex TrafficScript rule can avoid these particular pitfalls, in particular being able to operate where TLS heartbeat messages are required, providing detailed event reporting and enabling the data to be streamed by the traffic manager. You can find such a rule below, it improves upon the simple rule above in the following ways:
It can be used as both a request and response rule (providing protection both clients and servers from attacks requests and/or compromising responses).
It has a configurable threshold level (in the case where your applications require the heartbeat messages to operate correctly).
It has a customizable "what to do on detection" sub-routine (it currently records the detected attack in the error log and terminates the connection, but you have the full power of TrafficScript available to you to adopt a suitable strategy for your deployment).
# A TrafficScript rule for detecting suspicious TLS Heartbeat records, with
# a view to providing protection against the Heartbleed vulnerability
# (CVE-2014-0160).
#
# Copyright (c) 2014, Riverbed Technology
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this
# list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# This rule can be used as either a request and/or response rule on a virtual
# server operating at the TCP level (it is recommended for use only on
# virtual servers configured explicitly for SSL passthrough). Ensure the
# rule is configured to be executed "everytime" on the virtual server.
# The $threshold variable provides the primary configurable for this rule.
# It represents the threshold size for TLS Heartbeat records above
# which an error is generated and the connection dropped. Use a value of 0
# if you know heartbeat messages are never legitimately used, otherwise pick a
# suitable value above which no valid client/server would use in your
# deployment.
#
# Note: Any value above 0 does not prevent a heartbleed attack, it will just
# mean naive attacks are more likely to be detected and prevented.
#
$threshold = 0; # bytes above which we consider the heartbeat malicious.
# The action_to_take_on_detection subroutine is called by the rule when a
# heartbeat message exceeding the threshold is detected. This subroutine
# is provided to make it easy to customize the action to take upon
# identification.
#
# The parameter provides the length of the threshold exceeding record.
#
sub action_to_take_on_detection( $length )
{
# Create a description of who is involved and how.
$prefix = "request from ";
$from = ip_and_port_to_string( request.getRemoteIP(), request.getRemotePort() );
$to = "";
if( rule.getstate() == "RESPONSE" ) {
$prefix = "response from ";
$to = " to " . $from;
$from = ip_and_port_to_string( response.getRemoteIP(),
response.getRemotePort() );
}
$who = $prefix . $from . $to;
log.error( "Oversized Heartbeat packet detected, possible Heartbleed attack " .
"(CVE-2014-0160), " .
$length - $threshold . " bytes beyond threshold in " . $who );
# Could also add a specific alert, event.emit(...), to trigger notification to
# an administrator.
connection.discard(); # Do not send anything more to the client
}
# --- Primary rule logic starts here ---
# Get the current state of the TLS record reading.
#
# The state is used to maintain the state of TLS record reading across
# multiple executions of the rule for a single connection. This is
# effectively an ordered pair, stored as two separate variables. The
# semantics are:
#
# "remaining" The remaining data to be read for the current TLS record. Used
# to determine where the next header to be considered for
# heartbleed detection can be found in the stream.
#
# "saved" Saved data from a previous execution, to be prepended to
# whatever data has been read by this execution. Used to store
# partially complete SSL/TLS records (for when a record straddles
# TCP packet boundaries). This should be 4 bytes at most.
#
# Note the invariants below which follows from the semantics described.
$remaining_state_name = "heartbleed-remaining-" . rule.getname() .
"-" . rule.getstate();
$saved_state_name = "heartbleed-saved-" . rule.getname() . "-" . rule.getstate();
# The starting position of the next record (relative to this rule).
$pos = lang.toInt( connection.data.get( $remaining_state_name ) );
# The buffer to work with is whatever we have now, plus any saved data
# from a previous execution.
$buffer = lang.toString( connection.data.get( $saved_state_name ) );
# Sanity checks - can be removed - codifies the invariant that if we have
# some "saved data" from a previous execution, then the remaining data
# for a previous record must be 0. This is because we only save data when
# trying to retrieve the whole header (at the start of a new record).
lang.assert( string.length( $buffer ) == 0 || $pos == 0,
"Unexpected saved data in the middle of a record" );
lang.assert( string.length( $buffer ) < 5,
"Too much data saved across requests" );
# Append the rest of the buffer for consideration.
if( rule.getstate() == "RESPONSE" ) {
$buffer .= response.get( response.getLength() );
} else {
lang.assert( rule.getstate() == "REQUEST", "Invalid context to use this rule" );
$buffer .= request.get( request.getLength() );
}
# Check if we have enough data yet to reach the start of a new record.
if( string.length( $buffer ) < $pos ) {
# not read enough yet, remember how far through we are and try again later.
connection.data.set( $remaining_state_name, $pos - string.length( $buffer ) );
# if $pos is positive then saved data state must already be "" by assertion.
break; # exit the rule
}
# We have a new record to process in this rule execution, isolate the new
# data for inspection as the beginning of a new record.
$unchecked = string.skip( $buffer, $pos );
# Start with fresh state for the parsing loop.
$remaining = 0;
$saved = "";
while( string.length( $unchecked ) > 0 ) {
# Check we have enough data to extract the header details (5 bytes).
if( string.length( $unchecked ) < 5 ) {
# Not enough data to extract a header, save what data we have
# for the next execution of the rule.
$saved = $unchecked;
break;
}
# We have a full header; extract the header details.
$type = lang.ord( string.left( $unchecked, 1 ) );
$length = ( lang.ord( string.substring( $unchecked, 3, 3 ) ) << 8 ) |
lang.ord( string.substring( $unchecked, 4, 4 ) );
# Check if the record is OK, and deal with if it isn't.
if( $type == 24 && $length > $threshold ) {
action_to_take_on_detection( $length );
}
# Find the start point of the next record.
$rec_len = 5 + $length;
if( string.length( $unchecked ) > $rec_len ) {
# Still have some data left to process in this rule, go around again.
$unchecked = string.skip( $unchecked, $rec_len );
} else {
# The current record fits neatly into or extends beyond this buffer.
# Store how much data needs to be read (if any) beyond what we already
# have.
$remaining = $rec_len - string.length( $unchecked );
break;
}
}
connection.data.set( $remaining_state_name, $remaining );
connection.data.set( $saved_state_name, $saved );
sub ip_and_port_to_string( $ip, $port )
{
$ipversion = string.validIPAddress( $ip );
if( 6 == $ipversion ) {
return "[" . $ip . "]:" . $port;
} else {
return $ip . ":" . $port;
}
}
Recommendations
The best use of this rule depends on a number of factors specific to your deployment. Below we've included some high level guidance.
Use this as a request rule with threshold of zero; if you are concerned about vulnerable servers behind your STM deployment and you know heartbeat messages are not used by valid users.
Use this as a response rule with a threshold of zero; if you want to protect clients in front of your STM deployment.
You can use as both a request and response rule, each with differing actions, for example:
Detecting and logging all heartbeat messages from clients.
Terminating the connection if a large heartbeat message is seen in a response from a server (as an attempt to only prevent invalid heartbeat.
If you have a significant amount of RAM available in your STM machines you may find the simple rule presented at the beginning of the article provides improved performance under certain traffic loads.
... View more