Enhanced anti-DDoS using TrafficScript, Event Handlers and iptables

[#1]

Introduction

Many DDoS attacks work by exhausting the resources available to a website for handling new connections.  In most cases, the tool used to generate this traffic has the ability to make HTTP requests and follow HTTP redirect messages, but lacks the sophistication to store cookies.  As such, one of the most effective ways of combatting DDoS attacks is to drop connections from clients that don't store cookies during a redirect.

Before you Proceed

It's important to point out that using the solution herein may prevent at least the following legitimate uses of your website (and possibly others):

• Visits by user-agents that do not support cookies, or where cookies are disabled for any reason (such as privacy); some people may think that your website has gone down!
• Visits by internet search engine web-crawlers; this will prevent new content on your website from appearing in search results!

If either of the above items concern you, I would suggest seeking advice (either from the community, or through your technical support channels).

Solution Planning

Implementing a solution in pure TrafficScript will prevent traffic from reaching the web servers.  But, attackers are still free to consume connection-handling resources on the traffic manager.  To make the solution more robust, we can use iptables to block traffic a bit earlier in the network stack.  This solution presents us with a couple of challenges:

1. TrafficScript cannot execute shell commands, so how do we add rules to iptables?
2. Assuming we don't want to permanently block all IP addresses that are involved in a DDoS attack, how can we expire the rules?

Even though TrafficScript cannot directly run shell commands, the Event Handling system can.  We can use the event.emit() TrafficScript function to send jobs to a custom event handler shell script that will add an iptables rule that blocks the offending IP address.  To expire each rule can use the at command to schedule a job that removes it.  This means that we hand over the scheduling and running of that job over to the control of the OS (which is something that it was designed to do).
The overall plans looks like this:

1. Write a TrafficScript rule that emits a custom event when it detects a client that doesn't support cookies and redirects
2. Write a shell script that takes as its input:
   1. an --eventtype argument (the event handler includes this automatically)
   2. a --duration argument (to define the length of time that an IP address stays blocked for)
   3. a string of information that includes the IP address that is to be blocked
3. Create an event handler for the events that our TrafficScript is going to emit

TrafficScript

Code

$cookie = http.getCookie( "DDoS-Test" ); 
if( !$cookie ) {

   # Either it's the visitor's first time to the site, or they don't support cookies
   $test = http.getFormParam("cookie-test");

   if( $test != "1" ) {
      # It's their first time.  Set the cookie, redirect to the same page
      # and add a query parameter so we know they have been redirected.
      # Note: if they supplied a query string or used a POST,
      # we'll respond with a bare redirect
      $path = http.getPath();

      http.sendResponse( "302 Found", "text/plain", "",
         "Location: ". string.escape( $path ) .
         "?cookie-test=1\r\nSet-Cookie: DDoS-Test=1" );

   } else {

      # We've redirected them and attempted to set the cookie, but they have not
      # accepted.  Either they don't support cookies, or (more likely) they are a bot.

      # Emit the custom event that will trigger the firewall script.
      event.emit("firewall" , request.getremoteip());

      # Pause the connection for 100 ms to give the firewall time to catch up.
      # Note: This may need tuning.
      connection.sleep( 100 );

      # Close the connection.
      connection.close("HTTP/1.1 200 OK\n"); 
   }
}

Installation

This code will need to be applied to the virtual server as a request rule.  To do that, take the following steps:

1. In the traffic manager GUI, navigate to Catalogs → Rule
2. Enter ts-firewaller in the Name field
3. Click the Use TrafficScript radio button
4. Click the Create Rule button
5. Paste the code from the attached ts-firewaller.rts file
6. Click the Save button
7. Navigate to the Virtual Server that you want to protect ( Services → <Service Name> )
8. Click the Rules link
9. In the Request Rules section, select ts-firewaller from the drop-down box
10. Click the Add Rule button

Your virtual server should now be configured to execute the rule.

Shell Script

Code

#!/bin/bash

# Use getopt to collect parameters.
params=`getopt -o e:,d: -l eventtype:,duration: -- "[email protected]"`

# Evaluate the set of parameters.
eval set -- "$params"
while true; do

  case "$1" in
  --duration ) DURATION="$2"; shift 2 ;;
  --eventtype ) EVENTTYPE="$2"; shift 2 ;;
  -- ) shift; break ;;
  * ) break ;;
  esac
done

# Awk the IP address out of ARGV
IP=$(echo "${BASH_ARGV}" | awk ' { print ( $(NF) ) }' )

# Add a new rule to the INPUT chain.
iptables -A INPUT -s ${IP} -j DROP &&

# Queue a new job to delete the rule after DURATION minutes.
# Prevents warning about executing the command using /bin/sh from
# going in the traffic manager event log.
echo "iptables -D INPUT -s ${IP} -j DROP" |
at -M now + ${DURATION} minutes &> /dev/null

Installation

To use this script as an action program, you'll need to upload it via the GUI.  To do that, take the following steps:

1. Open a new file with the editor of your choice (depends on what OS you're using)
2. Copy and paste the script code into the editor
3. Save the file as ts-firewaller.sh
4. In the traffic manager UI, navigate to Catalogs → Extra Files → Action Programs
5. Click the Choose File button
6. Select the ts-firewaller.sh file that you just created
7. Click the Upload Program button

Event Handler

Now that we have a rule that emits a custom event, and a script that we can use as an action program, we can configure the event handler that will tie the two together.
First, we need to create a new event type:

1. In the traffic manager's UI, navigate to System → Alerting
2. Click the Manage Event Types button
3. Enter Firewall in the Name field
4. Click the Add Event Type button
5. Click the + next to the Custom Events item in the event tree
6. Click the Some custom events... radio button
7. Enter firewall in the empty field
8. Click the Update button

Now that we have an event type, we need to create a new action:

1. In the traffic manager UI, navigate to System → Alerting
2. Click on the Manage Actions button
3. In the Create New Action section, enter firewall in the Name field
4. Click the Program radio button
5. Click the Add Action button
6. In the Program Arguments section, enter duration in the Name field
7. Enter Determines the length of time in minutes that an IP will be blocked for in the Description field
8. Click the Update button
9. Enter 10 in the newly-appeared arg!duration field
10. Click the Update button

Now that we have an action configured, the only thing that we have left to do is to connect the custom event to the new action:

1. In the traffic manager UI, navigate to System → Alerting
2. In the Event Type column, select firewall from the drop-down box
3. In the Actions column, select firewall from the drop-down box
4. Click the Update button

That concludes the installation steps; this solution should now be live!

Testing

Testing the functionality is pretty simple for this solution.  Basically, you can monitor the state of iptables while you run specific commands from a command line.  To do this, ssh into your traffic manager and execute iptables -L as root.  You should check this after tech of the upcoming tests.
Since I'm using a Linux machine for testing, I'm going to use the curl command to send crafted requests to my traffic manager.  The 3 scenarios that I want to test are:

1. Initial visit: The user-agent is missing a query string and a cookie
2. Successful second visit: The user-agent has a query string and has provided the correct cookie
3. Failed second visit: The user ages has a query string (indicating that they were redirected), but hasn't provided a cookie

The respective curl commands that need to be run are:

curl -v http://<tmhost>/
curl -v http://<tmhost>/?cookie-test=1 -b "DDoS-Test=1"
curl -v http://<tmhost>/?cookie-test=1

Note: If you run these commands from your workstation, you will be unable to connect to the traffic manager in any way for a period of 10 minutes!