cancel
Showing results for 
Search instead for 
Did you mean: 

Pulse Secure vADC

Sort by:
The famous TrafficScript Mandelbrot generator!
View full article
Introduction Riverbed Stingray Traffic Manager is the only pure software ADC (Application Delivery Controller) available in the market today. As such it is the only ADC which can move with your business across Physical, Virtual, and Cloud platforms, and give you the exact same features and performance in each. Today Riverbed provides Stingray in the form of Virtual Appliances for many of the popular Hypervisors (including VMWare, Xen, and Hyper-V). The Virtual Appliance can also be provided in the standard OVA (Open Virtualization Archive) format for importing into a number of other Virtual platforms. For systems such as IBM PureFlex, where the Virtual Appliances are more closely integrated with the underlying platform it can be beneficial to build a completely custom appliance using the software edition of Stingray. IBM provides a tool precisely for this purpose: The Image Construction and Composition Tool (ICCT). This document will take you through the process of building a customised Stingray appliance for deployment on PureFlex using the ICCT. A very similar process can be followed to deploy an image into IBMs Smart Cloud Enterprise. IBM Image Construction and Composition Tool ICCT is a tool which allows a user on a PureFlex or Smart Cloud Enterprise platform to create custom Virtual Appliances. A full description of the ICCT application is beyond the scope of this document. We will assume that the ICCT has already been configured to communicate directly with your PureFlex or SCE environment and we will concentrate solely on the configuration elements required to build a custom appliance. These include: Managing ISO Resources, Managing Software Bundles, and Managing Images. ISO Resources An ISO resource is simply an ISO containing a compatible Operation System image. The ISO can be used to deploy new images from scratch. In terms of Stingray, the only PureFlex/SCE supported Operating Systems are Red Hat Enterprise Linux, SuSE Enterprise Linux. Software Bundles Along with this document, you should have been provided a prebuilt Stingray Software Bundle (a RAS file). The RAS file contains a number of scripts which are used by PureFlex or SCE in building and deploying custom appliances. A software Bundle includes four main configuration items: the installation script, the configuration script, a reset script, and some firewall configuration. Images An image is essentially a Virtual Machine. An image appears in ICCT either when it is created by deployment from an ISO, when it is imported from a connected SCE or PureFlex system, or when an existing image is extended. Build Process The process of building a custom appliance will be explained in detail in the section “Creating a custom Stingray appliance” below. As a quick overview the process is as follows: Take an image of a BaseOS and extend it. Add the software bundle to the extended image and configure the installer parameters. Deploy the new image (installer script is executed). Log on to running virtual machine and check installation succeeded. Capture the image. Export/deploy the image into PureFlex Manager/Smart Cloud Entry as an appliance Deploy an instance from the appliance image (the configure script is executed). The Stingray Software Bundle (RAS) The software bundle provided has everything you need to build a custom Stingray appliance for use in PureFlex or (with a little tweaking) Smart Cloud Enterprise. The RAS file includes a number of scripts which we discuss in turn. Installation Script The installation script “install.sh” is executed when the image is deployed for the first time. Its purpose is to download the Stingray software and Kernel modules from a HTTP repository and install them in to the image. The installer takes two arguments: stmUrl The URI for the Stingray software installer. Default version 9.1 from support.riverbed.com modsUrl The URI for the Stingray Kernel Modules installer Default: Version 2.5 from splash.riverbed.com The stingray installation needs nothing more than a Base OS install, however you will need to ensure that you have the development tools and kernel headers available in order to install the kernel modules. Note: It is recommended that you check the installer output in /var/log/provision.log after the initial deployment to ensure that everything worked. This is also a good opportunity to apply the latest security patches and bug fixes to the underlying OS. Configure Script The configuration script “configure.sh” is executed each time an instance is deployed from the appliance image. The configure script has the task of configuring the Stingray software in the appliance according to the parameters provided in the deployment form. The configure script in the provided RAS takes the following arguments: password The password for the Stingray admin user. Set password to “RANDOM” to have one generated for you. Default: “” accept_license Set to “Accept” if you agree to Stingray licensing terms. The configure script will not continue if this has not been set. Default: “No” license_key Should be “none” or a HTTP URI from where a license can be downloaded. Default: “none” join_cluster Should be “No” or “Yes”. Default: “No” cluster_host Should be “none” or an IP address or hostname of a Stingray to cluster with. Default: “none” cluster_tips Should be “No” or “Yes”. Default: “No” cluster_location The id of the location to join if we are joining a MSM cluster. Default: 0 cluster_external_ip If we are joining a cluster on the other side of a NAT device, then this should be our external IP Address. Default: “none” Reset Script The reset script “reset.sh” is executed whenever the appliance needs to be cleaned up and returned to an unconfigured state. I don’t believe it is ever actually used, but a reset script is provided. It takes no arguments. Firewall Configuration The final piece of configuration in a software bundle is the firewall configuration. This allows you to specify which ports should be open on the firewall. The Stingray bundle currently opens port 9090 and 9080 for cluster communications and administration. You will want to add all of the service ports you expect to use on your Stingray appliance. For example HTTP (80), HTTPS (443), etc. Miscellaneous All scripts write their output to /var/log/provision.log so you can always check that to see if there were any problems with the install, configure or reset steps. The software bundle includes AutoScaling drivers for Smart Cloud Enterprise, and also Smart Cloud Entry provisioning APIs. If you do not join a cluster during the deployment of an instance then the configure script will drop them into the extra files catalogue. However if you do join a cluster, then they will be copied into the root users home directory “/root”. Creating a custom Stingray Appliance Step 1: Upload your ISO The first step is to provide ICCT with an ISO containing the Operating System you wish to use. At the time of writing PureFlex and SmartCloud support RedHat Enterprise Linux and SuSE Enterprise Linux Operating Systems. Navigate to the “Manage ISO Resources” section and upload the ISO you wish to use as the base OS of your appliance. Step 2: Upload the Stingray RAS file The next step is to upload the Software Bundle (RAS file) to the ICCT server. Navigate to the “Build and Manage software Bundles” section and click on the import icon. The RAS file will need to be available either at a URL accessible by the ICCT server, or uploaded to the local file system of the ICCT server itself. Step 3: Create your new BaseOS Image At this point we have an ISO and our software bundle imported. However before we can use the software bundle we must have an image imported into ICCT which we can extend. We’re going to create a new image from the ISO; however you may chose to import a pre-existing Linux image if you have one. If you import a pre-exising image then you can skip to step 4. Navigate to “Build and Manage Images”. Click on the “New Image” icon and then select create from ISO On the next screen you will need to enter some information about the new image we are creating. You can give it a name such as “RHEL BaseOS”, A universal ID such as com.riverbed.rhel62.baseOS, a version (eg 6.2.0), and a description. You will next need to provide the ISO resource which we uploaded earlier, and a kickstart or AutoYAST file. You can download the default KickStart file from ICCT and then extend it. If you intend to make use of the Stingray kernel modules then I would recommend adding the following applications:  gcc, make, perl, kernel-devel. Next you can pick the hardware parameters for the new image. A summary is displayed on the final screen, click done to complete the image creation process. Step 4: Extend the BaseOS Image Now that we have a Base OS configuration, the next step is to add our software bundle and set the installer parameters. Select your new Base OS image in the left hand pane and then click the Extend icon Enter appropriate information for your Stingray Virtual Appliance: Name, Universal ID, Version and Description, and then click Create. Select the new Image in the left hand pane. Click the “Start Editing” Button. Scroll down to the “Software Bundles” and click the “Add” button. Add the Stingray software bundle. Once you have added the bundle you have the option of chosing a different version of the software to install. Expand the properties and change the locations of the installers to the desired versions. The Deploy options can be ignored as they are used at instance deployment not in the initial install phase. Once you are happy with your installation options, Click “Done Editing” and “Save”. Click on the “Synchronize” button to have ICCT deploy your new image and run through the installation process. Once the image is synchronized you will be able to see the details in the “Virtual System” section of the image details pane. At this point you can log into the Virtual System via SSH and confirm that the installer completed successfully by viewing the “/var/log/provision.log” file. This is also a good time to apply any OS patches and/or add additional software. Note:  On RHEL 6.2 the modules may fail to install because the shipped gcc does not match the compiler used to build the kernel (not on the original disc). Feel free to run a yum update at this point and rerun the Stingray modules installer. The provision log will tell you the temporary directory where they were downloaded. Once you are happy with the system, return to ICCT and click “Capture”. ICCT will shut down the VM and capture it as an appliance image, once this step completes you can export the image into PureFlex System Manager or Smart Cloud Entry. Your custom Stingray appliance is ready.
View full article
This Document provides step by step instructions on how to set up Brocade Virtual Traffic Manager for VMware Horizon View Servers.   This document has been updated from the original deployment guides written for Riverbed Stingray and SteelApp software.
View full article
Update: See also this new article including a simple template rule: A Simple Template Rule SteelCentral Web Analyzer - BrowserMetrix   Riverbed SteelCentral Web Analyzer is a great tool for monitoring end-user experience (EUE) of web applications, even when they are hosted in the cloud. And because it is delivered as true Software-as-a-Service, you can monitor application performance form anywhere, and drill down to analyse individual transactions by URL, location or browser type, and highlight requests which t ook too long to respond.   In order to track statistics, your web application needs to send statistics on each transaction to Web Analyzer (formerly BrowserMetrix) using a small piece of JavaScript, and it is very easy to inject the extra JavaScript code without needing to change the application itself. This Solution Guide (attached) shows you how to use TrafficScript to inject the JavaScript snippet into your web applications, by inspecting all web pages and inserting into the right place in each document:   No modification needed to your application Easy to select which pages you want to instrument Use with all applications in your data center, or hosted in the cloud Even works with compressed HTML pages (eg, gzip encoded) Create dynamic JavaScript code to track session-level information Use Riverbed FlyScript to automate the integration between Web Analyzer and Traffic Manager   How does it work? SteelApp Traffic Manager sits in front of the web applications on the right, and inspects each web page before it is sent to the client. Stingray checks to see if the page has been selected for analysis by Web Analyzer, and then constructs the JavaScript fragment and injects into the web page at the right place in the HTML document.   When the web page arrives at the client browser, the JavaScript snippet is executed.  It builds a transaction profile with timing information and submits the information to the Web Analyzer SaaS platform managed by Riverbed.  You can then analyze the results, in near-realtime, using the Web Analyzer web portal.   Thanks also to Faisal Memon for his help creating the Solution Guide.   Read more In addition to the attached deployment guide showing how to create complex rules for JavaScript Injection, you may be also be interested in this new article showing how to use a simple template rule wit Traffic Manager and SteelCentral Web Analyzer: A Simple Template Rule for SteelCentral Web Analyzer - BrowserMetrix   For similar solutions, check out the Content Modification examples in the Top vADC Examples and Use Cases article.   Updated 15th July 2014 by Paul Wallace. Article formerly titled "Using Stingray with OPNET AppResponse Xpert BrowserMetrix" Thanks also to Mike Iem for his help updating this article. 29th July 2014 by Paul Wallace. Added note about the new article including the simple template rule          
View full article
As the content on websites grow, the structure of their URLs can change dramatically.  The addition of new applications and components can play havoc with the established URL space, and the development cost of supporting the old links that have been published in articles, referenced by search engines and bookmarked by users can be very high.   Stingray is in a great place to address this problem, and let your applications use the URL spaces most suited to them with little concern for backwards compatibility.  This article presents one technique you can employ to access this problem in a scalable and manageable manner.   A simple example   Let's start with a simple example; suppose you published content at the following URLs:   www.example.com/news.html www.example.com/about.html www.example.com/careers.html www.example.com/demos.html   As your company grew, offices were added, more products were developed and content began to be broken out by location.  The original URL structure was no longer sustainable, and a deeper layer of content was necessary:   www.example.com/corporate/news.html www.example.com/product/news.html www.example.com/product/demos.html www.example.com/about/cambridge.html www.example.com/about/honalulu.html www.example.com/careers/overview.html www.example.com/careers/honalulu/engineering.html www.example.com/careers/cambridge/research.html   The challenge is to serve out the best content to people who make requests to the old URLs.  You want a better solution than manually adding redirects to your webserver configuration file; ideally a solution that can be used by your web content team without the intervention from IT.   A solution   Ideally, we would like to issue an HTTP redirect to the most appropriate page. This is, of course, simple using TrafficScript.   1 http.redirect( "http://www.example.com/corporate/news.html" );   ... but you want to avoid having to build and maintain a rule that looks like this:   1 2 3 4 5 6 7 8 9 10 11 12 13 $url = http.getPath();   if ( $url == "/news.html" ){       http.redirect( "http://www.example.com/product/news.html" );   } else if ( $url == "/about.html" ) {       http.redirect( "http://www.example.com/about/cambridge.html" );   } else if ...   }   Wouldn't it be easier if you could maintain a file with a list of redirects, and Stingray could act on that?   /news.html http://www.example.com/product/news.html /about.html http://www.example.com/about/cambridge.html   etc...?   You can use the ResourceTable libraries from HowTo: Store tables of data in TrafficScript - part 1 to help you do exactly that:   1 2 3 4 5 6 7 import ResourceTableSmall as table;    $path = http.getPath();    $redirect = table.lookup( "redirects.txt" , $path );   if ( $redirect ) http.redirect( $redirect );   Managing the file of redirects   That leaves us with one problem - how best to manage the (albeit simple) file that contains the redirects?  The format is simple enough (space-separated key / value pairs, one per line) that anyone can edit it, but how do they get it into the Stingray configuration without using the complex and powerful Stingray Admin Interface?   There are a couple of simple approaches:   Use the Stingray Admin Interface, and give the user a permissions group that only gives access to the Extra Files catalog; Push the file in using one of Stingray's configuration APIs - Feature Brief: Stingray's RESTful Control API or Feature Brief: Stingray's SOAP Control API; Copy the file in using SSH, and then invoke ZEUSHOME/zxtm/bin/replicate-config to push the configuration across the cluster.   The REST approach is particularly attractive - you can use browser plugins like Chrome's REST Console to push configuration files into Stingray.   However, you may not want someone to use either approach directly because it would imply giving them a full administrative logon to the Stingray cluster.  In that case, you could consider a simple commandline tool that they can use to upload the updated configuration file, using the Collected Tech Tips: Using the RESTful Control API or  Collected Tech Tips: SOAP Control API examples APIs.
View full article
This Document provides step by step instructions on how to set up Brocade Virtual Traffic Manager for Oracle EBS 12.1.   This document has been updated from the original deployment guides written for Riverbed Stingray and SteelApp software.
View full article
Stingray allows you to inspect and manipulate both incoming and outgoing traffic with a customized version of Java's Servlet API. In this article we'll delve more deeply into some of the semantics of Stingray's Java Extensions and show how to validate XML files in up- and in downloads using TrafficScript and Java Extensions.   The example that will allow us to illustrate the use of XML processing by Stingray is a website that allows users to share music play-lists. We'll first look at the XML capabilities of 'conventional' TrafficScript, and then investigate the use of Java Extensions.   A play-list sharing web site   You have spent a lot of time developing a fancy website where users can upload their personal play-lists, making them available to others who can then search for music they like and download it. Of course you went for XML as the data format, not least because it allows you to make sure uploads are valid. Therefore, you can help your users' applications by providing them with a way of validating their XML-files as they are uploaded. Also, to be on the safe side, whenever an application downloads an XML play-list it should be checked and only reach the user if it passes the validation.   XML provides the concept of schema files to describe what a valid document has to look like. One popular schema language is the W3C's XML Schema Definition (XSD), see http://www.w3.org/TR/xmlschema-0/. Given an XSD file, you can hand an XML document to a validator to find out whether it actually conforms to the data structure specified in the schema.   Coming back to our example of a play-list sharing website, you have downloaded the popular xspf (XML Shareable Playlist Format, 'spiff') schema description from http://xspf.org/validation/ . One of the tags allowed inside a track in XML files of this type is image. By specifying tags like <image> http://images.amazon.com/images/P/B000002J0B.01.MZZZZZZZ.jpg </image> a user could see the following pictures:     Validating XML with TrafficScript   How do you validate an XML file from a user against that schema? Stingray's TrafficScript provides the xml.validate() function. Here's a simple rule to check the response of a web server against a XSD:   1 2 3 4 5 6 7 8 9 10 $doc = http.getResponseBody();  $schema = resource.get( "xspf.xsd" );  $result = xml.validate.xsd( $doc , $schema );  if ( 1 == $result ) {      log .info( "Validation succeeded" );  } else if ( 0 == $result ) {      log .info( "Validation failed" );  } else {      log .info( "Validation error" );  }   Let's have a closer look at what this rule does:   First, it reads in the whole response by calling http.getResponseBody(). This function is very practical but you have to be extremely careful with it. The reason is that you do not know beforehand how big the response actually is. It might be an audio stream totaling many hundred megabytes in size. Surely you don't want Stingray to buffer all that data. Therefore, when using http.getResponseBody() you should always check the mime type and the content length of the response (see below for code that does this). Our rule then goes on to load the schema definition file with resource.get(), which must be located in ZEUSHOME/zxtm/conf/extra/ for this step to work. Finally it does the actual validation and checks the result. In this simple example, we are only logging the result, on your music-sharing web site you would have to take the appropriate action.   The last rule was a response rule that worked on the result from the back-end web server. These files are actually under your control (at least theoretically), so validation is not that urgent. Things are different if you allow uploads to your web site. Any user-provided data must be validated before you let it through to your back-ends. The following request rule does the XML validation for you:   1 2 3 4 5 6 7 8 9 10 11 12 $m = http.getMethod();  if ( 0 == string.icmp( $m , "POST" ) ) {      $clen = http.getHeader( "Content-Length" );      if ( $clen > 0 && $clen <= 1024*1024 ) {         $schema = resource.get( "xspf.xsd" );         $doc = http.getBody();         $result = xml.validate.xsd( $doc , $schema );         # handle result      } else {         # handle over-sized posts      }  }    Note how we first look at the HTTP method, then retrieve the length of the post's body and check it. That check, which is done in the line   1 if ( $clen > 0 && $clen <= 1024*1024 ) {   ...deserves a bit more comment: The variable $clen was initialized from the post's Content-Length header, so it could be the empty string at that stage. When TrafficScript converts data to integers, variables that do not actually represent numbers are converted to 0 (see the TrafficScript reference for more details). Therefore, we have to check that $clen is greater than zero and at most 1 megabyte (or whatever limit you choose to impose on the size of uploads). After checking the content length we can safely invoke getBody().   A malicious user might have faked the HTTP header to specify a length larger than his actual post. This would lead Stingray to try to read more data than the client sends, pausing on a file descriptor until the connection times out. Due to Stingray's non-blocking IO multiplexing, however, other requests would be processed normally.   Validating XML with Stingray's Java Extensions   After having explored TrafficScript's built-in XML support, let's now see how XML validation can be done using Java Extensions.   If you are at all familiar with Java Servlets, Stingray's Java Extensions should feel like home for you. The main differences are   You have a lot of Stingray's built-in functionality ready at hand via attributes. You can manipulate both the response (as in conventional Servlets) and the request (unique to Stingray's Servlet Extensions as Stingray sits between the client and the server).   There's lots more detail in the Feature Brief: Java Extensions in Stingray Traffic Manager.   The interesting thing here is that this flow actually applies twice: First when the request is sent to the server (you can invoke the Java extension from a request rule) and then again when the response is sent back to the client (allowing you to change the result from a response rule). This is very practical for your music-sharing web site as you only have to write one Servlet. However, you have to be able to tell whether you are working on the response or the request. The ZXTMHttpServletResponse object which is passed to both the doGet() and doPost() methods of the HttpServlet object has a method to find out which direction of the traffic flow you are currently in: boolean isResponseRule() . This distinction is never needed in conventional Servlet programming as in that scenario it's the Servlet's task to create the response, not to modify an existing response.   These considerations make it easy to design the Stingray Servlet for your web site:   There will be an init() method to read in the schema definition and to set up the xml.validation.Validator object. We'll have a single private validate() method to do the actual work. The doGet() method will invoke validate() on the server's response, whereas the doPost() method does the same on the body of the request   After all that theory it's high time for some real code (note that any import directives have been removed for the sake of readability as they don't add anything to our discussion - see Writing Java Extensions - an introduction ):   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 public class XmlValidate extends HttpServlet {      private static final long serialVersionUID = 1L;      private static Validator validator = null;      public void init( ServletConfig config ) throws ServletException {         super.init( config );         String schema_file = config.getInitParameter( "schema_file" );              if ( schema_file == null )            throw new ServletException( "No schema file specified" );              SchemaFactory factory = SchemaFactory.newInstance(            XMLConstants.W3C_XML_SCHEMA_NS_URI);              Source schemaFile = new StreamSource(new File(schema_file));         try {            Schema schema = factory.newSchema(schemaFile);            validator = schema.newValidator();         } catch( SAXException saxe ) {            throw new ServletException(saxe.getMessage());         }      }  // ... other methods below  }   The validate() function is actually very simple as all the hard work is done inside the Java library. The only thing to be careful about is to make sure that we don't allow concurrent access to the Validator object from multiple threads:   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 private boolean validate( InputStream in, HttpServletResponse res, String errmsg )         throws IOException      {         Source src=new StreamSource(in);         try {            synchronized( validator ) {               validator.validate(src);            }         } catch( SAXException saxe ) {            String msg = saxe.getMessage();            res.setContentType( "text/plain" );            PrintWriter out = res.getWriter();            out.println(errmsg);            out. print ( "Validation of the xml file has failed with error message: " );            out.println(msg);            return false;         }         return true;      }    Note that the only thing we have to do in case of a failure is to write to the stream that makes up the response. No matter whether this is being done in a request or a response rule, Stingray will take that as an indication that this is what should be sent back to the client. In the case of a request rule, Stingray won't even bother to hand on the request to a back-end server and instead send the result of the Java Servlet; in a response rule, the server's answer will be replaced by what the Servlet has produced.   Now we are ready for the doGet() method:   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 public void doGet( HttpServletRequest req, HttpServletResponse res )        throws ServletException, IOException     {        try {           ZXTMHttpServletResponse zres = (ZXTMHttpServletResponse) res;           if ( !zres.isResponseRule() ) {              log ( "doGet called in request rule ... bailing out" );              return ;           }           InputStream in = zres.getInputStream();           validate(in, zres, "The file you requested was rejected." );        } catch( Exception e ) {           throw new ServletException(e.getMessage());        }     }    There's not really much work left apart from calling our validate() method with the error message to append in case of failure. As discussed previously, we make sure that we are actually working in the context of a response rule because otherwise the response would be empty. Exactly the opposite has to be done when processing a post:   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 public void doPost( HttpServletRequest req, HttpServletResponse res )       throws ServletException, IOException    {       try {          ZXTMHttpServletRequest zreq = (ZXTMHttpServletRequest) req;          ZXTMHttpServletResponse zres = (ZXTMHttpServletResponse) res;            if ( zres.isResponseRule() ) {             log ( "doPost called in response rule ... bailing out" );             return ;          }            InputStream in = zreq.getInputStream();          if ( validate(in, zres, "Your upload was unsuccessful" ) ) {             // just let the post through to the backends          }       } catch(Exception e) {          throw new ServletException(e.getMessage());       }    }    The only thing missing are the rules to invoke the Servlet, so here they are (assuming that the Servlet has been loaded up via the 'Java' tab of the 'Catalogs' section in Stingray's UI as a file called XmlValidate.class ). First the request rule:   1 2 3 4 $m = http.getMethod();  if ( 0 == string.icmp( $m , "POST" ) ) {      java.run( "XmlValidate" );  }    and the response rule is almost the same:   1 2 3 4 $m = http.getMethod();  if ( 0 == string.icmp( $m , "GET" ) ) {      java.run( "XmlValidate" );  }    It's your choice: TrafficScript or Java Extensions   Which is better?   So now you are left with a difficult decision: you have two implementations of the same functionality, which one do you choose? Bearing in mind that the unassuming java.run() leads to a considerable amount of inter-process communication between the Stingray child process and the Java Servlet runner, whereas the xml.validate() is handled in C++ inside the same process, it is a rather obvious choice. But there are still situations when you might prefer the Java solution.   One example would be that you have to do XML processing not supported directly by Stingray. Java is more flexible and complete in the XML support it provides. But there is another advantage to using Java: you can replace the actual implementation of the XML functionality. You might want to use Intel's XML Software SuiteJ for Java, for example. But how do you tell Stingray's Java runner to use another XML library? Only two settings have to be adapted:   java!classpath /opt/intel/xmlsoftwaresuite/java/1.0/lib/intel-xss.jar java!command java -Djava.library.path=/opt/intel/xmlsoftwaresuite/java/1.0/bin/intel64 -server   This applies if you have installed Intel's XML Software SuiteJ in /opt/intel/xmlsoftwaresuite/java/1.0/ and are using the 64 bit version of the shared library. Both changes can be made in the 'Global Settings' tab of the 'System' section in Stingray's UI.
View full article
This article illustrates how to write data to a MySQL database from a Java Extension, and how to use a background thread to minimize latency and control the load on the database.   Being Lazy with Java Extensions   With a Java Extension, you can log data in real time to an external database. The example in this article describes how to log the ‘referring’ source that each visitor comes in from when they enter a website. Logging is done to a MySQL database, and it maintains a count of how many times each key has been logged, so that you can determine which sites are sending you the most traffic.   The article then presents a modification that illustrates how to lazily perform operations such as database writes in the background (i.e. asynchronously) so that the performance the end user observes is not impaired.   Overview - let's count referers!   It’s often very revealing to find out which web sites are referring the most traffic to the sites that you are hosting. Tools like Google Analytics and web log analysis applications are one way of doing this, but in this example we’ll show an alternative method where we log the frequency of referring sites to a local database for easy access.   When a web browser submits an HTTP request for a resource, it commonly includes a header called "Referer" which identifies the page that linked to that resource. We’re not interested in internal referrers – where one page in the site links to another. We’re only interested in external referrers. We're going to log these 'external referrers' to a MySQL database, counting the frequency of each so that we can easily determine which occur most commonly. Create the database   Create a suitable MySQL database, with limited write access for a remote user: % mysql –h dbhost –u root –p Enter password: ******** mysql> CREATE DATABASE website; mysql> CREATE TABLE website.referers ( data VARCHAR(256) PRIMARY KEY, count INTEGER ); mysql> GRANT SELECT,INSERT,UPDATE ON website.referers TO 'web'@'%' IDENTIFIED BY 'W38_U5er'; mysql> GRANT SELECT,INSERT,UPDATE ON website.referers TO 'web'@'localhost' IDENTIFIED BY 'W38_U5er'; mysql> QUIT;   Verify that the table was correctly created and the ‘web’ user can access it:   % mysql –h dbhost –u web –p Enter password: W38_U5er mysql> DESCRIBE website.referers; +-------+--------------+------+-----+---------+-------+ | Field | Type         | Null | Key | Default | Extra | +-------+--------------+------+-----+---------+-------+ | data  | varchar(256) | NO   | PRI |         |       | | count | int(11)      | YES  |     | NULL    |       | +-------+--------------+------+-----+---------+-------+ 2 rows in set (0.00 sec)   mysql> SELECT * FROM website.referers; Empty set (0.00 sec)   The database looks good...   Create the Java Extension   We'll create a Java Extension that writes to the database, adding rows with the provided 'data' value, and setting the 'count' value to '1', or incrementing it if the row already exists.   CountThis.java   Compile up the following 'CountThis' Java Extension:   import java.io.IOException; import java.io.PrintWriter; import java.sql.Connection; import java.sql.DriverManager; import java.sql.PreparedStatement; import javax.servlet.ServletConfig; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; public class CountThis extends HttpServlet { private static final long serialVersionUID = 1L; private Connection conn = null; private String userName = null; private String password = null; private String database = null; private String table = null; private String dbserver = null; public void init( ServletConfig config) throws ServletException { super.init( config ); userName = config.getInitParameter( "username" ); password = config.getInitParameter( "password" ); table = config.getInitParameter( "table" ); dbserver = config.getInitParameter( "dbserver" ); if( userName == null || password == null || table == null || dbserver == null ) throw new ServletException( "Missing username, password, table or dbserver config value" ); try { Class.forName("com.mysql.jdbc.Driver").newInstance(); } catch( Exception e ) { throw new ServletException( "Could not initialize mysql: "+e.toString() ); } } public void doGet( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { try { String[] args = (String[])req.getAttribute( "args" ); String data = args[0]; if( data == null ) return; if( conn == null ) { conn = DriverManager.getConnection( "jdbc:mysql://"+dbserver+"/", userName, password); } PreparedStatement s = conn.prepareStatement( "INSERT INTO " + table + " ( data, count ) VALUES( ?, 1 ) " + "ON DUPLICATE KEY UPDATE count=count+1" ); s.setString(1, data); s.executeUpdate(); } catch( Exception e ) { conn = null; log( "Could not log data to database table '" + table + "': " + e.toString() ); } } public void doPost( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { doGet( req, res ); } }   Upload the resulting CountThis.class file to Traffic Manager's Java Catalog. Click on the class name to configure the following initialization properties:   You must also upload the mysql connector (I used mysql-connector-java-5.1.24-bin.jar ) from dev.mysql.com to your Traffic Manager Java Catalog.   Add the TrafficScript rule   You can test the Extension very quickly using the following TrafficScript rule to log the each request:   java.run( "CountThis", http.getPath() );   Check the Traffic Manager  event log for any error messages, and query the table to verify that it is getting populated by the extension:   mysql> SELECT * FROM website.referers ORDER BY count DESC LIMIT 5; +--------------------------+-------+ | data                     | count | +--------------------------+-------+ | /media/riverbed.png      |     5 | | /articles                |     3 | | /media/puppies.jpg       |     2 | | /media/ponies.png        |     2 | | /media/cats_and_mice.png |     2 | +--------------------------+-------+ 5 rows in set (0.00 sec)   mysql> TRUNCATE website.referers; Query OK, 0 rows affected (0.00 sec)   Use 'Truncate' to delete all of the rows in a table.   Log and count referer headers   We only want to log referrers from remote sites, so use the following TrafficScript rule to call the Extension only when it is required:   # This site $host = http.getHeader( "Host" ); # The referring site $referer = http.getHeader( "Referer" ); # Only log the Referer if it is an absolute URI and it comes from a different site if( string.contains( $referer, "://" ) && !string.contains( $referer, "://".$host."/" ) ) { java.run( "CountThis", $referer ); }   Add this rule as a request rule to a virtual server that processes HTTP traffic.   As users access the site, the referer header will be pushed into the database. A quick database query will tell you what's there: % mysql –h dbhost –u web –p Enter password: W38_U5er mysql> SELECT * FROM website.referers ORDER BY count DESC LIMIT 4; +--------------------------------------------------+-------+ | referer                                          | count | +--------------------------------------------------+-------+ | http://www.google.com/search?q=stingray        |    92 | | http://www.riverbed.com/products/stingray      |    45 | | http://www.vmware.com/appliances               |    26 | | http://www.riverbed.com/                       |     5 | +--------------------------------------------------+-------+ 4 rows in set (0.00 sec)   Lazy writes to the database   This is a useful application of Java Extensions, but it has one big drawback. Every time a visitor arrives from a remote site, his first transaction is stalled while the Java Extension writes to the database. This breaks one of the key rules of website performance architecture – do everything you can asynchronously (i.e. in the background) so that your users are not impeded (see "Lazy Websites run Faster").   Instead, a better solution would be to maintain a separate, background thread that wrote the data in bulk to the database, while the foreground threads in the Java Extension simply appended the Referer data to a table:     CountThisAsync.java   The following Java Extension (CountThisAsync.java) is a modified version of CountThis.java that illustrates this technique:   import java.io.IOException; import java.sql.Connection; import java.sql.DriverManager; import java.sql.PreparedStatement; import java.sql.SQLException; import java.util.LinkedList; import javax.servlet.ServletConfig; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; public class CountThisAsync extends HttpServlet { private static final long serialVersionUID = 1L; private Writer writer = null; protected static LinkedList theData = new LinkedList(); protected class Writer extends Thread { private Connection conn = null; private String table; private int syncRate = 20; public void init( String username, String password, String url, String table ) throws Exception { Class.forName("com.mysql.jdbc.Driver").newInstance(); conn = DriverManager.getConnection( url, username, password); this.table = table; start(); } public void run() { boolean running = true; while( running ) { try { sleep( syncRate*1000 ); } catch( InterruptedException e ) { running = false; }; try { PreparedStatement s = conn.prepareStatement( "INSERT INTO " + table + " ( data, count ) VALUES( ?, 1 )" + "ON DUPLICATE KEY UPDATE count=count+1" ); conn.setAutoCommit( false ); synchronized( theData ) { while( !theData.isEmpty() ) { String data = theData.removeFirst(); s.setString(1, data); s.addBatch(); } } s.executeBatch(); } catch ( Exception e ) { log( e.toString() ); running = false; } } } } public void init( ServletConfig config ) throws ServletException { super.init( config ); String userName = config.getInitParameter( "username" ); String password = config.getInitParameter( "password" ); String table = config.getInitParameter( "table" ); String dbserver = config.getInitParameter( "dbserver" ); if( userName == null || password == null || table == null || dbserver == null ) throw new ServletException( "Missing username, password, table or dbserver config value" ); try { writer = new Writer(); writer.init( userName, password, "jdbc:mysql://"+dbserver+"/", table ); } catch( Exception e ) { throw new ServletException( e.toString() ); } } public void doGet( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { String[] args = (String[])req.getAttribute( "args" ); String data = args[0]; if( data != null && writer.isAlive() ) { synchronized( theData ) { theData.add( data ); } } } public void doPost( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { doGet( req, res ); } public void destroy() { writer.interrupt(); try { writer.join( 1000L ); } catch( InterruptedException e ) {}; super.destroy(); } }   When the Extension is invoked by Traffic Manager , it simply stores the value of the Referer header in a local list and returns immediately. This minimizes any latency that the end user may observe.   The Extension creates a separate thread (embodied by the Writer class) that runs in the background. Every syncRate seconds, it removes all of the values from the list and writes them to the database.   Compile the extension: $ javac -cp servlet.jar:zxtm-servlet.jar CountThisAsync.java $ jar -cvf CountThisAsync.jar CountThisAsync*.class   ... and upload the resulting CountThisAsync.jar Jar file to your Java catalog . Remember to apply the four configuration parameters to the CountThisAsync.jar Java Extension so that it can access the database, and modify the TrafficScript rule so that it calls the CountThisAsync Java Extension.   You’ll observe that database updates may be delayed by up to 20 seconds (you can tune that delay in the code), but the level of service that end users experience will no longer be affected by the speed of the database.
View full article
Important note - this article illustrates an example of authenticating traffic using Java Extensions.  Stingray TrafficScript also includes LDAP/Active Directory primitives, in the form of auth.query() , and these are generally simpler and easier to use than a Java-based solution.   Overview   A very common requirement for intranet and extranet applications is the need to authenticate users against an Active Directory (or LDAP) database. The Java Extension in this article describes how to do exactly that.   This article describes two Java Extensions that manage the HTTP Basic Authentication process and validate the supplied username and password against an Active Directory database. It shows how to use Initialization Parameters to provide configuration to an extension, and how authentication results can be cached to reduce the load on the Active Directory server.   A basic Java Extension   The first Java Extension verifies that the supplied username and password can bind directly to the LDAP database.  It's appropriate for simple LDAP deployments, but enterprise AD deployments may not give end users permissions to bind directly to the entire database, so the second example may be more appropriate.   The Java Extension (version 1)   import java.io.IOException; import java.io.PrintWriter; import java.util.Hashtable; import javax.naming.Context; import javax.naming.directory.DirContext; import javax.naming.directory.InitialDirContext; import javax.servlet.ServletConfig; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import com.zeus.ZXTMServlet.ZXTMHttpServletRequest; public class LdapAuthenticate extends HttpServlet { private static final long serialVersionUID = 1L; private String dirServer; private String realm; public void init( ServletConfig config) throws ServletException { super.init( config ); dirServer = config.getInitParameter( "DB" ); realm = config.getInitParameter( "Realm" ); if( dirServer == null ) throw new ServletException( "No DB configured" ); if( realm == null ) realm = "Secure site"; } public void doGet( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { try { ZXTMHttpServletRequest zreq = (ZXTMHttpServletRequest)req; String[] userPass = zreq.getRemoteUserAndPassword(); if( userPass == null ) throw new Exception( "No Authentication details" ); Hashtable<String, String> env = new Hashtable<String, String>(); env.put( Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory"); env.put( Context.PROVIDER_URL, "LDAP://" + dirServer ); env.put( Context.SECURITY_AUTHENTICATION, "DIGEST-MD5" ); env.put( Context.SECURITY_PRINCIPAL, userPass[0] ); env.put( Context.SECURITY_CREDENTIALS, userPass[1] ); DirContext ctx = new InitialDirContext( env ); ctx.close(); // No exceptions thrown... must have been successful ;-) return; } catch( Exception e ) { res.setHeader( "WWW-Authenticate", "Basic realm=\"" + realm + "\"" ); res.setHeader( "Content-Type", "text/html" ); res.setStatus( 401 ); String message = "<html>" + "<head><title>Unauthorized</title></head>" + "<body>" + "<h2>Unauthorized - please log in</h2>" + "<p>Please log in with your system username and password</p>" + "<p>Error: " + e.toString() + "</p>" + "</body>" + "</html>"; PrintWriter out = res.getWriter(); out.println( message ); } } public void doPost( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { doGet( req, res ); } }     Configuring the Java Extension   Upload the LdapAuthenticate Java Extension into the Java Catalog page. Click on the LdapAuthenticate link to edit the properties of the extension, and add two Initialization Parameters:   DB specifies the name of the LDAP or Active Directory database, and Realm specifies the authentication realm.   These parameters are read when the extension is initialized, which occurs the first time the extension is used by Stingray:   public void init( ServletConfig config) throws ServletException { super.init( config ); dirServer = config.getInitParameter( "DB" ); realm = config.getInitParameter( "Realm" ); if( dirServer == null ) throw new ServletException( "No DB configured" ); if( realm == null ) realm = "Secure site"; }   If you change the value of one of these parameters, use the 'Force Reload' option in the Stingray Admin Server to unload and reload this extension.   Either use the auto-generated rule, or create a new TrafficScript rule to call the extension on every request to an HTTP virtual server:   java.run( "LdapAuthenticate" );   Testing the Java Extension   When you try to access the web site through Stingray, you will be prompted for a username and password; the LdapAuthenticate extension checks that the username and password can bind to the configured Active Directory database, and refuses access if not: If you are unable to log in, cancel the prompt dialog box to see the error reported by the Java extension. In the following case, there was a networking problem; the extension could not contact the database server provided in the 'DB' parameter: Caching the Authentication Results   Caching the Authentication response from the Java extension will improve the performance of the web site and reduce the load on the database server.   You can modify the TrafficScript rule that calls the extension so that it records successful logins, caching them for a period of time. The following rule uses the data.set() TrafficScript function to record successful logins, caching this information for 10 minutes before attempting to reauthenticate the user against the database server.   $auth = http.getHeader( "Authorization" ); if( data.get( $auth ) < sys.time() ) { data.remove( $auth ); java.run( "LdapAuthenticate" ); # if we got here, we were authenticated. # Cache this information for 600 seconds data.set( $auth, sys.time()+600 ); }   A more sophisticated Java Extension implementation   In enterprise deployments, users often cannot bind to the LDAP or Active Directory database directly.  This example runs a custom search against the Active Directory database to locate the distinguishedName corresponding to the userid provided in the login attempt, then attempts to verify that the user can bind using their distinguishedName and the password they provided.   The Java Extension (version 2)   import java.io.IOException; import java.io.PrintWriter; import java.util.Hashtable; import javax.naming.NamingEnumeration; import javax.naming.Context; import javax.naming.directory.Attribute; import javax.naming.directory.Attributes; import javax.naming.directory.DirContext; import javax.naming.directory.InitialDirContext; import javax.naming.directory.SearchControls; import javax.naming.directory.SearchResult; import javax.servlet.ServletConfig; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import com.zeus.ZXTMServlet.ZXTMHttpServletRequest; public class LdapAuthenticate extends HttpServlet { private static final long serialVersionUID = 1L; private String dirServer; private String realm; private String authentication; private String bindDN; private String bindPassword; private String baseDN; private String filter; public void init( ServletConfig config) throws ServletException { super.init( config ); dirServer = config.getInitParameter( "DB" ); if( dirServer == null ) throw new ServletException( "No DB configured" ); realm = config.getInitParameter( "Realm" ); if( realm == null ) realm = "Secure site"; authentication = config.getInitParameter( "authentication" ); if( authentication == null ) authentication = "simple"; bindDN = config.getInitParameter( "bindDN" ); if( dirServer == null ) throw new ServletException( "No bindDN configured" ); bindPassword = config.getInitParameter( "bindPassword" ); if( dirServer == null ) throw new ServletException( "No bindPassword configured" ); baseDN = config.getInitParameter( "baseDN" ); if( dirServer == null ) throw new ServletException( "No baseDN configured" ); filter = config.getInitParameter( "filter" ); if( dirServer == null ) throw new ServletException( "No filter configured" ); } public void doGet( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { try { ZXTMHttpServletRequest zreq = (ZXTMHttpServletRequest)req; String[] userPass = zreq.getRemoteUserAndPassword(); if( userPass == null ) throw new Exception( "No Authentication details" ); Hashtable<String, String> env = new Hashtable<String, String>(); env.put( Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory"); env.put( Context.PROVIDER_URL, "LDAP://" + dirServer ); env.put( Context.SECURITY_AUTHENTICATION, authentication ); /* Bind with admin credentials */ NamingEnumeration<SearchResult> ne; String searchfilter = filter.replace( "%u", userPass[0] ); try { env.put( Context.SECURITY_PRINCIPAL, bindDN ); env.put( Context.SECURITY_CREDENTIALS, bindPassword ); DirContext ctx = new InitialDirContext( env ); String[] attrIDs = { "distinguishedName" }; SearchControls sc = new SearchControls(); sc.setReturningAttributes( attrIDs ); sc.setSearchScope(SearchControls.SUBTREE_SCOPE); ne = ctx.search(baseDN, searchfilter, sc); ctx.close(); } catch( Exception e ) { throw new Exception( "Failed to bind with master credentials: " + e.toString() ); } if( ne == null || !ne.hasMore() ) { throw new Exception( "No such user " + userPass[0] ); } SearchResult sr = (SearchResult) ne.next(); Attributes attrs = sr.getAttributes(); Attribute dnAttr = attrs.get("distinguishedName"); String dn = (String) dnAttr.get(); /* Now bind using dn with the user credentials */ try { env.put( Context.SECURITY_PRINCIPAL, dn ); env.put( Context.SECURITY_CREDENTIALS, userPass[1] ); DirContext ctx = new InitialDirContext( env ); ctx.close(); } catch( Exception e ) { throw new Exception( "Failed to bind with user credentials: " + e.toString() ); } // No exceptions thrown... must have been successful ;-) return; } catch( Exception e ) { res.setHeader( "WWW-Authenticate", "Basic realm=\"" + realm + "\"" ); res.setHeader( "Content-Type", "text/html" ); res.setStatus( 401 ); String message = "<html>" + "<head><title>Unauthorized</title></head>" + "<body>" + "<h2>Unauthorized - please log in</h2>" + "<p>Please log in with your system username and password</p>" + "<p>Error: " + e.toString() + "</p>" + "</body>" + "</html>"; PrintWriter out = res.getWriter(); out.println( message ); } } public void doPost( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { doGet( req, res ); } }   This version of the Java Extension takes additional initialization parameters: It first searches the database for a distinguishedName using a query resembling:   $ ldapsearch -h DB -D bindDN -w bindPassword -b baseDN filter distinguishedName   Where the %u in the filter is replaced with the username in the login attempt.   It then attempts to bind to the database using a query resembling:   $ ldapsearch -h DB -D distinguishedName -w userpassword   ... and permits access if that bind is successful.
View full article
This article combines two Stingray technologies – Java Extensions and the Control API – and shows you how to query the status of Stingray with a simple, authenticated web request to generate an immediate status report.   Introduction   Stingray’s admin interface gives you plenty of useful information about the performance and health of Stingray, but sometimes you may want a lighter, quicker way of checking the status. Apache's ServerStatus page is a good model of a report that you can access from within a public website using a privileged URL.   This article presents a Java Extension that generates a similar report. The Java Extension is run when a particular URL is requested and appropriate authentication is given; the extension queries Stingray via the SOAP-based Control API and retrieves a range of status information and a list of the recently-processed connections.   Before you proceed   First, follow the instructions in the article Using Stingray's SOAP Control API with Java to create an appropriate Stingray-API.jar file. Upload this interface file to the Java Extensions catalog in the Stingray admin interface, and ensure that all of the the required dependencies are also installed in the Java catalog.   The Java Extension   The attached Java Extension source file (ServerStatus.java) is quite long; you can shortcut building it from source by using the attached ServerStatus.jar file and uploading it straight to the Java Extensions catalog in Stingray.   Compiling the extension   At compilation time, the extension will require the Stingray-API.jar and Apache Axis 1.4 jar files for type checking. For example, if you use the Eclipse IDE, you should add these files as 'External Jars' in the build path.  From the command line:   $ javac -cp Stingray-API.jar:zxtm-servlet.jar:servlet.jar:axis-1_4/lib/* ServerStatus.java   The extension compiles to several separate class files. You can upload each class file to the JAva catalog, or you can use the 'Export' command (in Eclipse) to create a Java jar file containing the compiled class files from the ServerStatus project. From the command line:   $ jar -cvf ServerStatus.jar ServerStatus*.class     Using the ServerStatus Java Extension   The ServerStatus java extension prompts for a username and password; provide the username and password for the 'admin' user in the Stingray Admin Interface.   Use the following RuleBuilder request rule to invoke the extension:     ... or the equivalent TrafficScript rule:   $path = http.getPath(); if( $path == "/serverstatus" ) java.run( "ServerStatus" );   Then, go to http://www.site.com/serverstatus to run the extension: Security   The extension first checks that the HTTP request that has invoked it contains a username and password. If the username and password is missing, the extension returns a '401 Authenticate' message to prompt the caller to provide them. The caller should provide the username and password for the 'admin' user in the Stingray Admin Interface.   The extension then attempts to connect to the local Stingray Control API interface using the supplied username/password pair. If the connection fails because the authentication details are invalid, the extension again prompts for a new username and password.   Use SSL!   When you provide the admin user and password, it’s very advisable to only do so over an SSL-enabled website. You can ensure that the user name and password is never requested by a non-SSL site by modifying the rule as follows:   $path = http.getPath(); if( $path == "/serverstatus" && ssl.isSSL() ) java.run( "ServerStatus" );   If necessary, configure a special SSL virtual server in Stingray to host the extension. The extension will return identical results no matter which virtual server it is invoked from.   Read more   Tech Tip: Reading Stingray's internal diagnosis report using Perl and SOAP Java Extensions - Overview Using Stingray's SOAP Control API with Java
View full article
Recent investigations have revealed an error in the PHP and Java floating point library that may be exploited to cause a denial of service against a web service. You can use Stingray Traffic Manager to filter incoming traffic and discard requests that may seek to exploit this fault.   Background   In January 2011, a bug was discovered in PHP's floating point library. Under certain circumstances, an attempt to convert the string '2.2250738585072011e-308' into a floating point value would hang the PHP runtime.   A similar problem was discovered in the Java runtime (and compiler). The two articles give a detailed description of the nature of the problem and its cause (relating to the parsing of a number close to DBL_MIN, the smallest non-zero number that can be represented as a floating point).   The implications   What are the implications to a web developer or security team? This fault can be exploited to mount a denial-of-service attack if an attacker can send a carefully-crafted request that causes the PHP or Java runtime to attempt to convert a string into the problematic floating-point value. Web developers are accustomed to treating user input with suspicion - for example, careful escaping to prevent SQL injection attacks - but who would have thought that an innocuous floating point number could pose a similar threat? Any application code that parses input into a floating point could be vulnerable; for example, a mapping API that takes coordinates as input may be vulnerable.   However, there's an even simpler potential problem that is inherent the HTTP protocol; the family of 'Accept' HTTP headers use floating point scores that may be exploitable in certain implementations.   Accept: text/*;q=0.3, text/html;q=0.7, text/html;level=1, text/html;level=2;q=0.4, /;q=0.5 Accept-Charset: iso-8859-5, unicode-1-1;q=0.8 Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0 Accept-Language: da, en-gb;q=0.8, en;q=0.7   The wisest solution to protecting against this vulnerability would be to deploy a Web Application Firewall (see Stingray Application Firewall) and verify that the baseline protection rules detect attempts to exploit this attack. It's also possible to detect and drop these attacks using a TrafficScript rule, and this article presents a couple of solutions.   Floating Point: Solution 1   The following TrafficScript request rule checks all of the headers in each HTTP request. If the headers contain the sequence of digits that are the signature of this number, then the rule logs a warning and drops the request immediately.   $headers = http.getHeaders(); foreach( $header in hash.keys( $headers ) ) { $value = $headers[$header]; # remove any decimal points $value = string.replace( $value, '.', '' ); if( string.contains( $value, "2225073858507201" ) ) { log.warn( "suspect request - dropping" ); connection.discard(); } }   The result:     Floating Point: Solution 2   This more advanced solution checks both headers and form parameters, logs a more descriptive error message and illustrates the use of TrafficScript subroutines to minimise duplicated code:   # Checks the array of key-value (headers or form parameters) # If any value contains the suspect floating point value, return the # name of the header or form parameter sub check( $h ) { foreach( $k in hash.keys( $h ) ) { $v = $h[$k]; # remove any decimal points $v = string.replace( $v, '.', '' ); if( string.contains( $v, "2225073858507201" ) ) return $k; } } # Log the request and drop it immediately sub logAndDrop( $reason, $k, $v ) { $ip = request.getRemoteIP(); $country = geo.getCountry( $ip ); if( !$country ) $country = 'unknown'; $msg = 'Request from ' . $ip . ' (' . $country . ') ' . ' contained suspicious ' . $reason . ': ' . $k . ': ' . $v; log.warn( $msg ); # Optional - raise an event to trigger a configured event handler # event.emit( "FloatingPointAttack", $msg ); connection.discard(); } $headers = http.getHeaders(); if( $h = check( $headers ) ) logAndDrop( "header", $h, $headers[$h] ); $params = http.getFormParams(); if( $h = check( $params ) ) logAndDrop( "parameter", $h, $params[$h] );   The result, from an internal IP address (192.168.35.1) and using a querystring ?userid=2.2250738585072011e-308:     There is a very, very slim risk of false positives with these rules (dropping connections which would not have a malicious effect), but the probability of the string "2225073858507201" appearing is miniscule (except perhaps for blog posts about this very vulnerability...).
View full article
Here (attached) is a library that uses TrafficScript array and hashes to provide another new data structure, the set: an unordered collection in which an element occurs either once or not at all. An example use would be "words I have seen on the page". The trick to implementing this is to realise that TrafficScript already has an efficient set implementation: the hash (associative array). Specifically, you can put your data in the keys of the hash and use an arbitrary constant as the value. This means that inserting, deleting and checking membership of the set are all fast operations. While you could use that trick directly on an ad-hoc basis in individual rules, this library will improve readability and provide some type checking. If you're curious and use lang.dump($some_set) to inspect the data structure, you'll see something like this (note that the order of hash elements is arbitrary): [ "type" => "set", "values" => [ "foo" => 1, "123" => 1, "bar" => 1 ] ] One limitation of this structure is that only scalars can be members of sets, since only scalars can be hash keys. In this library, if you insert an array, each element will be inserted, and if you try to insert a hash, you'll get a warning and nothing will be inserted. The library includes the following functions: set.new() Returns a new (empty) set. set.destroy( $set ) Destroy a set. set.insert( $set, $value ) Insert a value (or another set or an array of values) into the set. set.remove( $set, $value ) Remove a value (or set or array of values) from the set. set.contains( $set, $value ) Check if the set contains a particular value. set.toarray( $set ) Return all the values in the set. set.empty( $set ) Empty a set. set.union( $a, $b ) Returns the set of elements that are in $a or $b. set.intersection( $a, $b ) Returns the set of elements that are in $a and $b. set.difference( $a, $b ) Returns the set of elements that are in $a and not in $b. set.count( $set ) Count the number of items in the set. set.subseteq( $a, $b ) Check if $a is a (non-strict) subset of $b. set.superseteq( $a, $b ) Check if $a is a (non-strict) superset of $b. To use it, add the library to your TrafficScript rules catlog, and then, in another rule, use: import libSet.rts as set; and all the 'set' functions above will be available. Here's an example of how you could use it. This rule will expect the words in $target to occur somewhere on the page, and write a log line if any of them are missing. import libSet.rts as set; $ctype = http.getResponseHeader( "Content-Type" ); if( ! string.startswith( $ctype, "text/html" ) ) break; $target = set.new(); set.insert( $target, ["riverbed","news","articles"] ); $used = set.new(); $words = string.split(string.lowercase(http.getResponseBody())); foreach( $w in $words ) {    set.insert( $used, $w ); } $unused = set.toarray( set.difference( $target, $used )); if( array.length( $unused ) ) {    log.info( http.getPath().": " . array.join(array.sort($unused),", ") );
View full article
This simple TrafficScript rule helps you debug and instrument the traffic manager.  It adds a simple overlay on each web page containing key information that is useful in a test and debugging situation:     Served from: identifies the server node that generated the response Took XX seconds: using the technique in HowTo: Log slow connections in Stingray Traffic Manager identifies the time between recieving the HTTP request and completion of response rules Took YY retries: if present, indicates that the traffic manager had to retry the request one or more times before getting a satisfactory response   The TrafficScript response rule is as follows:   $ct = http.getResponseHeader( "Content-Type" ); if( !string.startsWith( $ct, "text/html" ) ) break; $style = ' <style type="text/css"> .stingrayTip { position:absolute; top:0; right:0; background:black; color:#acdcac; opacity:.6; z-index:100; padding:4px; } .stingrayTip:hover { opacity:.8; } </style>'; $text = "Served from " . connection.getNode(); if( request.getRetries() ) $text .= "<br>Took " . request.getRetries() . " retries"; if( connection.data.get("start") ) $text .= "<br>Took " . (sys.time.highres()-connection.data.get("start")) . " seconds"; if( connection.data.get("notes") ) $text .= "<br>" . connection.data.get( "notes" ); $body = http.getResponseBody(); $html = '<div class="stingrayTip">' . $text . '</div>'; $body = string.regexsub( $body, "(<body[^>]*>)", $style."$1\n".$html."\n", "i" ); http.setResponseBody( $body );   Using the timing information   To get the 'took XX seconds' information, place the rule as late as possible in your chain of response rules, and add the following rule as a request rule, to run as early as possible:   $tm = sys.time.highres(); connection.data.set("start", string.sprintf( "%f", $tm ) );   See HowTo: Log slow connections in Stingray Traffic Manager for more information.   Additional information   You can present additional information in the info box by storing it in a connection-local variable called 'notes', for example:   $cookies = http.getHeader( "Cookie" ); $msg = "Cookies: " . $cookies; connection.data.set( 'notes', $msg );   Read more   This example uses techniques similar to the Watermarking web content with Stingray and TrafficScript article.
View full article
Having described Watermarking PDF documents with Stingray and Java and Watermarking Images with Java Extensions, this article describes the much simpler task of adding a watermark to an HTML document.   Stingray's TrafficScript language is fully capable of managing text content, so there's no need to revert to a more complex Java Extension to modify a web page:   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 # Only process text/html responses  $ct = http.getResponseHeader( "Content-Type" );  if ( ! string.startsWith( $ct , "text/html" ) ) break;       # Calculate the watermark text  $text = "Hello, world!" ;       # A new style, named watermark, that defines how the watermark text should be displayed:  $style = '  <style type= "text/css" >  .watermark {      color: #d0d0d0;      font-size: 100pt;      -webkit-transform: rotate(-45deg);      -moz-transform: rotate(-45deg);      -o-transform: rotate(-45deg);      transform: rotate(-45deg);      position: absolute;      width: 100%;      height: 100%;      margin: 0;      z- index : 100;      top:200px;      left:25%;      opacity: .5;  }  </style>';       # A div that contains the watermark text  $div = '<div class="watermark">' . $text . '</div>' ;       # Imprint in the body of the document  $body = http.getResponseBody();  if ( string.regexmatch( $body , "^(.*)</body>(.*?)$" , "i" ) ) {      http.setResponseBody( $1 . $style . $div . "</body>" . $2 );  }  - See more at: https://splash.riverbed.com/docs/DOC-1664 #sthash.BsuXFRP2.dpuf   This rule has the following effect...: Of course, you can easily change the watermark text:   1 $watermark = "Hello, world!" ;   ... perhaps to add more debugging or instrumentation  to the page.   The CSS style for the watermark is based on this article , and other conversations on stackoverflow; you'll probably need to adapt it to get precisely the effect that you want.   This rule uses a simple technique to append text to an HTML document (see the Tracking user activity with Google Analytics article for another example). You could use it to perform other page transforms, such as the common attempt to apply copy-protection by putting a full-size transparent layer over the entire HTML document.
View full article
Content and Intellectual Property protection is a serious issue for any web site providing paid-for content. The article How to stop 'login abuse', using TrafficScript article describes how Stingray can be used to detect when a username and password is reused from different locations; this article showcases the power of Stingray's Java Extensions to apply a dynamically-generated, visible watermark to every image served up by your website.   The article covers how to use the Eclipse IDE or the command line to build the extension, how to apply the extension to your traffic, some optimization tricks to maximise performance, and how to debug and patch the code on-the-fly.   For more information on Java Extensions, you may like to read the article Feature Brief: Java Extensions in Stingray Traffic Manager.     Prerequisites   Before you begin, make sure that you have:   A working copy of Stingray, with the correct Java Runtime Environment (Sun JRE 1.5+) installed on the server (or just deploy the Stingray virtual appliance); A copy of the Java Platform JDK (to compile from the command line), and optionally an IDE such as Eclipse.   Configure the Stingray to load-balance traffic to a suitable website; you can use a public website like www.riverbed.com (remember to add a rule to set the host header if necessary). Check you can receive the website content through Stingray.   Step 1: Create your Java Extension   If you want to skip this section, just grab the ImageWatermark.class file attached to this article and proceed with that.   Go to the Stingray Admin interface, and locate the Catalog->Java Extensions page. On that page, locate the links to the Java Servlet API and ZXTM Java Extensions API files, and save these two Jar files in a convenient, long-term location:   Download the Java Servlet API and Stingray Java Extensions API files   If you are using the command line   Save the .jar files in the current directory, and create a new file named ImageWatermark.java .   If you are using Eclipse:   In Eclipse, create a new project:   Project Type: Java Project; Project Name: WaterMark; use the default options; Java Settings: Select the 'Libraries' tar and add the two external Jar files that you stored in the previous step:   Once you've created the project, go to the Package Explorer. Right-click on your project and create a new Class named 'ImageWatermark' with the default options.   The Java Code   Paste the following code into the ImageWatermark.java file that is created:   import java.io.IOException; import javax.servlet.ServletException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; // Additional imports import java.awt.*; import java.awt.color.*; import java.awt.geom.*; import java.awt.image.*; import java.io.*; import javax.imageio.ImageIO; import com.zeus.ZXTMServlet.*; public class ImageWatermark extends HttpServlet { private static final long serialVersionUID = 1L; public void doGet( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { try { ZXTMHttpServletResponse zres = (ZXTMHttpServletResponse) res; String ct = zres.getHeader( "Content-Type" ); if( ct == null || ! ct.startsWith( "image/" ) ) return; InputStream is = zres.getInputStream(); BufferedImage img = ImageIO.read( is ); Graphics2D g = (Graphics2D)img.getGraphics(); int width = img.getWidth(); int height = img.getHeight(); if( width < 200 || height < 30 ) return; String[] args = (String[])req.getAttribute( "args" ); String message = ( args != null ) ? args[0] : "Hello world!"; g.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON); g.setComposite( AlphaComposite.getInstance( AlphaComposite.SRC_OVER, (float)0.5 )); Font myFont = new Font( "Sans", Font.PLAIN, 18 ); Rectangle2D bb = myFont.getStringBounds( message, g.getFontRenderContext() ); int x = 2; int y = (int)bb.getHeight(); g.setFont( myFont ); g.setColor( Color.darkGray ); g.drawString( message, x, y ); zres.setHeader( "Content-Type", "image/png" ); ImageIO.write( img, "PNG", zres.getOutputStream() ); } catch( Exception e ) { log( req.getRequestURI() + ": " + e.toString() ); } } public void doPost( HttpServletRequest req, HttpServletResponse res ) throws ServletException, IOException { doGet( req, res ); } }   From the command line, you can compile this as follows:   $ javac -cp servlet.jar:zxtm-servlet.jar ImageWatermark.java   This will create an 'ImageWatermark.class' file in the current directory.   Using Eclipse, p aste this source in and hit ‘Ctrl-Shift-O’ to get the correct imports. Then save the file – this will automatically compile it; check that there were no errors in the compilation.   This should create the output file ImageWatermark.class in somewhere like HOMEDIR/workspace/ImageWatermark/bin .   Step 2: Load the extension into Stingray and watermark some images   Go to the Java Extensions catalog page and upload the Java Extension 'class' file for the WaterMark extension.   When you upload the class file, the Stingray Admin Server will automatically create a simple RuleBuilder rule that invokes the Java Extension.   Configure your virtual server to run the RuleBuilder rule on each response, then shift-reload the webpage that is delivered through Stingray to clear your cache and reload each image: Note the little "Hello world!" watermark on the top left of any images larger then 200x30 pixels.   Step 3: Optimize the way that the extension is called   The Java Extension is called by the RuleBuilder rule on every HTTP response. However, the extension only processes images; HTML, CSS and other document types are ignored.   Selectively running Java Extensions   Invoking a Java Extension carries some overhead, so it is prudent to ensure that Extensions are only invoked when they are needed. With a small change to the rule, you can ensure that this is the case.   First, convert the "ImageWatermark" RuleBuilder rule to TrafficScript by editing the rule and using the "Convert Rule" button. This will create a simple TrafficScript rule which calls the WaterMark extension:   java.run( "ImageWatermark" );   Edit the rule to add a condition that only runs the WaterMark extension when the object type is an image:   $contenttype = http.getResponseHeader( "Content-Type" ); if( string.startsWith( $contenttype, "image/" ) ) { java.run( "ImageWatermark" ); }   Passing parameters to a Java Extension   You can pass parameters from TrafficScript to a Java Extension. They are passed in as additional arguments to the ‘java.run()’ TrafficScript function:   $ip = request.getRemoteIP(); $time = sys.timeToString( sys.time() ); $message = "IP: ".$ip.", ".$time; $contenttype = http.getResponseHeader( "Content-Type" ); if( string.startsWith( $contenttype, "image/" ) ) { java.run( "ImageWatermark", $message ); }   The Java extension will read these arguments using the 'args' attribute, which returns a string array of the argument values:   String[] args = (String[])req.getAttribute( "args" ); String message = ( args != null ) ? args[0] : "Hello world!";   Use the Stingray Admin Interface to load in the new copy of the Java extension.   Now, when you shift-reload the web page (to clear the cache) the watermark text on the image will contain the message created in the TrafficScript rule, with the IP address of the remote user and the time when the image was downloaded.   Of course, you could also generate this message directly in the Java Extension, but quick code changes (such as modifying the text in the message) are easier when the code resides in TrafficScript rather than a compiled Java class.   Step 4: Live debugging and hot code patching   Finally, refer to the "Remote Debugging" section of the Java Development Guide (Stingray Product Documentation). This describes how to configure the arguments used to start the Java Virtual Machine (JVM) so that it can accept live debugging sessions from a remote debugger, and how to configure Eclipse to connect to the JVM.   You can edit and save the code in Eclipse. When you save the code, Eclipse compiles it and patches the code in the JVM on the fly. Try changing the point size of the font or the color and see the effects immediately:   Font myFont = new Font( "Serif", Font.BOLD, 12 );   and...   g.setColor( Color.red );   Live patching in Eclipse is a great way to debug, test and update code, but remember that it only patches the code in the live Java VM. When Stingray restarts, it will fall back to the version of the Java class that you originally uploaded, so once you’re finished, remember to upload the compiled class through the Admin interface so that it persists.   Read more Feature Brief: Java Extensions in Stingray Traffic Manager Writing Java Extensions - an introduction Watermarking PDF documents with Stingray and Java
View full article
Not content with getting your Traffic Manager to tweet through the event handing system (Traffic Managers can Tweet Too), this article explains how you can tweet directly from TrafficScript.   This is more than just a trivial example; sending tweets from the event handling system requires an 'action program', and the Stingray Virtual Appliance does not include the necessary third-party libraries to conduct the OAuth-authenticated transaction.  This example also illustrates how you can construct and perform OAuth authentication from TrafficScript.   Getting started - Create your Twitter 'Application'   Tp get started, you'll need to create a twitter 'Application' that will receive and process your twitter status updates.  Follow the instructions in the Traffic Managers can Tweet Too article, up to the point where you have an application with the two public/secret key pairs:   The TrafficScript Rule   The following TrafficScript rule posts status updates using your application's and user's OAuth credentials.  The document Authorizing a request | Twitter Developers describes the process for authorizing a request using OAuth to the twitter service.   The rule is for test and demonstration purposes.  If you assign it to an HTTP virtual server (as a request rule), it will intercept all requests and look for a query-string parameter called 'status'.  If that parameter exists, it will post the value of status to twitter using the credentials in the rule.  Make sure to update the rule with the correct parameters from your twitter application.   The rule uses the TrafficScript HMAC library described in the document HowTo: Calculating HMAC hashes in TrafficScript.   # TrafficScript Twitter Client import libHMAC.rts as hmac; # Key client parameters # https://dev.twitter.com/apps/<app-id>/oauth $oauth_consumer_key = "Pplp3z4ogRW4wKP3YOlAA"; $oauth_token = "1267105740-xhMWKdsNqoKAof7wptTZ5PmNrodBJcQm1tQ5ssR"; $consumer_secret = "jbWlWYOSgzC9WXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"; $oauth_token_secret = "p8GCJUZLXk1AeXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"; $method = "POST"; $url = "https://api.twitter.com/1/statuses/update.json"; #------- $status = http.getFormParam( "status" ); if( ! $status ) { http.sendResponse( 200, "text/plain", "Please provide a status in the querystring (?status=...)", "" ); } $oauth = [ "oauth_consumer_key" => $oauth_consumer_key, "oauth_nonce" => string.base64encode( string.randomBytes( 32 )), "oauth_signature_method" => "HMAC-SHA1", "oauth_timestamp" => sys.time(), "oauth_token" => $oauth_token, "oauth_version" => "1.0" ]; $pstring = ""; foreach( $k in array.sort( hash.keys( $oauth ))) { $pstring .= string_escapeStrict( $k ) . "=" . string_escapeStrict( $oauth[ $k ] ) . "&"; } $pstring .= "status=". string_escapeStrict( $status ); $sbstring = string.uppercase( $method ) ."&" . string_escapeStrict( $url ) . "&" . string_escapeStrict( $pstring ); $skey = string_escapeStrict( $consumer_secret ) . "&" . string_escapeStrict( $oauth_token_secret ); $oauth["oauth_signature"] = string.base64encode( hmac.SHA1( $skey, $sbstring ) ); $body = "status=". string_escapeStrict( $status ); $oauthheader = "OAuth "; foreach( $k in hash.keys( $oauth )) { $oauthheader .= string_escapeStrict( $k ) . "=\"" . string_escapeStrict( $oauth[ $k ] ) . "\", "; } $oauthheader = string.drop( $oauthheader, 2 ); $r = http.request.post( $url, $body, "Authorization: ".$oauthheader, 10 ); $response = "Response code: ".$1." (".$4.")\n" . "Content Type: ".$2."\n" . "\n" . $r . "\n\n" . "I sent: POST " . $url ."\n" . $body . "\n\n" . " pstring: ".$pstring."\n". " sbstring: ".$sbstring."\n". " skey: ".$skey."\n". " signature: ".$oauth["oauth_signature"]."\n\n". $oauthheader ."\n"; http.sendResponse( 200, "text/plain", $response, "" ); # %-encoding to the strict standards of RFC 3986 sub string_escapeStrict( $a ) { $r = ""; while( string.length( $a ) ) { $c = string.left( $a, 1 ); $a = string.skip( $a, 1 ); $i = ord( $c ); if( ($i >= 0x30 && $i <= 0x39 ) || ( $i >= 0x41 && $i <= 0x5A ) || ( $i >= 0x61 && $i <= 0x7A ) || $i == 0x2D || $i == 0x2E || $i == 0x5F || $i == 0x7E ) { $r .= $c; } else { $h = ($i & 0xF0 ) >> 4; $l = ($i & 0x0F ); $r .= "%" . string.substring( "0123456789ABCDEF", $h, $h ) . string.substring( "0123456789ABCDEF", $l, $l ); } } return $r; }   Submit a request to this virtual server, and if all is well you should get a 200 status code response:   ... along with your first 'Hello, World' from TrafficScript:     One thing to note - twitter does rate-limit and de-dupe tweets from the same source, so repeatedly submitting the same URL without changing the query string each time is not going to work so well.   Good luck!
View full article
If tweeting is good enough for the Pope, then Stingray can certainly get in on the game!  If you use tools to monitor Twitter accounts, this article will explain how you can direct Stingray's event log messages to a nominated twitter account so that you can be alerted immediately a key event occurs:     You might also like to check out the companion article - TrafficScript can Tweet Too.   Creating the Twitter Application   Posting to Twitter is now a little involved.  You need to create an application with the correct access (Read + Write), and then create the appropriate oauth parameters so that clients can authenticate themselves to this application.   To start off, log on to https://dev.twitter.com/apps and create an application as follows:     Follow the steps in https://dev.twitter.com/docs/auth/tokens-devtwittercom to create the access tokens so that you can post to your application from your own twitter account.   The OAuth authentication for your application requires two public/secret key pairs that you can generate from the management website:     Configuring Stingray   Create the event action   Download and edit the following event handling script, updating the oauth tokens to match your own application and access tokens:   #!/usr/bin/python import sys import socket import oauth2 as oauth def oauth_req(url, http_method="GET", post_body=None, http_headers=None): consumer = oauth.Consumer(key="Pplp3z4ogRW4wKP3YOlAA", secret="jbWlWYOSgzC9XXXXXXXXXXXXXXXX") token = oauth.Token(key="1267105740-xhMWKdsNqoKAof7wptTZ5PmNrodBJcQm1tQ5ssR", secret="p8GCJUZLXk1AeXXXXXXXXXXXX") client = oauth.Client(consumer, token) resp, content = client.request( url, method=http_method, body=post_body, headers=http_headers ) return content msg = sys.argv[-1] words = msg.split( '\t' ) host = socket.gethostname().split( '.' )[0] update = oauth_req( "https://api.twitter.com/1.1/statuses/update.json", "POST", "status=" + words[0] + ": " + words[-1] + " (" + words[-2] + " " + words[1].split('/')[-1] + ") #" + host )   From your Stingray Admin Server, create a new External Program Action named 'Tweet', and upload the edited event handling script.   Note that the script won't work on the Stingray Virtual Appliance (because it does not have a python interpreter), and on a Linux or Solaris host, you will need to install the python oauth2 module ( pip install oauth2 ).   Test the event out with the: 'Update and Test' button in the edit page for the Tweet action, and review the event log to confirm that there are no errors:     You should see the test tweet in your twitter feed:     Wire the action to the appropriate events   Wire up an event handler to trigger the Tweet action when the appropriate events are raised:     ... and then site back and follow your twitter stream to see what's up with your Stingrays.
View full article
Stingray's Feature Brief: Stingray Content Caching capability allows the traffic manager to operate as a high-performance proxy cache, with full control using TrafficScript. When Stingray is deployed in a cluster, each Stingray device manages its own local cache.  When an item expires from one Stingray's cache, it retrieves that item from the origin server the next time that it is used. Each Stingray manages its own cache independently In the majority of situations, this deployment pattern is entirely appropriate.  In some specialized situations, it is preferable to have 100% consistency between the caches in the traffic managers and to absolutely minimize the load on the origin servers.  This article describes the recommended deployment configuration, meeting the following technical goals: Fault Tolerance - Failure: the pair of Stingray Traffic Managers operate in a fully-fault tolerant fashion.  Cache consistency between peers ensures that in the event of planned or unintentional failure of one of the peers in the cluster, the remaining peer has a complete and up-to-date cache of content and the load on the origin servers does not increase. Fault Tolerance - Recovery: when the failed Stingray traffic manager recovers, it is automatically reintroduced into the cluster.  Its cache is automatically repopulated over a period of time (corresponding to the cache refresh period) without increasing the load on the origin servers. Load Minimization: the configuration minimizes the load on the origin servers.  In particular, one peer member will never request content from the origin server when the other peer has content that is still current. Stingray’s cache flood protection is used in this configuration to manage the effects of multiple simultaneous requests for the same content that is no longer fresh (defined by a cache refresh time).  If multiple requests arrive at the same instant, a maximum of one request per second will be forwarded to the origin servers. Operational Benefits The configuration delivers the following operational benefits: Resilience: The service is fully resilient in the event of planned or unplanned failure of one traffic manager, or one or more origin servers. Optimal end user performance: the load on the origin servers is minimized in accordance with the fully-user-configurable cache policy, meaning that the origin servers are most unlikely to be overloaded and end user performance will not suffer. ... letting you deliver reliable services with predictable performance whilst minimizing infrastructure costs and eliminating overspend to accommodate occasional spikes of traffic or visitor numbers. Solution Architecture Stingray Traffic Managers are deployed in a fault-tolerant pair, and external traffic directed to a public-facing IP address (“Traffic IP address”).  Both traffic managers are active, and the one that owns the traffic IP handles incoming traffic: Failover Scenarios During normal operation, the primary traffic manager receives traffic and responds directly from its local cache.  If a cache miss occurs (i.e when priming the cache, or when content expires or needs refreshing), the traffic manager first checks with its secondary peer and retrieves the secondary’s cached version if that is still valid (not expired or needing refreshed).  It caches the copy retrieved from the secondary according to the local cache policy. If the secondary does not have valid content, then the resource will be retrieved from the origin servers (load balancing as appropriate) and cached by both traffic managers. If the secondary traffic manager fails, the primary will continue to respond directly from its local cache whenever possible.  If a cache miss occurs or a refresh is needed, content is retrieved from the origin servers and cached in the primary traffic manager’s cache. When the secondary traffic manager recovers, it may have an out-of-date cache (in the event of a network failure), or the cache may be completely empty (in the event of a software restart).  This cache is fully repopulated with the working set of documents within the configured refresh time.  There is no risk of the traffic manager serving stale content, and the load on the origin server is not increased during the repopulation. If the primary traffic manager fails, the secondary will take over with a fully primed cache and respond to all traffic.  If a cache miss occurs or a refresh is needed, the secondary will retrieve the content from the origin servers and cache it locally. When the primary traffic manager recovers, its cache may be out-of-date or completely empty (as above).  The primary takes back responsibility for user traffic and will update its cache rapidly by retrieving fresh content from the secondary traffic manager on every cache miss or refresh.  There is no risk of the primary traffic manager returning stale content, and the load on the origin servers is not increased during the repopulation. Stingray Configuration The configuration instructions are based on the following context: Two traffic managers, named ‘stingray-1’ and ‘stingray-2’ are configured in a fault tolerant cluster. A single-hosted traffic IP group with one IP address (‘traffic IP’) is configured to receive incoming traffic, and ‘stingray-2’ is designated a passive member of that group.  External DNS is configured to resolve to the traffic IP address.  This ensures that in normal operation, stingray-1 will be the active traffic manager with respect to handling user traffic. Content is stored on three webservers (webserver1, webserver2, webserver3) and the intent is that requests are load-balanced across these. The desired cache policy is to cache HTTP content for a long period of time (99915 seconds), but refresh content with one origin request every 15 seconds. Configure the pools Configure two pools as follows:   Name: Origin Servers, pool members: webserver1:80, webserver2:80, webserver3:80   Name: Cache Servers, pool members: stingray-2:80, webserver1:80, webserver2:80, webserver3:80 Configure both pools to use an appropriate load balancing algorithm.  Additionally, configure priority lists for the Cache Servers pool as follows: Cache Servers pool: priority list configuration This configuration ensures that when the Cache Servers pool is selected, all traffic is sent to stingray-2 if it is available, or is load-balanced across the three origin webservers if not. Note: if any changes are made to the origin servers (nodes are added or removed), both the Origin Servers and Cache Servers pools must be updated. Configure the Virtual Server Create an HTTP virtual server named Cache Service, and set the default pool to Origin Servers.  Note: This virtual server should generally listen on all IP addresses.  If it’s necessary to restrict the IP addresses it listens on, it should listen on the public traffic IP and on the IP address that is resolved by ‘stingray-2’ (the first node in the Cache Servers pool). Configure caching in the virtual server to cache content for the desired period (e.g. 99915 seconds), and to refresh it (with a maximum of one request per second) once it has been cached for 15 seconds, as follows: Caching settings for the Cache Service virtual server Add a request rule named Synchronize Cache to the virtual server to select the Cache Servers pool (overriding the default Origin Servers pool). This rule selects the Cache Servers pool when traffic is received on the primary traffic manager (‘stingray-1’). Note: you will need to update this rule if the hostname for the primary traffic manager is not stingray-1. This completes the configuration for the Cache Service virtual server and related pools and rule: Testing the caching service Testing this configuration must be done with due care because the presence of multiple caches can make debugging dificult.  The following techniques are useful: Sample Traffic Test with a small sample set of content to verify that the cache policies function as desired.  For example, the following command will repeatedly request a single cacheable resource: bash$ while sleep 1 ; do wget http://host/cacheablecontent.gif ; done Activity Monitor Use the activity monitor to chart connections made to the origin servers: Note that the activity monitor charts generally merge data from all traffic managers in a cluster; for clarity, this chart plots traffic individually per traffic manager.  Observe that despite the front-end load of 1 request per second (not plotted), only one request every 15 seconds is sent to the origin server to refresh the cache.  All requests originate from stingray-2. If Stingray-2 is suspended or shut down, the activity monitor report run from stingray-1 will verify that there is no increase in origin server traffic; likewise if stingray-1 fails and the activity monitor chart is run from stingray-2. Connections report The connections report will assist in identifying where traffic is routed. In the following report, requests for the same 866-byte resource were issued once a second, and received by stingray-1 and responded from the local cache (‘ To ’ is ‘ none ’). At 10:47:25, a cache refresh event occurred.  Stingray-1 forwarded the request to stingray-2 (192.168.35.11).  Stingray-2’s cache also required a refresh (because the caches are synchronized) so stingray-2 requested the content from one of the origin servers (192.168.207.103). Conclusion The synchronization solution effectively meets the goals of reliability, performance and minimization of load on the origin servers.  It may be extended from an active-active pair of Stingray traffic managers to a larger cluster if required, which will increase the level of redundancy in the system, but at the expense of a small (probably insignificant) increase in latency as caches must be synchronized across a larger set of devices. Priming the cache It is generally not a good idea to pre-prime a cache because the act of priming the cache puts a large one-time load on the origin servers.  In the majority of situations, it is better to use this synchronization solution and allow the caches to fill on demand, in response to end user traffic. If it is necessary to pre-prime the cache, this can be done using synthetic transactions to submit requests through the stingray cluster.  For example, ‘wget –r’ was used with success during advanced testing of this solution. 
View full article
SOA applications need just as much help as traditional web applications when it comes to reliability, performance and traffic management. This article provides four down-to-earth TrafficScript examples to show you how you can inspect the XML messages and manage SOA transactions. Why is XML difficult? SOA traffic generally uses the SOAP protocol, sending XML data over HTTP. It's not possible to reliably inspect or modify the XML data using simple tools like search-and-replace or regular expressions. In Computer Science terms, regular expressions match regular languages, whereas XML is a much more structured context-free language. Instead of regular expressions, standards like XPath and XSLT are used to inspect and manipulate XML data. Using TrafficScript rules, Stingray can inspect the payload of an SOAP request or response and use XPath operations to extract data from it, making traffic management decisions on this basis. Stingray can check the validity of XML data, and use XSLT operations to transform the payload to a different dialect of XML. The following four articles give examples of traffic inspection and management in an SOA application contect. Other examples of XML processing include embedding RSS data in an HTML document. Routing SOAP traffic Let’s say that the network is handling requests for a number of different SOAP methods. The traffic manager is the single access point – all SOAP traffic is directed to it. Behind the scenes, some of the methods have dedicated SOAP servers because they are particularly resource intensive; all other methods are handled by a common set of servers. The following example uses Stingray's pools. A pool is a group of servers that provide the same service. Individual pools have been created for some SOA components, and a ‘SOAP-Common-Servers’ pool contains the nodes that host the common SOA components. # Obtain the XML body of the SOAP request $request = http.getBody(); $namespace = "xmlns:SOAP-ENV=\" http://schemas.xmlsoap.org/soap/envelope/ \""; $xpath = "/SOAP-ENV:Envelope/SOAP-ENV:Body/*[1]"; # Extract the SOAP method using an XPath expression $method = xml.XPath.matchNodeSet( $request, $namespace, $xpath ); # For ‘special’ SOAP methods, we have a dedicated pool of servers for each if( pool.getActiveNodes( "SOAP-".$method ) > 0 ) {    pool.select( "SOAP-".$method ); } else {    pool.select( "SOAP-Common-Servers" ); } TrafficScript: Routing SOAP requests according to the method Why is this useful? This allows you to deploy SOA services in a very flexible manner. When a new instance of a service is added, you do not need to modify every caller that may invoke this service. Instead, you need only add the service endpoint to the relevant pool. You can rapidly move a service from one server to another, for resourcing or security reasons (red zone, green zone), and an application can be easily built from services that are found in different locations. Ensuring fair access to resources With Stingray, you can also monitor the performance of each pool of servers to determine which SOAP methods are running the slowest. This can help troubleshoot performance problems and inform decisions to re-provision resources where they are needed the most. You can shape traffic – bandwidth or transactions per second – to limit the resources used and smooth out flash floods of traffic. With the programmability, you can shape different types of traffic in different ways. For example, the following TrafficScript code sample extracts a ‘username’ node from the SOAP request. It then rate-shapes SOAP requests so that each remote source (identified by remote IP address and ‘username’ node value) can submit SOAP requests at a maximum of 60 times per minute: # Obtain the source of the request $ip = request.getRemoteIP(); # Obtain the XML body of the SOAP request $request = http.getBody(); $namespace = "xmlns:SOAP-ENV=\" http://schemas.xmlsoap.org/soap/envelope/ \""; $xpath = "/SOAP-ENV:Envelope/SOAP-ENV:Body/*/username/text()"; # Extract the username using an XPath expression $username = xml.XPath.matchNodeSet( $request, $namespace, $xpath ); # $key uniquely identifies this type of request from this source. $key = $ip . ", " . $username; # The 'transactions' rate shaping class limits each type to 60 per minute rate.use( "transactions", $key ); TrafficScript: Rate-shaping different users of SOAP traffic Why is this important? An SOA component may be used by multiple different SOA applications. Different applications may have different business priorities, so you might wish to prioritize some requests to a component over others. Applying ‘service governance’ policies using Stingray's rate shaping functionality ensures that all SOA applications get fair and appropriate access to critical components, and that no one application can overwhelm a component to the detriment of other applications. This can be compared to time-sharing systems – each SOA application is a different ‘user’, and users can be granted specific access to resources, with individual limits where required. When some SOA applications are externally accessible (via a web-based application for example), this is particularly important because a flash flood or malicious denial-of-service attack could ripple through, affecting many internal SOA components and internal applications. Securing Traffic Suppose that someone created a web services component for a travel company that enumerated all of the possible flights from one location to another on a particular day. The caller of the component could specify how many hops they were prepared to endure on the journey. Unfortunately, once the component was deployed, a serious bug was found. If a caller asked for a journey with the same start and finish, the component got stuck in an infinite loop. If a caller asked for a journey with a large number of hops (1000 hops perhaps), the computation cost grew exponentially, creating a simple, effective denial of service attack. Fixing the component is obviously the preferred solution, but it’s not always possible to do so in a timely fashion. Often, procedural barriers make it difficult to make changes to a live application. However, by controlling and manipulating the SOA requests as they travel over the network, you can very quickly roll out a security rule on your SDC to drop or modify the SOAP request. Here’s a snippet: $request = http.getBody(); $from = xml.XPath.matchNodeSet( $request, $namespace, "//from/text()" ); $dest = xml.XPath.matchNodeSet( $request, $namespace, "//dest/text()" ); # The error response; can read a precanned response from disk and return # it as a SOAP response if( $from == $dest ) {    $response = resource.get( "FlightPathFaultResponse.xml" );    connection.sendResponse( $response ); } $hops = xml.XPath.matchNodeSet( $request, $namespace, "//maxhops/text()" ); if( $hops > 3 ) {    # Apply an XSLT that sets the hops node to 3    $transform = resource.get( "FlightPath3Hops.xslt" );    http.setBody( xml.XSLT.transform( $request, $transform ) ); } TrafficScript: Checking validity of SOAP requests Why is this important? Using the Service Delivery Controller to manage and rewrite SOA traffic is a very rapid and lightweight alternative to rewriting SOA components. Patching the application in this way may not be a permanent solution, although it’s often sufficient to resolve problems. The real benefit is that once a fault is detected, it can be resolved quickly, without requiring in-depth knowledge of the application. Development staff need not be pulled away from other projects immediately. A full application-level fix can wait until the staff and resources are available; for example, at the next planned update of the component code. Validating SOAP responses If a SOAP server encounters an error, it may still return a valid SOAP response with a ‘Fault’ element inside. If you can look deep inside the SOAP response, you’ve got a great opportunity to work around such transient application errors. If a server returns a fault message where the faultcode indicates there was a problem with the server, wouldn’t it be great if you could retry the request against a different SOAP server in the cluster? $response = http.getResponseBody(); $ns = "xmlns:SOAP-ENV=\" http://schemas.xmlsoap.org/soap/envelope/ \""; $xpath = "/SOAP-ENV:Envelope/SOAP-ENV:Body/SOAP-ENV:Fault/faultcode/text()"; $faultcode = xml.XPath.matchNodeSet( $request, $namespace, $xpath ); if( string.endsWith( $faultcode, "Server" ) ) {    if( request.retries() < 2 ) {       request.avoidNode( connection.getNode() );       request.retry();    } } TrafficScript: If we receive a Server fault code in the SOAP response, retry the request at most 2 times against different servers Why is this important? This particular example shows how the error checking used by an SDC can be greatly extended to detect a wide range of errors, even in responses that appear “correct” to less intelligent traffic managers. It is one example of a wide range of applications where responses can be verified, scrubbed and filtered. Undesirable responses may include fault codes, sensitive information (like credit card or social security numbers), or even incorrectly-localized or formatted responses that may be entirely legitimate, but cannot be interpreted by the calling application. Pinpointing errors in a loosely-coupled SOA application is a difficult and invasive process, often involving the equivalent of adding ‘printf’ debug statements to the code of individual components. By inspecting responses at the network, it becomes much easier to investigate and diagnose application problems and then work round them, either by retrying requests or transforming and rewriting responses as outlined in the previous example.
View full article
Bandwidth can be expensive. So it can be annoying if other websites steal your bandwidth from you. A common problem is when people use 'hot-linking' or 'deep-linking' to place images from your site on to their own pages. Every time someone views their website, you will pick up the bandwidth tab, and users of your website may be impacted because of the reduced bandwidth. So how can this be stopped? When a web browser requests a page or an image from your site, the request includes a ' Referer ' header (The misspelling is required in the specs!). This referrer gives the URL of the page that linked to the file. So, if you go to https://splash.riverbed.com/ , your browser will load the HTML for the page, and then load all the images. Each time it asks the web server for an image, it will report that the referrer was https://splash.riverbed.com/ . We can use this referrer header to check that the image is being loaded for your own site, and not for someone else's. If another website embedded a link to one of these images, the Referer: header would contain the URL of their site instead. This site has a more in-depth discussion of bandwidth-stealing; the Stingray approach is an alternative to the Apache solution it presents. Solving the problem with RuleBuilder RuleBuilder is a simple, GUI front-end to TrafficScript that lets you create straightforward 'if condition then action'-style policies.  Use the Stingray Admin Server to create a new RuleBuilder rule as follows: You should then associate that with your virtual server, configuring it to run as a Request Rule: All done. This rule will catch any hotlink requests for content where the URL ends with ' .gif ', ' .jpg ' or ' .png ', and redirect to the image: http://upload.wikimedia.org/wikipedia/commons/thumb/2/2a/Stop_sign.svg/200px-Stop_sign.svg.png TrafficScript improvements We can make some simple improvements to this rule: We can provide a simple list of file extensions to check against, rather than using a regular expression.  This is easier to manage, though not necessarily faster We can check that the referer matches the host header for the site.  That is a simple approach that avoids embedding the domain (e.g. riverbed.com) in the rule, thus making it less likely to surprise you when you apply the rule to a different website First convert the rule to TrafficScript.  That will reveal the implementation of the rule, and you can edit the TrafficScript version to implement the additional features you require: $headerReferer = http.getheader( "Referer" ); $path = http.getpath(); if( string.contains( $headerReferer, "riverbed.com" ) == 0         && $headerReferer != ""         && string.regexmatch( $path, "\\.(jpg|gif|png)$" ) ){         http.redirect( " http://upload.wikimedia.org/wikipedia/commons/thumb/2/2a/Stop_sign.svg/200px-Stop_sign.svg.png " ); } The RuleBuilder rule, converted to TrafficScript Edit the rule so that it resembles the following: $extensions = [ 'jpg', 'jpeg', 'gif', 'png', 'svg' ]; $redirectto = " http://upload.wikimedia.org/wikipedia/commons/thumb/2/2a/Stop_sign.svg/200px-Stop_sign.svg.png "; ####################################### $referer = http.getheader( "Referer" ); $host    = http.getHostHeader(); $path    = http.getpath(); $ext     = ""; if( string.regexMatch( $path, '\.(.*?)$' ) ) $ext = $1; if( array.contains( $extensions, $ext )    && $referer != ""    && !string.contains( $referer, $host )    && $path != $redirectto ) {       http.redirect( $redirectto ); } Alternate rule implementation
View full article