Managing the growth of website content

[#1]

As the content on websites grow, the structure of their URLs can change dramatically.  The addition of new applications and components can play havoc with the established URL space, and the development cost of supporting the old links that have been published in articles, referenced by search engines and bookmarked by users can be very high.

Stingray is in a great place to address this problem, and let your applications use the URL spaces most suited to them with little concern for backwards compatibility.  This article presents one technique you can employ to access this problem in a scalable and manageable manner.

A simple example

Let's start with a simple example; suppose you published content at the following URLs:

www.example.com/news.html

www.example.com/about.html

www.example.com/careers.html

www.example.com/demos.html

As your company grew, offices were added, more products were developed and content began to be broken out by location.  The original URL structure was no longer sustainable, and a deeper layer of content was necessary:

www.example.com/corporate/news.html

www.example.com/product/news.html

www.example.com/product/demos.html

www.example.com/about/cambridge.html

www.example.com/about/honalulu.html

www.example.com/careers/overview.html

www.example.com/careers/honalulu/engineering.html

www.example.com/careers/cambridge/research.html

The challenge is to serve out the best content to people who make requests to the old URLs.  You want a better solution than manually adding redirects to your webserver configuration file; ideally a solution that can be used by your web content team without the intervention from IT.

A solution

Ideally, we would like to issue an HTTP redirect to the most appropriate page. This is, of course, simple using TrafficScript.

1	http.redirect( "http://www.example.com/corporate/news.html" );

... but you want to avoid having to build and maintain a rule that looks like this:

1 2 3 4 5 6 7 8 9 10 11 12 13

$url = http.getPath(); if( $url == "/news.html" ){    http.redirect( "http://www.example.com/product/news.html" ); } else if( $url == "/about.html" ) {    http.redirect( "http://www.example.com/about/cambridge.html" ); } else if ... }

Wouldn't it be easier if you could maintain a file with a list of redirects, and Stingray could act on that?

/news.html http://www.example.com/product/news.html

/about.html http://www.example.com/about/cambridge.html

etc...?

You can use the ResourceTable libraries from HowTo: Store tables of data in TrafficScript - part 1 to help you do exactly that:

1 2 3 4 5 6 7

import ResourceTableSmall as table;  $path = http.getPath();  $redirect = table.lookup( "redirects.txt", $path ); if( $redirect ) http.redirect( $redirect );

Managing the file of redirects

That leaves us with one problem - how best to manage the (albeit simple) file that contains the redirects?  The format is simple enough (space-separated key / value pairs, one per line) that anyone can edit it, but how do they get it into the Stingray configuration without using the complex and powerful Stingray Admin Interface?

There are a couple of simple approaches:

• Use the Stingray Admin Interface, and give the user a permissions group that only gives access to the Extra Files catalog;
• Push the file in using one of Stingray's configuration APIs - Feature Brief: Stingray's RESTful Control API or Feature Brief: Stingray's SOAP Control API;
• Copy the file in using SSH, and then invoke ZEUSHOME/zxtm/bin/replicate-config to push the configuration across the cluster.

The REST approach is particularly attractive - you can use browser plugins like Chrome's REST Console to push configuration files into Stingray.

However, you may not want someone to use either approach directly because it would imply giving them a full administrative logon to the Stingray cluster.  In that case, you could consider a simple commandline tool that they can use to upload the updated configuration file, using the Collected Tech Tips: Using the RESTful Control API or  Collected Tech Tips: SOAP Control API examples APIs.