libTable.rts: Interrogating tables of data in TrafficScript

[#1]

This article presents a TrafficScript library that give you easy and efficient access to tables of data stored as files in the Stingray configuration:

libTable.rts

Download the following TrafficScript library from gihtub and import it into your Rules Catalog, naming it libTable.rts:

libTable.rts

# libTable.rts
#
# Efficient lookups of key/value data in large resource files (>100 lines) 
# Use getFirst() and getNext() to iterate through the table
 
sub lookup( $filename, $key ) {      
   update( $filename );       
   $pid = sys.getPid();      
   return data.get( "resourcetable".$pid.$filename."::".$key );   
}  
 
sub getFirst( $filename ) {
   update( $filename );
   $pid = sys.getPid();      
   return data.get( "resourcetable".$pid.$filename.":first" );
}
 
sub getNext( $filename, $key ) {
   update( $filename );
   $pid = sys.getPid();      
   return data.get( "resourcetable".$pid.$filename.":next:".$key );
}
 
# Internal functions
 
sub update( $filename ) {      
   $pid = sys.getPid();       
   $md5 = resource.getMD5( $filename );   
   if( $md5 == data.get( "resourcetable".$pid.$filename.":md5" ) ) return;       
  
   data.reset( "resourcetable".$pid.$filename.":" );      
   data.set( "resourcetable".$pid.$filename.":md5", $md5 );       
     
   $contents = resource.get( $filename );      
 
   $pkey = "";
   foreach( $l in string.split( $contents, "\n" ) ) {  
      if( ! string.regexmatch( $l, "(.*?)\\s+(.*)" ) ) continue;  
      $key = string.trim( $1 );         
      $value = string.trim( $2 );          
      data.set( "resourcetable".$pid.$filename."::".$key, $value );      
      
      if( !$pkey ) {
         data.set( "resourcetable".$pid.$filename.":first", $key );
      } else {
         data.set( "resourcetable".$pid.$filename.":next:".$pkey, $key );
      }
      $pkey = $key;
   }   
}

Usage:

import libTable.rts as table;  
  
$filename = "data.txt";  
  
# Look up a key/value pair  
$value = table.lookup( $filename, $key );  
  
# Iterate through the table  
for( $key = table.getFirst( $filename ); $key != ""; $key = table.getNext( $filename, $key ) ) {  
  $value = table.lookup( $filename, $key );  
}  

The library caches the contents of the file internally, and is very efficient for large files.  For smaller files, it may be slightly more efficient to search these files using a regular expression, but the convenience of this library may outweigh the small performance gains.

Data file format

This library provides access to files stored in the Stingray conf/extra folder (by way of the Extra Files > Miscellaneous Files) section of the catalog.  These files can be uploaded using the UI, the SOAP or REST API, or by manually copying them in place and initiating a configuration replication.

Files should contain  key-value pairs, one per line, space separated:

key1value1

key2value2

key3value3

Preservation of order

The lookup operation uses an open hash table, so is efficient for large files. The getFirst() and getNext() operations will iterate through the data table in order, returning the keys in the order they appear in the file.

Performance and alternative implementations

The performance of this library is investigated in the article Investigating the performance of TrafficScript - storing tables of data.  It is very efficient for large tables of data, and marginally less efficient than a simple regular-expression string search for small files.

If performance is of a concern and you only need to work with small datasets, then you could use the following library instead:

libTableSmall.rts

# libTableSmall.rts: Efficient lookups of key/value data in a small resource file (<100 lines) 

sub lookup( $filename, $key ) { 
   $contents = resource.get( $filename ); 
   if( string.regexmatch( $contents, '\n'.$key.'\s+([^\n]+)' ) ) return $1; 
   if( string.regexmatch( $contents, '^'.$key.'\s+([^\n]+)' ) ) return $1; 
   return ""; 
}