About yvan

yvan · ‎04-18-2013

To get started quickly with Python on Stingray Traffic Manager, go straight to PyRunner.jar: Running Python code in Stingray Traffic Manager. This article dives deep into how that extension was developed. As is well documented, we support the use of Java extensions to manipulate traffic. One of the great things about supporting "Java" is that this really means supporting the JVM platform... which, in turn, means we support any language that will run on the JVM and can access the Java libraries. Java isn't my first choice of languages, especially when it comes to web development. My language of choice in this sphere is Python; and thanks to the Jython project we can write our extensions in Python! Jython comes with a servlet named PyServlet which will work with Stingray out-of-the-box, this is a good place to start. First let us quickly set up our Jython environment, working Java support is a prerequisite of course. (If you're not already familiar with Stingray Java extensions I recommend reviewing the Java Development Guide available in the Stingray Product Documentation and the Splash guide Writing Java Extensions - an introduction ) Grab Jython 2.5 (if there is now a more recent version I expect it will work fine as well) Jython comes in the form of a .jar installer, on Linux I recommend using the CLI install variant: java -jar jython_installer-2.5.0.jar -c Install to a path of your choosing, I've used: /space/jython Upload /space/jython/jython.jar to your Java Extensions Catalog In the Java Extensions Catalog you can now see a couple of extensions provided by jython.jar, including org.python.util.PyServlet. By default this servlet maps URLs ending in .py to local Python files which it will compile and cache. Set up a test HTTP Virtual Server (I've created a test one called "Jython" tied to the "discard" pool), and add the following request rule to it: if (string.endsWith(http.getPath(), ".py")) { java.run( "org.python.util.PyServlet" ); } The rule avoids errors by only invoking the extension for .py files. (Though if you invoke PyServlet for a .py file that doesn't exist you will get a nasty NullPointerException stack trace in your event log.) Next, create a file called Hello.py containing the following code: from javax.servlet.http import HttpServlet class Hello(HttpServlet): def doGet(self, request, response): toClient = response.getWriter() response.setContentType ("text/html") toClient.println("<html><head><title>Hello from Python</title>" + "<body><h1 style='color:green;'>Hello from Python!</h1></body></html>") Upload Hello.py to your Java Extensions Catalog. Now if you visit the file Hello.py at the URL of your VirtualServer you should see the message "Hello from Python!" Hey, we didn't have to compile anything! Your event log will have some messages about processing .jar files, this only happens on first invocation. You will also get a warning in your event log every time you visit Hello.py: WARN java/* servleterror Servlet org.python.util.PyServlet: Unknown attribute (pyservlet) This is just because Stingray doesn't set up or use this particular attribute. We'll ignore the warning for now, and get rid of it when we tailor PyServlet to be more convenient for Stingray use. The servlet will also have created some some extra files in your Java Libraries & Data Catalog under a top-level directory WEB-INF. It is possible to change the location used for these files, we'll get back to that in a moment. All quite neat so far, but the icing on the cake is yet to come. If you open up $ZEUSHOME/conf/jars/Hello.py in your favourite text editor and change the message to "Hello <em>again</em> from Python!" and refresh your browser you'll notice that the new message comes straight through. This is because the PyServlet code checks the .py file, if it is newer than the cached bytecode it will re-interpret and re-cache it. We now have somewhat-more-rapid application development for Stingray extensions. Bringing together the excellence of Python and the extensiveness of Java libraries. However, what I really want to do is use TrafficScript at the top level to tell PyServlet which Python servlet to run. This will require a little tweaking of the code. We want to get rid of the code that resolves .py file from the request URI and replace it with code that uses an argument passed in by TrafficScript. While we're at it, we'll make it non-HTTP-specific and add some other small tweaks. The changes are documented in comments in the code, which is attached to this document. To compile the Stingray version of the servlet and pop the two classes into a single convenient .jar file execute the following commands. $ javac -cp $ZEUSHOME/zxtm/lib/servlet.jar:/space/jython/jython.jar ZeusPyServlet.java $ jar -cvf ZeusPyServlet.jar ZeusPyServlet.class ZeusPyServletCacheEntry.class (Adjusting paths to suit your environment if necessary.) Upload the ZeusPyServlet.jar file to your Java Extensions Catalog. You should now have a ZeusPyServlet extension available. Change your TrafficScript rule to load this new extension and provide an appropriate argument. if (string.endsWith(http.getPath(), ".py")) { java.run( "ZeusPyServlet", "Hello.py" ); } Now visiting Hello.py works just as before. In fact, visiting any URL that ends in .py will now generate the same result as visiting Hello.py. We have complete control over what Python code is executed from our TrafficScript rule, much more convenient. If you continue hacking from this point you'll soon find that we're missing core parts of python with the setup described so far. For example adding import md5 to your servlet code will break the servlet, you'd see this in your Stingray Event Log: WARN servlets/ZeusPyServlet Servlet threw exception javax.servlet.ServletException: Exception during init of /opt/zeus/zws/zxtm/conf/jars/ServerStatus.py WARN Java: Traceback (most recent call last): WARN Java: File "/opt/zeus/zws/zxtm/conf/jars/ServerStatus.py", line 2, in <module> WARN Java: import md5 WARN Java: ImportError: No module named md5 This is because the class files for the core Python libraries are not included in jython.jar. To get a fully functioning Jython we need to tell ZeusPyServlet where Jython is installed. To do this you must have Jython installed on the same machine as the Stingray software, and then you just have to set a configuration parameter for the servlet, in summary: Install Jython on your Stingray machine, I've installed mine to /space/jython In Catalogs > Java > ZeusPyServlet add some parameters: Parameter: python_home, Value: /space/jython (or wherever you have installed Jython) Parameter: debug, Value: none required (this is optional, it will turn on some potentially useful debug messages) Back in Catalogs > Java you can now delete all the WEB-INF files, now that Jython knows where it is installed it doesn't need this Go to System > Traffic Managers and click the 'Restart Java Runner ...' button, then confim the restart (this ensures no bad state is cached) Now your Jython should be fully functional, here's a script for you to try that uses MD5 functionality from both Python and Java. Just replace the content of Hello.py with the following code. from javax.servlet.http import HttpServlet from java.security import MessageDigest from md5 import md5 class Hello(HttpServlet): def doGet(self, request, response): toClient = response.getWriter() response.setContentType ("text/html") htmlOut = "<html><head><title>Hello from Python</title><body>" htmlOut += "<h1>Hello from Python!</h1>" # try a Python md5 htmlOut += "<h2>Python MD5 of 'foo': %s</h2>" % md5("foo").hexdigest() # try a Java md5 htmlOut += "<h2>Java MD5 of 'foo': " jmd5 = MessageDigest.getInstance("MD5") digest = jmd5.digest("foo") for byte in digest: htmlOut += "%02x" % (byte & 0xFF) htmlOut += "</h2>" # yes, the Stingray attributes are available htmlOut += "<h2>VS: %s</h2>" % request.getAttribute("virtualserver") htmlOut += "</body></html>" toClient.println(htmlOut) An important point to realise about Jython is that beyond the usual core Python APIs you cannot expect all the 3rd party Python libraries out there to "just work". Non-core Python modules compiled from C (and any modules that depend on such modules) are the main issue here. For example the popular Numeric package will not work with Jython. Not to worry though, there are often pure-Python alternatives. Don't forget that you have all Java libraries available too; and even special Java libraries designed to extend Python-like APIs to Jyhon such as JNumeric, a Jython equivalent to Numeric. There's more information on the Jython wiki. I recommend reading through all the FAQs as a starting point. It is perhaps best to think of Jython as a language which gives you the neatness of Python syntax and the Python core with the utility of the massive collection of Java APIs out there.

yvan · ‎12-10-2012

Simply WURFL (08/07/2009) WURFL?! Yes, another acronym: <a title="WURFL" href="http://wurfl.sourceforge.net/" target="_blank">*Wireless Universal Resource FiLe*</a> The WURFL website expands on this: " <a title="What's the WURFL? - Luca Passani" href="http://wurfl.sourceforge.net/backgroundinfo.php" target="_blank">*You can think of the WURFL as a global database of all devices and their capabilities.*</a> " Such a database is especially useful in the current mobile device landscape where a vast array of HTML/WML/Flash/AJAX features are implemented differently (or not at all.) In essence, WURFL makes it easier for you to improve the web experience you offer people when they are on the move. WURFL is the product of much diligence and attention to detail by Luca Passani and many contributors of device information. As an added bonus it is open and free for anyone to use. This is in the best interests of mobile device users everywhere of course! Luckily for us the WURFL project goes all the way and provides a Java API. To get started with WURFL you first need to ensure your Stingray Traffic Manager has working Java support and then download the following items. <a title="Download WURFL" href="http://sourceforge.net/projects/wurfl/files/WURFL/" target="_blank">*wurfl-latest.zip*</a> - the WURFL itself <a title="web_browsers_path.xml" href="http://wurfl.sourceforge.net/web_browsers_patch.xml" target="_blank">*web_browsers_patch.xml*</a> - an optional file that adds data for desktop web browsers <a title="Download WURFL Java API" href="http://sourceforge.net/projects/wurfl/files/WURFL%20Java%20API/" target="_blank">*wurfl-1.0.jar*</a> - get the most recent WURFL .jar file, 1.0 at the time of writing <a title="Collections Downloads" href="http://commons.apache.org/downloads/download_collections.cgi" target="_blank">*commons-collections*</a> - extract the commons-collections- <a title="Lang Downloads" href="http://commons.apache.org/downloads/download_lang.cgi" target="_blank">*commons-lang*</a> - extract the commons-lang- <a title="Logging Downloads" href="http://commons.apache.org/downloads/download_logging.cgi" target="_blank">*commons-logging*</a> - extract the commons-logging- <a title="backport-util-concurrent" href="http://sourceforge.net/projects/backport-jsr166/files/backport-jsr166/" target="_blank">*backport-util-concurrent*</a> - download most appropriate 3.x release (backport-util-concurrent-Java60-3.1.tar.gz in my case) and extract the backport-util-concurrent.jar file from it Upload the files specified using the Calalogs > Java UI page on your Stingray Traffic Manager. Note that the wurfl-latest.zipcan be uploaded as-is, but you can choose to unzip it and upload the wurfl.xml file. Now you're all set to experiment with the following examples. The first sample servlet is a very simple use-case of WURFL, useful as a base for your own servlet or for debugging. The intention is for it to introduce the WURFL API as it fits within the framework of Stingray Java Extensions. WURFL is typically configured using the ConfigListener model, but we're not TomCat and don't go as far as implementing all the nuts and bolts required by full web applications. Our WURFL servlet must perform the required initialisation itself, so I've implemented an init method that sets up a WURFLManager. As much work as possible is done at servlet initialisation time. Then all the doGet method needs to do is check the request against the pre-initialised WURFLManager. You can download <a href="http://blogs.riverbed.com/files/zeuswurflinfoservlet.java" target="_blank">*the code*</a> or the <a href="http://blogs.riverbed.com/files/zeuswurflinfoservlet.class" target="_blank">*compiled class*</a> for this servlet. To compile the code yourself execute the following javac command. This example is given for compiling on the Stingray host system after all the .jar files mentioned above have been uploaded to your Stingray Traffic Manager, you can also compile the servlet on another machine as outlined in our <a href="https://support.riverbed.com/docs/stingray/trafficmanager.htm" target="_blank">*Java Devlopment Guide*</a> . javac -cp "$ZEUSHOME/zxtm/lib/*:$ZEUSHOME/zxtm/conf/jars/wurfl-1.0.jar" \ StingrayWURFLInfoServlet.java To get the servlet working follow these steps: Upload the ZeusWURFLInfoServlet.class file using Stingray Catalogs > Java UI page, leave the "Automatically create TrafficScript rule" checkbox ticked If you want normal web browsers to be detected properly click on the ZeusWURFLInfoServlet link on the UI page and add the parameter "wurfl_patches" with the value "web_browsers_patch.xml" (which you must also upload to the Catalogs > Javapage as covered above) Set up a Virtual Server for testing, I set one up listening on port 60000 and tied it to the special "discard" pool Associate the auto-generated ZeusWURFLInfoServlet rule with the test Virtual Server as a request rule Visit your virtual server in some different web browsers! The result should be a page showing some general information about your browser at the top followed by the full table of WURFL capabilities and their values. The following screenshots show the top part of the output using a few browsers available in the Stingray office. Firefox on Ubuntu Linux Opera on HTC S710 IE on HTC S710 Safari on iPhone This is neat and all as a demo, but not particularly useful. What we really want is a module that can export capabilities so that they can be used by TrafficScript and other extensions. We also don't want to specifically be a doGet processor, so we'll modify the servlet along the lines described in the <a href="https://community.brocade.com/t5/Code-Samples/Writing-TrafficScript-functions-in-Java/td-p/17094" target="_blank">*Writing TrafficScript functions in Java*</a> article. There are a lot of capabilities covered by WURFL, the tables in the above screenshots go on for several pages - more than 500 capabilities in all. So we'll make it possible for TrafficScript to specify what capability fields it wants. This gives us a servlet that could be used from TrafficScript using the following function. sub checkWURFL() { $ua = http.getHeader( "User-Agent" ); if (!string.length($ua)) return; $markup = data.get("WURFL" . $ua . "preferred_markup"); $datarate = data.get("WURFL" . $ua . "max_data_rate"); if (string.length($markup)) { log.info("Returning cached values for User-Agent: " . $datarate . ", " . $markup); $1 = $markup; $2 = $datarate; return; } # no cached values for the UA, so run it through WURFL java.run( "ZeusWURFLServlet", "max_data_rate", "preferred_markup" ); $markup = connection.data.get("preferred_markup"); $datarate = connection.data.get("max_data_rate"); data.set("WURFL" . $ua . "preferred_markup", $markup); data.set("WURFL" . $ua . "max_data_rate", $datarate); log.info("Returning fresh WURFL values for User-Agent: " . $datarate . ", " . $markup); $1 = $markup; $2 = $datarate; return; } # simple case to test the checkWURFL function checkWURFL(); http.sendResponse("200", "text/html", "" . "h1. Max Data Rate: " . $2 . "kbps " . "h1. Preferred Markup: " . $1 . " ", ""); The corresponding Java code and .jar file are available for you to download. <a href="https://community.brocade.com/t5/Articles/Apache-Commons-Logging-08-07-2009/m-p/23438#M101" target="_blank">*Zeus*</a> http://blogs.riverbed.com/files/zeuswurflservlet.javaWURFLServlet.java <a href="http://blogs.riverbed.com/files/zeuswurflservlet.jar" target="_blank">*ZeusWURFLServlet.jar*</a> Rather than the doGet method implemented in the ZeusWURFLInfoServlet we now have a simpler service method. public void service(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { String[] args = (String[])request.getAttribute( "args" ); if (args == null || args.length == 0) { throw new ServletException("error: no arguments supplied"); } Device device = manager.getDeviceForRequest(request); Map capabilities = device.getCapabilities(); for (int i = 0; i < args.length; ++i) { String cap = (String)capabilities.get(args); if (cap == null) { log("no capability found matching: " + args); } else { ((ZXTMServletRequest)request).setConnectionData( args, cap ); } } } Note that for this implementation I've included another class that properly directs org.apache.commons.logging messages to the Stingray Event Log properly. The logging code is <a href="http://blogs.riverbed.com/stingray/2009/08/apache-commons-logging.html" target="_self">*documented here*</a> , you can also download ZeusServletLogger.class here or from that page. The commands used to build the jar file on my Stingray host machine follow. javac -cp "$ZEUSHOME/zxtm/lib/*:$ZEUSHOME/zxtm/conf/jars/wurfl-1.0.jar" \ ZeusWURFLServlet.java jar -cvf ZeusWURFLServlet.jar ZeusWURFLServlet.class ZeusServletLogger.class There is much more you could do with the WURFL Java Extension provided in this article. Think of it as a starting point for developing your own solutions to improve the web browsing experience of your mobile users. A few examples: The max_data_rate value retrieved above could be used to reduce image quality or size for people with low bandwidth devices. This would result in a snappier web browsing experience for these people as there would be less data for them to retrieve over their slow links. The preferred_markup value can be used to direct clients to different backend pools based on whether they can handle XHTML, or should be served WML. The streaming_flv value can be checked to see if the device has Flash video support and can thus be sent to your full bells-and-whistles website. A scaled down version could be made for those that only have Flash Lite, which is specified by the value of flash_lite_version. Devices that don't support flash at all (such as the iPhone) can be sent to a plain HTML version of the site, or WML as in the previous bullet point. Speaking of the iPhone, it doesn't have Flash but its browser does have excellent AJAX support. You know your site is being visited by an iPhone user using the normal iPhone web browser when model_name is "iPhone" and mobile_browser is "Safari". If there are important differences between iPhone OS releases you can also check model_extra_info ordevice_os_version for this detail. For AJAX in general there are a whole set of specific properties: ajax_manipulate_css ajax_manipulate_dom ajax_support_event_listener ajax_support_events ... Up to date documentation on all the WURFL capabilities can be found on the <a title="WURFL Devices and WURFL Capabilities" href="http://wurfl.sourceforge.net/help_doc.php" target="_blank">*WURFL website*</a> . Any of these values can also be passed on to your backend nodes of course. You could add special headers containing the values, a cookie, or a URL argument. You could also cache browser capabilities uniquely to each device with cookies or another method of session tracking, rather than cache the capabilities based solely on the user agent. Then you could offer users the ability to override special mobile device modes. We would love to hear your ideas and learn how we can help you in this exciting area - the opportunities are practically limitless.

Making Stingray more RAD with Jython!

Simply WURFL (08/07/2009)