Hello everyone, GSLB is one of these things that is a bit involved to configure, which is why it was in front of our mind once we've released Terraform Provider for vTM. The template is now ready for its first trial flight, and you're very welcome to give it a go and let us know what you think:   https://github.com/dkalintsev/Terraform-vTM-GSLB The link above should automatically display the contents of the README file that covers why's, what's, and how's. Please sound off in comments if you get lost, or if it's missing something obvious. Note: by default, health check is configured for Pulse PCS (Full HTTPS, reading path "/dana-na/auth/url_default/welcome.cgi" on port TCP/443 on all nodes at all locations). This fairly easy to change - see if you can figure out how! :) In brief summary, you'll need: -    One, but better two vTM clusters for your Primary and Secondary -    Some traffic IPs that these vTMs can use for TIP Groups -    Info for two locations:     o    Endpoint IP address(es) at each for our global endpoint     o    Geo coordinates README very briefly covers how to set up Terraform and the provider (it's as simple as literally downloading and copying two executable binary files) in the "Example use" section, but if this is your first rub with Terraform, you may want to start here: https://community.pulsesecure.net/t5/Pulse-Secure-vADC/Let-s-Terraform-the-vTM-Part-1-4/ta-p/38617 After you've applied your template to your vTM(s), you can edit the template or change input parameters (e.g., change, add, or remove IPs associated with a particular Location) and then simply re-run "terraform apply". Terraform will calculate the necessary changes and update it all for you, as needed - Locations, Monitors, and contents of the DNS zone, without the need to tear anything down or clicking through any dialogs. Hope you find this useful!   P.S. Template's README file has a Disclaimer that hopefully clarifies its purpose and support status, but it's probably worth repeating - as it stands, this is not an artefact that carries Pulse TAC support. For any issues please leave a comment here, and we'll address as soon as we can. Thanks :) ... View more

Continuing from Part 1, Part 2, and Part 3, in this final instalment we’ll finish our configuration by adding things like SSL offload and L7 routing.   vTM has an embedded language called TrafficScript, that can be used to run business logic over Requests (what vTM sees from the client) and/or Responses (what vTM sees from the pool node). It has full access to everything in Requests and Responses (if vTM is configured to perform SSL Offload), which includes headers, cookies, request strings, body, etc.. Based on the logic you put into your TrafficScript (TS), vTM can do all kinds of things – check out my older post on integrating 3rd party services with your Single Page App web site, for example. For this post, we’ll use it to do L7 routing. Terraform supports including external code in-line, which is perfect for small chunks of such code, but I like separating concerns. Create a directory files under our working directory try-vtmtf, and inside it create a file called vs1_request_rule.tpl with the following contents: if(http.getPath() == "/api") { pool.use("${pool_name}"); } This is a template for TS code with one variable – pool_name. Running terraform apply will inject a name of our API pool to produce the actual TrafficScript code. Our VS1 will then run it on all incoming requests, and if HTTP Path contains the string /api, vTM will send that request to the nodes in the pool with the name of our API pool that we’ve passed to it. Let’s add this into our main.tf: # Our Request Rule needs API pool name, so we handle this through a template data "template_file" "vs1_request_rule" { template = "${file("${path.module}/files/vs1_request_rule.tpl")}" vars { pool_name = "${vtm_pool.api_pool.name}" } } resource "vtm_rule" "vs1_l7_routes" { name = "${local.uniq_id}_VS1-L7-Routes" content = "${data.template_file.vs1_request_rule.rendered}" } This will create a template data source called vs1_request_rule of type template_file with one variable – pool_name that we set to the name of our API pool. When accessed, this data source will take the contents of the template file set in template parameter, inject the variable(s), and return the resulting file. We can see how this is used in the vtm_rule resource, where content parameter consumes what this data source produces. Since template_file requires yet another provider, we’ll need to run terraform init once more before proceeding, so that Terraform can download it. Once done, you can run terraform plan and see the rendered contents of the template in the proposed creation of the vtm_rule. Note: In cases where your rules are small (such as above), you may prefer to include them directly into the template using heredoc syntax. It is certainly more compact, and comes to personal preference as to whether you want to mix Terraform templating code (HCL) with other code (TrafficScript) or not. Here’s an example for how you could do the same as above (separate TrafficScript file and data "template_file" "vs1_request_rule") using this combined approach, where the TS code is included between the EOF markers. In this case we inject the pool name by referring directly to the appropriate Terraform resource – ${vtm_pool.api_pool.name}. resource "vtm_rule" "vs1_l7_routes" { name = "${local.uniq_id}_VS1-L7-Routes" # Content is TrafficScript, with an embedded Terraform variable # content = <<EOF if(http.getPath() == "/api") { pool.use("${vtm_pool.api_pool.name}"); } EOF } To make this rule active we’ll need to add it to our VS as a parameter; but let’s take care of SSL offload prerequisites first. Our template requires a certificate, so we are going to create a self-signed one. In our try-vtmtf container, run the following:  openssl req -newkey rsa:2048 -nodes \ -keyout domain.key -out domain.csr \ -subj "/C=US/ST=New York/L=Brooklyn/O=Example Brooklyn Company/CN=*.corp.com" openssl x509 -signkey domain.key \ -in domain.csr -req \ -days 365 -out domain.crt This should have produced three files – domain.crt, domain.csr, and domain.key. Paste their contents (starting / ending with -----BEGIN.. / -----END..) into the three new variables in our terraform.tfvars file between the respective EOFmarkers:  ssl_cert_pub = <<EOF -----BEGIN CERTIFICATE----- MIIDUjCCAjoCCQDGcM4C0OfWCDANBgkqhkiG9w0BAQsFADBrMQswCQYDVQQGEwJV [ ... skipped ... ] 64+OgfhZ2bb3eiFsm2Azf55AWqbb5pr3tDwrBidg5R176Cl6l00= -----END CERTIFICATE----- EOF ssl_cert_pri = <<EOF -----BEGIN PRIVATE KEY----- MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCq0CEK/ua37JDM [ ... skipped ... ] htQ3q8eLTYwApsXNHdGwQyME -----END PRIVATE KEY----- EOF ssl_cert_req = <<EOF -----BEGIN CERTIFICATE REQUEST----- MIICsDCCAZgCAQAwazELMAkGA1UEBhMCVVMxETAPBgNVBAgMCE5ldyBZb3JrMREw [ ... skipped ... ] K4nc/E5mq4JkmBEdiDg/yxUnUvI= -----END CERTIFICATE REQUEST----- EOF Hopefully now it’s double-clear why we don’t want to check terraform.tfvarsinto code repository!  Let’s add the corresponding variables into our variables.tf:  variable "ssl_cert_pri" { description = "Private SSL Cert for the Virtual Server" } variable "ssl_cert_pub" { description = "Public SSL Cert for the Virtual Server" } variable "ssl_cert_req" { description = "Signing Request for the SSL Cert for the Virtual Server" } And the code for the SSL Server Key resource to the main.tf:  # SSL Server Certificates for the Virtual Server's SSL Offload. # resource "vtm_ssl_server_key" "ssl_cert" { name = "${local.uniq_id}-server-corp.com" note = "SSL Server Cert for corp.com" private = "${var.ssl_cert_pri}" public = "${var.ssl_cert_pub}" request = "${var.ssl_cert_req}" } Finally, let’s update our Virtual Server so that it: Uses our TrafficScript Rule; Has SSL Offload turned on; and Listens on the port 443 instead of 80. Replace the resource "vtm_virtual_server" "vs1" { .. } that you have in your main.tf with the following: # The Virtual Server # resource "vtm_virtual_server" "vs1" { name = "${local.uniq_id}_VS1" enabled = "true" listen_on_any = "${local.should_listen_on_any}" # We need to use compact() to get rid of values "" which will be there # in case we didn't have any TIP Groups listen_on_traffic_ips = ["${compact(local.tigs_list)}"] # Default pool = "Main" pool = "${vtm_pool.main_pool.name}" port = "443" protocol = "http" ssl_decrypt = "true" ssl_server_cert_default = "${vtm_ssl_server_key.ssl_cert.name}" request_rules = ["${vtm_rule.vs1_l7_routes.name}"] } The notable changes are: port: 80 -> 443 Added ssl_decrypt, ssl_server_cert_default, and request_rules Now, run terraform apply, and if all went well – you should have your configuration in its final state, all wrapped up and working! If not – do not despair: my copy of the template is available in my GitHub repo, where you can download it and compare with yours. And once you’re done playing – run terraform destroy, to see the configuration you’ve applied to your vTM disappear as a whole, leaving the rest of the configuration (if any) intact. Happy terraforming!   This article is part of a series, beginning with Infrastructure as Code - Terraform with Pulse vADC These articles are republished with permission of @dkalintsev, originally published here: https://telecomoccasionally.wordpress.com/2018/05/17/lets-terraform-the-vtm-part-1-4/ ... View more

Carrying on from Part 1 and Part 2, in this instalment we’ll continue adding to our configuration. If you’re back with us after a break – feel free to go over the part 1 and 2 again, and if you’re following along – make sure your set-up is all good, and the very last exercise from Part 2 completes correctly. In the last exercise we’ve added one pool that can be populated with any number of nodes by supplying IP: Port for those nodes through a variable. Since the second pool isn’t any different, simply add the following to your main.tf, variables.tf, and terraform.tfvars, and re-run terraform apply to see if it works: # main.tf # data "vtm_pool_nodes_table_table" "api_pool_nodes" { count = "${length(var.api_nodes)}" node = "${var.api_nodes[count.index]}" priority = "1" state = "active" weight = "1" } resource "vtm_pool" "api_pool" { name = "${local.uniq_id}_API-Pool" monitors = ["Ping"] nodes_table_json = "[${join(",", data.vtm_pool_nodes_table_table.api_pool_nodes.*.json)}]" } # variables.tf # variable "api_nodes" { description = "List of nodes for the 'API' pool" default = [] } # terraform.tfvars api_nodes = ["192.168.1.100:80", "192.168.2.100:80"] If all went well, your vTM will now have two pools with two nodes in each. So, all these resources.. You probably been wondering about where you can find all the possible choices you have for vTM resources in your templates. Those familiar with vTM’s REST API have likely noticed that the resource names look suspiciously like vTM REST resources with vtm_ slapped in front, with resource parameters very much a 1:1 transplant. And you will be right – the resources (and data sources) that vTM Provider exposes closely match those in the REST API. There are a few minor changes where internal Terraform variables clashed with parameter names in vTMs REST schema and have been slightly changed to aviod it. In addition, however, vTM Provider exposes a number of special data sources: *_list and *_table. You’ll meet them in this blog post very soon. The complete set of resources, data sources, and useful examples can be found in the vTM Terraform Provider documentation (REST API v4.0 / vTM 17.2 and later) or (REST API v5.2 / vTM 18.1 and later).   Traffic IP Groups and conditions You may wish to have a template that works in different environments; e.g., when your vTM cluster is running in AWS, you may be using Elastic IP (EIP) based Traffic IP Group (TIG), or when you’re running in a Docker container for a dev/test task, you may not want to create a TIG at all. Let’s add some logic to our template as an example for how you could handle this. First, let’s create a variable vtm_tig_eips in variables.tf: variable "vtm_tig_eips" { description = "List of AWS Elastic IPs to be used for Traffic IP Group" default = [] } As you see, by default we set this variable to be an empty list, which suits our playpen environment. Now let’s add the code to out main.tf that will create a Traffic IP Group in case our list had actual AWS EIPs in it: # This returns a list populated with "name" values of all traffic managers # in the target cluster. We need this to create the Traffic IP Group. # data "vtm_traffic_manager_list" "cluster_machines" { # No parameters needed } locals { tig_name = "${local.uniq_id}_TrafficIPGroup" } # Traffic IP Group for our Virtual Server. # By default, we create the "singlehosted" type. # resource "vtm_traffic_ip_group" "tip_group" { count = "${signum(length(var.vtm_tig_eips))}" name = "${local.tig_name}" mode = "ec2vpcelastic" ipaddresses = "${var.vtm_tig_eips}" machines = ["${data.vtm_traffic_manager_list.cluster_machines.object_list}"] }  The above introduces a couple points worth noticing: First, we use Data Source vtm_traffic_manager_list to get the list of vTMs in a cluster we’re working on. We need to know this as it’s a required parameter machines for the vtm_traffic_ip_group resource. Since this lookup is dynamic, this code will work on vTM clusters of any size, including a single-node. This is an example of one of the special resource types (*_list) provided by the vTM Terraform Provider. Second, we use count as a condition to create the resource. The length(var.vtm_tig_eips) will return a number of IPs supplied in the variable, or 0. We’re good for when it returns a 0, which will cause this resource to not be created; but we don’t want more than one Traffic IP Group in case our variable had multiple IPs in it! This is where Terraform’s built-in function signum() comes handy – it will keep 0 a 0, and convert any positive number into 1, which is what we want. Since we have not included any IPs for our vtm_tig_ips, running terraform plan should say that no changes are required. Last but not least, we defined a name for our TIP Group as a Local Variable. The need for this will become clear in a few moments. Let’s test if our logic works. The configuration may not be valid, but at least we’ll know it was applied. Add the following to our terraform.tfvars, and run terraform apply:  vtm_tig_eips = ["2.2.2.2", "5.5.5.5"] Your vTM’s log should display a Warning “VPC Elastic IP addresses can only be configured on EC2-VPC instances”, and Services -> Traffic IP Groups should have a broken TIP Group with our two IP addresses “2.2.2.2” and “5.5.5.5”. Let’s undo this for now. Edit the terraform.tfvars, put a # in front of the vtm_tig_eips line, and then re-run terraform apply. The TIP Group should disappear from your vTM. Ok, let’s add our Virtual Server (VS) before winding up for today. To start with, let’s add a simple HTTP VS. We’ll add SSL Offload and L7 routing to it in our next part. Edit your main.tf and add the following block:  locals { # If var.vtm_tig_eips is empty, we should not use TIP Group should_listen_on_any = "${length(var.vtm_tig_eips) == 0 ? true : false}" # This is to provide the list of TIP Group Names to listen_on_traffic_ips # paremeter of the Virtual Server. If We don't have any Traffic IPs, this # list should be empty; if we do - it should have a name of our TIP Group. tigs_list = ["${length(var.vtm_tig_eips) == 0 ? "" : local.tig_name}"] } # The Virtual Server # resource "vtm_virtual_server" "vs1" { name = "${local.uniq_id}_VS1" enabled = "true" listen_on_any = "${local.should_listen_on_any}" # We need to use compact() to get rid of values "" which will be there # in case we didn't have any TIP Groups listen_on_traffic_ips = ["${compact(local.tigs_list)}"] # Default pool = "Main" pool = "${vtm_pool.main_pool.name}" port = "80" protocol = "http" } Since we’re working around two cases here – we either do or don’t have Traffic IP Group – we need to make sure our Virtual Server has correct values for the related parameters: listen_on_any and listen_on_traffic_ips. The code above first defines a couple Local Variables that are set based on condition, and then uses them as values for the respective parameters of the VS1. listen_on_traffic_ips deserves a special mention. For it to not take effect when we don’t use TIPs, the value it receives must be an empty list – []. However the best we can do with the tigs_list variable is to set it to a list that has one empty element, [""], which isn’t good enough. Luckily for us, compact() can get rid of the empty element, and we’re off to races! Now, why did we define a local variable to hold a name for our TIP Group? This is to handle the case where the vtm_tig_ips is empty, and the resource "vtm_traffic_ip_group" "tip_group" is not created. Usually we would refer to the .name parameter of our resource to fetch what we need; but what do we do if the resource creation is conditional? Unfortunately at the moment Terraform will always evaluate both sides of a condition. This means we can’t just create a logic that says “if there are Traffic IPs, create TIP Group and get the name from it; otherwise don’t create a group and set the name to []”. If we did, Terraform would always try to evaluate both, and fail in the case where we don’t have any TIPs specified. So the work-around for that in our specific case is to create a name for our TIP Group as a local variable that is not dependent on whether actual TIP Group resource exists or not. Let’s run terraform apply once more to marvel at our creation, and wrap up with that for now. To sum it up, in this part we talked about: Handling conditional logic in our templates, How to use that logic to affect creation of resources (through count), and How to use it to manipulate parameters for resources that depend on such conditions. See you in the final Part 4! This article is part of a series, beginning with Infrastructure as Code - Terraform with Pulse vADC These articles are republished with permission of @dkalintsev, originally published here: https://telecomoccasionally.wordpress.com/2018/05/17/lets-terraform-the-vtm-part-1-4/ ... View more

  Continuing from Part 1, today we’ll make our template do something useful. If you’re following along – make sure your set-up is all good, and the very last exercise of Part 1 completes correctly. Ready? Let’s carry on!   To remind you what we’re templating, here’s our sample app’s diagram again:     As far as vTM goes, we have four elements we’ll need to configure: 2 x Pools (Main and API) Traffic IP Group Virtual Server Note that our playpen setup is missing a few bits and pieces, to say the least. We have a single vTM, no ability to raise Traffic IPs, and no actual backend servers. However, the thing is that the template we’ll create will work just fine if we point it at a different vTM (cluster) in a real environment, and give it the right IPs as the input parameters! But first things first Remember how in the last exercise we had to type the input values of the variables by hand? This will get old very quickly, so let’s take care of this. When you run Terraform, it will look in the current directory for the file named terraform.tfvars, that can provide values for the variables defined within the template. Following the reasoning in Part 1, I personally like to keep values in this file that are specific to the copy of the template I’m deploying. Let’s make ours. Create a new file called terraform.tfvars in our try-vtmtf directory, and add the following to it: vtm_rest_ip = "172.16.0.10" vtm_password = "abc123" Note that this file often contains sensitive information (as you can see above) such as cloud and login credentials, so it’s an extremely good idea to add terraform.tfvars into your ~/.gitignore_global. The topic of handling sensitive info with terraform is a lot bigger than this blog allows, so keep in mind this file, along with the state files that terraform will create once you run the apply command. Wait a sec.. Most resources in vTM have a name field that is used by other resources to refer to them; e.g., a Virtual Server refers to the pool by its name. Since our templates could be in theory applied to the same infrastructure (in this case – vTM) multiple times, we need to make sure that we: Can find all bits that a particular instance of a template has created; and Our resource names (assigned in the template!) don’t clash. Fortunately, terraform supports generation of random strings that are stable within a template instance, but is created anew once instance is destroyed or an entirely new instance of the template is created. Let’s add one. # Random string to make sure each deployment of this template # includes unique string in all resources' names. This should allow # deployment of more than one copy of this template to the same vTM # cluster as long as the unique things like IP addresses used for # Traffic IP Groups are taken care of elsewhere. # resource "random_string" "instance_id" { length = 4 special = false upper = false } locals { # Create a local var with the value of the random instance_id uniq_id = "${random_string.instance_id.result}" } The code above creates a local variable uniq_id, that is a String that is a random mix of 4 lower-case letters and numbers. We’ll use it when we get to name our resources. Let’s add our first pool We have two pools, each of which serves different parts of our faux web site. In our diagram we have two nodes in each pool. Let’s start with the Main pool. Edit out main.tf, and add the following to it: resource "vtm_pool" "main_pool" { name = "${local.uniq_id}_Main-Pool" monitors = ["Ping"] nodes_table { node = "10.1.0.10:80" } nodes_table { node = "10.2.0.10:80" } }   Hmm, hold on. Looks like we’ll need to create a separate nodes_table section for each node we want in our pool. But how can we template the configurations where we don’t know the number of pool nodes ahead of time, e.g., when a list of these nodes is passed in as a parameter? It looks like Terraform doesn’t handle this very well at this point; so what do we do? Fortunately, vTM Provider offers an ability to describe resource parameters that take input as tables, such as nodes_table above, through special data sources. It can then take output of these data sources as the parameter value. Don’t worry, it will come clearer in a moment. Data sources support use of count (which we’ll talk about below soon), so we’re in luck! Let’s see what this looks like. Replace the resource "vtm_pool" "main_pool" { .. } that you’ve just added (per above) with the following:  data "vtm_pool_nodes_table_table" "main_pool_nodes" { # Repeat as many times as we have nodes in our node list variable count = "${length(var.main_nodes)}" # Get the node from the var.main_nodes list node = "${var.main_nodes[count.index]}" } resource "vtm_pool" "main_pool" { name = "${local.uniq_id}_Main-Pool" monitors = ["Ping"] # The data.vtm_pool_nodes_table_table.main_pool_nodes.*.json returns a list # of string, each string is a JSON for one node. We need to wrap this into # a JSON list, so we add "[]" at the ends, and join the node strings with "," # in the middle for a resulting "[{..},{..}]" string that nodes_table_json # is expecting # nodes_table_json = "[${join(",", data.vtm_pool_nodes_table_table.main_pool_nodes.*.json)}]" } Ok, there’s a fair bit going on above. First of all, we have a new variable, main_nodes, which we’ll need to add to our variables.tf in a moment. Second, vtm_pool_nodes_table_table is a Data Source, which is a type of special read-only resource. This particular one is designed to create a JSON string populated with values for specified parameters. In our case, it’s a pool node parameter node. You can also see a parameter count, that is set to the length of variable main_nodes. Since this variable is of a type list, length will equal the number of entries in it. In our case, the list will be populated with IP: Port strings for our pool nodes, e.g., "10.1.0.10:80" and "10.2.0.10:80". The way count works is that when it’s present in a resource or data source, terraform will create count many copies of this resource or data source. In our case there will be two, each of which will return a JSON representation of the corresponding node, with value for the node derived from a count-th element within the main_nodes list. Then we get to our pool. The first thing you’ll notice is that value for the nameparameter is made up by adding our uniq_id string to the literal _Main-Pool, so as discussed above we avoid potential clash, and can identify bits of vTM config that belong together. The second bit is that nodes_table_json parameter is formed by combining the outputs of count outputs of the vtm_pool_nodes_table_table into a string that conforms to syntax of a JSON table. It works like this: Each vtm_pool_nodes_table_table output is a {} join() connects them all into one string joined by ,, essentially {},{} Finally, they are combined with literal [ and ] at the start and the end, to a resulting [{},{}] JSON string we want. The cool thing about it is that it will work irrespective of how many items are in the main_nodes list – zero or a hundred, so all we need to do is to generate a main_nodes list that matches our environment, and terraform will do the rest. Whew. Ok, let’s add our new variable main_nodes to our variables.tf and terraform.tfvars, and give it a spin! variables.tf: variable "main_nodes" { description = "List of nodes for the 'Main' pool" # For example, ["1.1.1.1:80", "2.2.2.2:80"] default = [] } terraform.tfvars:  main_nodes = ["10.1.0.10:80", "10.2.0.10:80"]   Note that variable "main_nodes" is supplied with a default value of []. This will cause Terraform to not prompt you for the value of this variable if you have not specified it, for example, through terraform.tfvars as above. I did this because a configuration where a pool has no nodes is technically valid and useful. An example would be a situation where my configuration may evolve and change when I repeatedly apply the same template, but with different input parameters. I may start with the empty list of nodes in one pool but add them later; while the rest of the configuration will be ready in place. Now, let’s see what this can do! In our try-tfvtm Docker container (or your own setup), from inside our try-tfvtm directory, run: terraform init, followed by terraform plan. You need to run the init again because we’ve added a new provider to our template – random, and terraform needs to download a copy of that Provider plugin from the Internet. If all went well, you should see terraform think for a little while, then spit out a bit of output ending with something like: **Plan: 2 to add, 0 to change, 0 to destroy.** Take your time to have a look at the output. This is the representation of what terraform will do, once you run terraform apply. You’ll notice that terraform shows many more parameters for the resource we’ve defined (vtm_pool.main_pool). The ones that we haven’t specified are the defaults. This output is often useful in determining how your resource will be configured in the parts that you didn’t specify; but also for figuring out what parameters your resource has for you to play with. Before wrapping up this part, let’s run the terraform apply, and see what happens. Terraform will again display the plan output and ask you to type “yes” to confirm that you let it make the proposed changes. If you don’t want to be prompted, add -auto-approve to the end of the command, so it reads in full as terraform apply -auto-approve If all went well, you should see: **Apply complete! Resources: 2 added, 0 changed, 0 destroyed.** You can wrap up with that, or go to https://172.16.0.10:9090 (change the 172.16.0.10 to your vTM’s IP), and visit Services -> Pools, where you should see a new pool named [4 random symbols]_Main-Pool with 2 nodes in it – 10.1.0.10 and 10.2.0.10, both with port 80. To sum it up, in this part we learned: How to save values for Terraform variable parameters in terraform.tfvars file, How to name vTM resources to avoid conflicts between multiple instances of a template applied to the same vTM, What are the Data Sources and Resources, and How to create a vTM Pool with a variable number of nodes in it. See you in Part 3! This article is part of a series, beginning with Infrastructure as Code - Terraform with Pulse vADC These articles are republished with permission of @dkalintsev, originally published here: https://telecomoccasionally.wordpress.com/2018/05/17/lets-terraform-the-vtm-part-1-4/ ... View more

The freshly-released Pulse Virtual Traffic Manager (vTM) v18.1 comes out with a Terraform Provider for vTM. The provider ships with 100% coverage for all vTM’s REST API resources, and includes support for the API version 4.0 that goes back to vTM 17.2, and the API version 5.2 that includes all the newest features that shipped with vTM 18.1. In this 4-part post we’ll do a quick introduction of Terraform provider for vTM, and show how it can help you support the needs of your applications.   Terraform  Hopefully Terraform does not need much introduction. It is one of the most prominent Infrastructure as Code tools, that can provision a wide variety of IaaS, PaaS, SaaS, and on-prem services through the use of Providers. The role of a Provider is to take the details needed to create / read / update / delete resources specific to a particular Service (e.g., AWS), and make them available through the Terraform’s language (HCL). The beauty of this approach is in that you can describe a collection of resources supported by any combination of the Providers using the same language (HCL). And you can mix and cross-reference these resources in one template, too. It’s worth noting that Pulse were clearly not the first ones to realise that this approach would fit very well with treating vTM as an infrastructure that provides load balancing “services” to applications sitting behind it. At this point I’m personally aware of two independent implementations of Terraform provider for vTM, created by customers themselves: One and Two. So without further ado, let’s dive into it. An example topology For this blog I wanted a very simple topology that’s fairly typical: a single domain (corp.com) with a single property (www), and two backends – main (/) and API (/api). A cluster of 2 x vTMs sitting in front of the infrastructure that serves this provides a few basic functions: Terminates/offloads SSL Performs L7 routing based on the request Path Provides High Availability for the IPs that match the property’s FQDN by using Traffic IPs.   If you’d like to follow at home I’ve created a Docker image (based on Apline Linux) with terraform itself + both versions of terraform provider for vTM installed in /usr/local/bin. You can run a copy of it so that you can test things out as we go:   mkdir try-vtmtf && cd try-vtmtf docker run -it --rm -v $PWD:/root/try-vtmtf -w /root/try-vtmtf dk114/try-vtmtf:1.1 /bin/ash terraform --version # You should see "Terraform v0.11.7" The docker run command included above will start a Docker container, and mount your current directory (try-vtmtf) under the path /root/try-vtmtf. Any changes you make to files that reside in this directory will persist on the your main host running container. Also any changes you make to files in this directory from outside the container will be immediately visible to programs running inside (such as terraform). This means you can edit the files we’ll be working on using an editor on your host, and terraform inside the container will “see” your changes as soon as you save them to disk. If you prefer to create your own setup, download the following: Terraform itself Terraform provider for the vTM REST API or Unofficial build of Terraform provider for vTM – darwin64 The MacOS/darwin provider will come down named as terraform-provider-vtm_v4.0.0-darwin. You’ll need to rename it and remove the -darwin part, so it’s called just terraform-provider-vtm_v4.0.0, then make it executable (it’s a static binary), and place somewhere in your $PATH. Note that the official instructions for installing plugins differs from this approach. I found that both work. Placing plugin somewhere in $PATH however has benefit of making it available to users system-wide without asking them to make their own copy. You can continue working in another Terminal window on our template in the directory you’ve created (try-vtmtf). If you already have a copy of vTM you can play with – note down its management IP address, make sure REST API access is enabled (typically on port 9070), and that your machine can reach it. If not – you can get one like so: docker run --name vTM --rm -e ZEUS_DEVMODE=yes -e ZEUS_EULA=accept -e ZEUS_PASS=abc123 -p 9090:9090 -p 9070:9070 -p 80:80 -p 443:443 --privileged -t -d dk114/pulse-vtm:17.4 This should download a Docker image with a vTM 17.4, and run it in the Developer Mode (all features on, throughput-limited to 1Mbps). It sets password for the admin user to abc123; change as you see fit. You should then be able to reach it from your terraform container on the IP of your machine (172.16.0.10 in my case): curl -u admin:abc123 -k https://172.16.0.10:9070/ # I get the response: # {"children":[{"name":"api","href":"/api/"}]} If you got to this point successfully – sweet! Let’s begin Terraform works on templates; but with a little twist – when you run terraform <operation>, it will look inside your current directory for all files named *.tf, combine them behind the scenes, and treat them as if it was a single file. So each template will need its own separate directory; in our case the one we’ve created and changed into – try-tfvtm. Most templates have one or more variables that can be set with default values, or given those values at the run time. Terraform then takes those values, combines them with the template code, and calculates the final desired state, which is what it will work to achieve when you run terraform apply. Also templates are often shared between people, and to help quickly figure out what are the variables used in a template, it’s often useful to split variable definitions into a separate file. Let’s do that, and create two files: main.tf and variables.tf: # Inside our try-vtmtf directory: touch main.tf variables.tf Edit the main.tf, and add the following block to it: provider "vtm" { base_url = "https://${var.vtm_rest_ip}:${var.vtm_rest_port}/api" username = "${var.vtm_username}" password = "${var.vtm_password}" verify_ssl_cert = false version = "~> 4.0.0" } This block of code will tell terraform that we have a vTM (or maybe a vTM cluster) that can be reached on https://${var.vtm_rest_ip}:${var.vtm_rest_port}, authenticated to using ${var.vtm_username} and ${var.vtm_password}; that it’s using a self-signed cert (that we should none the less trust for now), and that we’ll talk to this vTM using REST API v4.0. At the time of writing this includes vTM versions 17.2x, 17.3, 17.4, and 18.1. All these ${} bits are variables, and we’ll need to define them. Let’s do that. Save your main.tf for now, edit variables.tf, and add the following to it: variable "vtm_rest_ip" { description = "IP or FQDN of the vTM REST API endpoint, e.g. '192.168.0.1'" } variable "vtm_rest_port" { description = "TCP port of the vTM REST API endpoint" default = "9070" } variable "vtm_username" { description = "Username to use for connecting to the vTM" default = "admin" } variable "vtm_password" { description = "Password of the $vtm_username account on the vTM" } As you can see, two of our variables – vtm_rest_port and vtm_username have defaults. When thinking about when to set a default, I personally think “is this value going to be applicable in all/most cases where I’ll be using this template?” If yes, then I set the default. If not – we’ll be passing value for it during run time.   Ok, it’s time to give it a go, even though our template doesn’t do anything useful yet! Save both our files, and from inside your try-vtmtf container check that you’re in /root/try-vtmtf directory and can see your main.tf and variables.tf files in it. If we’re good, do this: ls -l # total 8 # -rw-r--r-- 1 root root 234 Apr 26 03:51 main.tf # -rw-r--r-- 1 root root 423 Apr 26 03:51 variables.tf terraform init # You should see messages saying: # "Initializing provider plugins... # Terraform has been successfully initialized!" terraform plan # Terraform should prompt you for the values of # var.vtm_password and # var.vtm_rest_ip Give it the values, and if all is good and your try-vtmtf container can reach your vTM – you should see a message informing you in happy green letters: “No changes. Infrastructure is up-to-date.” To sum it up, in this part we learned: What is a Terraform template, How to declare a connection to a vTM (cluster), and What the variables look like in Terraform and what they’re useful for. At this point, pour yourself some beverage to celebrate getting your set-up into the shape for the Part 2, where we’ll actually create some useful configuration.     This article is part of a series, beginning with Infrastructure as Code - Terraform with Pulse vADC These articles are republished with permission of @dkalintsev, originally published here: https://telecomoccasionally.wordpress.com/2018/05/17/lets-terraform-the-vtm-part-1-4/     ... View more

Pulse Virtual Traffic Manager v18.1 has introduced plugin-based Service Discovery that shipped bundled with two plugins: for Google Cloud, and for DNS.   The included DNS plugin was designed to help with a specific use case, where an authoritative DNS server returns a *subset* of the records every time it is queried, instead of the full set of records.   An example of this is AWS Route53 serving A-records for a large Elastic Load Balancer (ELB). Route53 will return up to 8 healthy records. This means that for ELBs with more than 8 nodes DNS query will only ever return 8, which a non-authoritative DNS server will cache and return for all subsequent queries.   If a regular DNS resolver is used to populate vTM pool nodes, this Route53 behaviour may lead to excessive vTM pool node churn. Additionally, traffic will only be ever sent to a maximum of 8 nodes.   To work around this issue, the bundled DNS plugin implements the following behaviour:   - For the hostname specified, find the authoritative DNS server(s) - Send a query for hostname's A-records directly to the discovered authoritative DNS servers - Cache the received results, along with each record's TTL - Check the cache for any existing records with TTL that hasn't expired - Combine the new records with the cached records, and return that superset as the result for vTM to use.   This behaviour has a side effect for some publicly registered domains with name servers that can't resolve the records within the domain, such as internally used domains. In this case, DNS resolver plugin will fail to work because in the process of discovering authoritative DNS servers upper level domain servers will respond with nameservers that can't resolve the names in question.   To work around this situation, DNS plugin can be run with an option "--nameservers" with IP address(es) of the internal authoritative DNS servers for the internal domain. This will bypass the logic for authoritative nameserver discovery. ... View more