What to do if you need to access a restricted Stingray Traffic Manager

[#1]

In a large distributed cluster that spans a public network, you may wish to apply fine-grained control over which traffic managers can be used to make configuration updates, and which traffic managers are restricted to operate in a 'read-only' mode with respect to configuration:

The administrator can control which devices can update the configuration in the cluster. 

stingray-1 is grayed out because the administrator is currently using the admin interface on that traffic manager

A restricted traffic manager can still receive configuration updates, and will still broadcast state/statistical data, but effectively becomes unable to replicate configuration updates out to other cluster members.

How do I administer a restricted traffic manager?

Restricted traffic managers receive configuration updates and management commands from your other traffic managers, so no regular administration is necessary. For security purposes, the Administration UI and Control API on restricted traffic managers is disabled. However, there may be times when you need to modify the machine-specific settings (e.g. Networking, Time/Date, SNMP, or EC2-specific settings) that are only accessible through the UI of the traffic manager concerned.

You can  temporarily enable that traffic manager's control!canupdate setting in order to access its Administration UI. This can only be achieved through one of your unrestricted traffic managers, on the System > Security administration page.

How to avoid your entire cluster becoming uncontactable

Should you run into a situation where your unrestricted cluster members become uncontactable for some reason, you may not be able to administer your cluster. You cannot to add new traffic managers in order to regain control of the cluster. In order to mitigate this risk, it is strongly advised that you maintain at least one redundant unrestricted traffic manager in your cluster at all times.

It's too late for that - what can I do?

Option 1: Manually promote a traffic manager

You can re-enable control over your restricted cluster members by making one of them temporarily unrestricted. This will give you access to the Admin UI of that traffic manager, or alternatively allow a new master traffic manager to be deployed and joined to the cluster.

Due to the secure nature of inter-cluster communications, your other cluster members will not immediately recognise this config change as authentic. Instead, you will need to update each cluster member's stored config individually with details of the first traffic manager's change of status.

The following method describes this procedure.  It assumes you have SSH access to each of the traffic managers in your cluster:

• Nominate one of the restricted traffic manager machines as a temporary master. SSH to it as a suitable super-user;
• Edit the ZEUSHOME/zxtm/global.cfg file, and remove the control!canupdate config line;
• On each of your other working cluster members, SSH to them and locate the ZEUSHOME/zxtm/conf/zxtms file for the traffic manager you plan to promote
• Edit this file and remove the control!canupdate config line;
• Your temporary master traffic manager should now be accessible through its Admin UI and it should be able to make configuration changes.

You should remove any failed traffic managers from your cluster.

If you then want to add a new cluster member and give it update permissions:

• On the System > Security page of the temporary master Admin UI, ensure control!canupdate!default is set to Yes. This will mean that new cluster members are added with unrestricted status by default;
• Set up the new traffic manager instance as normal;
• Join this new traffic manager to the cluster;
• Optionally: set the original traffic manager back to restricted by disabling its control!canupdate setting from the UI of the new traffic manager

You can then use the new traffic manager to manage your cluster.

If there is any doubt as to the integrity of existing traffic managers, you should re-install them from scratch before joining them to any cluster where they may have unrestricted status.

Option 2: Add a new traffic manager to the cluster

If you cannot trust the integrity of any of the remote traffic managers, you will need to create a new cluster and import the configuration from the old cluster to the new.

• SSH on to a restricted traffic manager machines as a suitable super-user;
• Take a config backup using zconf;
• Install a new Stingray Traffic Manager instance to be used as the new master, and import the config backup to it via zconf;
• On the System > Security page, ensure control!canupdate!default is set to No. This will mean that joining cluster members are added with restricted status by default;
• Reset each remote traffic manager to factory defaults. This will completely wipe your traffic manager configuration, so you may wish to create a cautionary backup first via zconf:
  • For Virtual Appliances, run z-reset-to-factory-defaults from the command line, and then re-run the the Initial Configuration Wizard.
  • For software installations, use the ZEUSHOME/zxtm/configure script;
• Join each clean traffic manager to the new master traffic manager cluster. Repeat until all traffic managers are present.

This article was originally written by Tim Stace