HowTo: NetScaler AAA with Multi-Domain and Duo Integration

Products:

• NetScaler 11.1 b57.11
• Duo Security MFA

Background:
Sharing some lessons learned from a customer environment we’d worked in wherein the team previously migrated the F5 appliances (18 of them) to NetScaler, which included a selection of multi-domain authentication websites fronted by F5 APM which were moved to NetScaler AAA. Their new security mandate required protecting various websites behind the NetScaler AAA vServer with Duo, and this wasn’t a particularly straightforward process. The main goal here is to implement Duo alongside a multi-domain drop-down configuration using only basic authentication policies and no nFactor (as we didn’t have much luck getting Duo working with nFactor). To minimize disruption to the live sites, a new AAA vServer was created (along with public IP, NAT, and DNS entry) to allow for granular transition of websites between current and new auth server.

This post will address a number of key challenges with AAA; adding a domain drop-down without the need to use complex nFactor (which provides multi-domain drop-downs via login schemas) and advanced authentication configs, and integrating Duo MFA with NetScaler AAA. Duo publishes documentation on integrating Duo with NetScaler but its relegated to NetScaler Gateway only. And even then, have found their instructions to be a little more cumbersome than they need to be (updating the gateway_login_form_view.js file can be handled either via rewrite or direct modifications to the file and making it persist across reboots with rc.netscaler entries).

In addition to this getting a domain drop-down into an AAA page can be a bit onerous. A colleague within Citrix had previously implemented this for the customer for single-factor authentication in order to accommodate for authentication against multiple LDAP servers via advanced authentication configurations and login schemas, but this did not extend well to Duo with the “next factor” settings as the Duo UI post LDAP authentication was having trouble displaying itself as per below (note, this is 11.1 firmware, old lock icon was used as part of an earlier custom theme).

Beyond that, getting the multi-domain drop-down to render and function correctly took a little while as well, and a little bit of reverse engineering. At least in the 11.1 build we were working with, the provisions in the tmindex_view.js file was pretty half-baked and non-functional. Once we were able to identify the necessary value to use for the drop-down menu (it’s ‘domain’ in the TM config, wherein with Gateway it’s ‘userDomains’) and get that drop-down finally showing up, it was completely blank thus useless.

After some cross referencing between the tmindex_view.js and gateway_login_form_view.js implementation of the domain drop-down configs, we’d zeroed in on an offending line that was preventing the domain text from properly displaying.

Note: This post assumes the reader is familiar with implementing domain drop-downs on NetScaler Gateway in firmware 11.0 and above, creating a basic NetScaler AAA vServer, as well as configuring expression policies to activate based on cookie values in the domain drop-down.

Pre-Requisites:

• It is also assumed that the reader has already created the initial AAA vServer, bound a cert to it, and has session policies created for SSO to their web apps
• Reader has created the basic LDAP policies and servers for the various domains to activate based on domain cookie
• Reader has created the Duo RADIUS policy and server (assuming here there’s only one of perhaps two for cascading failover, but as Duo auth servers should have a 90 second timeout or more to accommodate for enrolment, that can be a long delay in event of issues.) Recommend load balancing multiple Duo servers using Carl’s guide as a reference or Citrix documentation.

Step 1: tmindex_view.js modifications for domains:
With respect to this AAA implementation and Duo, the tmindex_view.js takes place of the gateway_login_form_view.js file.

First task, fire up WinSCP or other SCP client update the tmindex_view.js file on your active and passive NetScaler nodes. You’ll find this in the /netscaler/ns_gui/vpn/js directory. Back it up of course. Update the bolded text in the file with the bolded lines below:

//domain drop down markup
var domain = ns_getcookie("userDomains");

function create_drop_down(domain)
{
var field_domain = $("<div></div>").addClass('field clearfix');
var left_domain = $("<div></div>").addClass('left');
var domain_span = $("<span></span>").addClass('plain input_labels form_text').attr("id","domain");
var right_domain = $("<div></div>").addClass('right');
var select = $('<select name="domainvalue"></select>').attr({"size":"1","class":"domain_select"});
var domains = domain.split(',');
for (j=0; j < domains.length ;j++)
{
var option = $("<option></option>").attr({"value":domains[j]}).text(domains[j]);
select.append(option);
}
right_domain.append(select);
field_domain.append(left_domain.append(domain_span),right_domain);
return field_domain;
}

The first line changes the domain cookie variable from ‘domain’ to ‘userDomains’. The reason we opted to do this was we found a reference to cookie cleanup line items in the js files for ‘userDomains’ and not for ‘domain’. Tried both however and it worked fine so may not be critical to change this, just bear in mind what you use for your domain drop-down rewrite config later. The result should look as so:

Step 2: Modifications for Duo integration:

While we’re in here, let’s go ahead and inject the Duo code needed to suppress the “Password 2” field on the logon page. This is more or less the same location to edit as found on the Duo’s website for the gateway_login_form_view.js file.

Step 3: Ensure file updates persist across reboots:

To ensure the js file updates persist across reboots of the appliances, perform the following tasks on both active and passive nodes, we’re following the same logic as found on CTX126206 as it still functions on 11.011.1 at least:

• If not already created, create a folder called customizations in /var/customizations
• Copy your updated tmindex_view.js file to this location, ensure permissions remain 755 or 555 on the file
• Add the following line to rc.netscaler (/flash/nsconfig) to ensure the file is copied back and overwrites the default on reboot
  • cp /var/customizations/tmindex_view.js /netscaler/ns_gui/vpn/js/tmindex_view.js

Again, ensure the file (in this case rc.netscaler) is also updated on your passive node.

Step 4: Classic domain drop-down for AAA:

NetScaler has not historically allowed for direct binding of rewrite policies to an AAA vServer, which has forced the use of rewrites to be bound globally for injecting common logon page items such as footer text, etc. Using expression policies focused on hostname, etc. one can adequately target a particular vServer with a policy, without affecting others.

Now let’s create the necessary rewrite configs and bind it globally. Adjust your host header to match the hostname of your AAA server and the domain names. They’ve been provided as CLI commands for ease of implementation. Adjust policy and action names as necessary. Important that the global binding must be a RESPONSE and not a REQUEST otherwise the drop-down will never appear.

add rewrite action act_insert_domain_dropdown_AAA insert_http_header Set-Cookie ""userDomains=Domain1,Domain2,Domain3;path=/;Secure""
add rewrite policy pol_insert_domain_dropdown_AAA”http.REQ.HOSTNAME.CONTAINS("auth.domain.com")" act_insert_domain_dropdown_AAA
bind rewrite global pol_insert_domain_dropdown_AAA 100 NEXT -type RES_DEFAULT

Domain drop-down should now be in-place on the appliance. If it does not show up, first try to load the page in a private browser window or flush the Integrated Caching groups on the NetScaler (doesn’t require actually enabling IC feature).

Step 5: Bind your policies:

At this stage, if not already done, ensure your basic authentication policies are bound (LDAP for primary auth and should have a policy for each domain with corresponding cookie variable in the policy expression matching the domain drop-down value, and likely a RADIUS policy with an ns_true value for secondary auth).

You’ll want to ensure your 401 or forms-based LBVS is bound at this stage as well and appropriately configured to employ the FQDN of the AAA server. It is assumed the reader is experienced with that task (per pre-requisites list above).

We noted some difficulties implementing the session policies on this AAA server, however. One is understandably required for each domain in order to provide an SSO experience to the website(s) behind the AAA setup but received errors when attempting to bind them in the GUI, citing issues with advanced policies being present (which there were not on this particular vServer anywhere). Furthermore, we were unable to implement even with proper syntax, advanced expression policies for the SSO session policies in the GUI, and were getting “invalid” errors when doing so. We resorted to building our SSO policies via CLI which worked no problem and were successfully bound after the fact. Maybe an issue with our 11.1 build, who knows. Here’s the syntax we used to set a cookie expression on an SSO policy. It’s assumed your SSO session profiles are already created.

add tm sessionPolicy POL_SSO_Domain1 "HTTP.REQ.HEADER("Cookie").CONTAINS("Domain1")" PRO_SSO_Domain1

Step 6: Moment of Truth (testing!):

Once session policies are configured and bound to the AAA server, you should be good to go. Try hitting the URL for your web app you’ve associated with the AAA vServer. Here’s a sample after some branding adjustments:

Special shout out to our rockstar Citrix Systems colleague Howard Weise for his moral support and AAA expertise as this particular solution was developed.

Update September 10, 2019: Looks like Duo finally has a solution for nFactor integration that doesn’t involve a bunch of heavy LoginSchema customization. This will render the config of this blog less relevant with nFactor now the gold standard for ADC authentication. With Citrix deprecating classic policy expressions and planning to remove support in later firmware versions for classic policies, this was a hurdle for many Duo users. Check out the blog post here. Note that ADC 12.1 is needed alongside the 3.1 Duo Auth Proxy build and a proxy config update.