Skip to main content

Issue and Background

Earlier this year, we worked with a customer to develop a Citrix ADC-powered alternative to a next-generation firewall (NGFW) where based on user ID, users would have network authorization rules applied to various backend networks and servers. For that creative solution, you can read up on it here: HowTo: Create a Citrix Gateway SSL VPN with SSO via Kerberos.

Months of testing were quite successful, however, one strange issue kept appearing: on initial connection establishment, there was a delay of up to 60 seconds establishing the VPN connection. This created a poor user experience and needed resolution before entering production.

The key factor that would help us zero in on the root causes was the issue never occurred if the client was connected to a network with Internet access. For the customer use case, this solution was developed for use in a high-security setting thus no Internet connectivity is permitted.

Root Cause and Resolution

Ultimately there were two causes, both of which contributed to the lengthy connection time.

  • CEIP call-outs to Google Analytics
  • CRL checking against the SSL VPN certificate

For both issues, we were able to discern the root causes through examination of the client-side logs for the Citrix Gateway plugin.

Here is an example of failure states for the Google Analytics calls.

2021-05-26 15:02:41.180 | Tid: 10020 | ERROR  | sendGARequestThread | 165 | Failed at HttpSendRequest Error code: 12029

2021-05-26 15:02:46.190 | Tid: 10020 | DEBUG  | Sending v=1&aip=1&tid=UA-59929653-34&cid=4e4e34ea-e640-4db6-972d-3aa1be72ad09&an=WinVPN&av=21.3.1.2&ul=en&t=screenview&cd=init&cd1=Win10_64-bit&cd2=11.1411.18362.0 for Google Analytics request.

2021-05-26 15:03:07.227 | Tid: 10020 | ERROR  | sendGARequestThread | 165 | Failed at HttpSendRequest Error code: 12029

 

For the Google Analytics telemetry, this was surprisingly difficult to zero in on. There are no clearly identifiable means within the client UI to disable the Customer Experience Improvement Program (CEIP) as there is in the Citrix Workspace app. The fix is however documented but is done more as a footnote in this Citrix document here.

By setting the registry key on the client as follows, we disable the call-outs to Google Analytics by the VPN client. Be sure to quit and reload the VPN client after implementation.

“HKLM\Software\Citrix\Secure Access Client\DisableGA” of type REG_DWORD to 1

We will see this evidence before and after in logs when the Citrix Gateway VPN plugin loads:

Before registry change:

2021-06-24 14:15:19.460 | Tid: 19704 | DEBUG | Google Analytics is enabled on this machine

 

After registry change:

2021-06-24 14:24:13.708 | Tid: 23692 | DEBUG | Google Analytics is disabled on this machine

 

For the second issue contributing to connection delay, we had a CRL issue to deal with. The VPN plugin performs a CRL check as part of the connection process. As they were using a certificate from a commercial CA, a similar issue related to no outbound Internet connectivity was encountered. This solution was being addressed through an exception rule for CRL checking against DigiCert to resolve the matter. Alternatively, the customer is looking to correctly configure CRLs on their internal CA infrastructure and avoid outbound Internet calls for CEL checks altogether.

Hopefully, these tips will assist others who might encounter Citrix SSL VPN challenges in very restricted use cases.

  • Michael Shuster
    Michael Shuster

    Michael is Ferroque's founder and a noted Citrix authority, overseeing operations and service delivery while keeping a hand in the technical cookie jar. He is a passionate advocate for end-user infrastructure technology, with a rich history designing and engineering solutions on Citrix, NetScaler, VMware, and Microsoft tech stacks.

Subscribe
Notify of
guest

4 Comments
Inline Feedbacks
View all comments
NDS
NDS
2 years ago

Hi Michael,

I am interested in how the troubleshoot for the CRL topic was done. The VPN Client needs to connect to the CRL defined in the Cert bind to the NetScaler?

NDS
NDS
2 years ago

Hi again, Google Analytics disabled and double checked CRL is reachable with or without the VPN. Our main problem is not the time it takes to log-in but a general slowness while using SmartCard and VPN. We have other authentication methods that works perfect, SAML or Radius are fast. But authenticating with SmartCard to the VPN provokes almost the double average load times in simple webs. We appreciate lot of activity in the SC Led reader, Citrix states that this is because the Communication of each session relies on Windows Store which request the Key to the SC. Can you… Read more »

NDS
NDS
2 years ago
Reply to  NDS

Commenting in case anyone ever arrives here and find this useful:
Issue: SSLVPN Extremely Slow while authenticating via SmartCard.
Configuration applied to make it work:

  • Auth Profile pointing to AAA VSERVER with Authentication Host. (for example auth.acme.com)
  • auth.acme.com points to an IP owned by the NS
  • This is the special configuration, the rest is as Citrix explains in their guide.

Redefine Your Approach to Technology and Innovation

Schedule a call to discover how customized solutions crafted for your success can drive exceptional outcomes, with Ferroque as your strategic ally.