Issue: Long Delay Establishing Citrix SSL VPN Connection

Issue and Background

Earlier this year, we worked with a customer to develop a Citrix ADC-powered alternative to a next-generation firewall (NGFW) where based on user ID, users would have network authorization rules applied to various backend networks and servers. For that creative solution, you can read up on it here: HowTo: Create a Citrix Gateway SSL VPN with SSO via Kerberos.

Months of testing were quite successful, however, one strange issue kept appearing: on initial connection establishment, there was a delay of up to 60 seconds establishing the VPN connection. This created a poor user experience and needed resolution before entering production.

The key factor that would help us zero in on the root causes was the issue never occurred if the client was connected to a network with Internet access. For the customer use case, this solution was developed for use in a high-security setting thus no Internet connectivity is permitted.

Root Cause and Resolution

Ultimately there were two causes, both of which contributed to the lengthy connection time.

• CEIP call-outs to Google Analytics
• CRL checking against the SSL VPN certificate

For both issues, we were able to discern the root causes through examination of the client-side logs for the Citrix Gateway plugin.

Here is an example of failure states for the Google Analytics calls.

2021-05-26 15:02:41.180 | Tid: 10020 | ERROR  | sendGARequestThread | 165 | Failed at HttpSendRequest Error code: 12029

2021-05-26 15:02:46.190 | Tid: 10020 | DEBUG  | Sending v=1&aip=1&tid=UA-59929653-34&cid=4e4e34ea-e640-4db6-972d-3aa1be72ad09&an=WinVPN&av=21.3.1.2&ul=en&t=screenview&cd=init&cd1=Win10_64-bit&cd2=11.1411.18362.0 for Google Analytics request.

2021-05-26 15:03:07.227 | Tid: 10020 | ERROR  | sendGARequestThread | 165 | Failed at HttpSendRequest Error code: 12029

For the Google Analytics telemetry, this was surprisingly difficult to zero in on. There are no clearly identifiable means within the client UI to disable the Customer Experience Improvement Program (CEIP) as there is in the Citrix Workspace app. The fix is however documented but is done more as a footnote in this Citrix document here.

By setting the registry key on the client as follows, we disable the call-outs to Google Analytics by the VPN client. Be sure to quit and reload the VPN client after implementation.

“HKLM\Software\Citrix\Secure Access Client\DisableGA” of type REG_DWORD to 1

We will see this evidence before and after in logs when the Citrix Gateway VPN plugin loads:

Before registry change:

2021-06-24 14:15:19.460 | Tid: 19704 | DEBUG | Google Analytics is enabled on this machine

After registry change:

2021-06-24 14:24:13.708 | Tid: 23692 | DEBUG | Google Analytics is disabled on this machine

For the second issue contributing to connection delay, we had a CRL issue to deal with. The VPN plugin performs a CRL check as part of the connection process. As they were using a certificate from a commercial CA, a similar issue related to no outbound Internet connectivity was encountered. This solution was being addressed through an exception rule for CRL checking against DigiCert to resolve the matter. Alternatively, the customer is looking to correctly configure CRLs on their internal CA infrastructure and avoid outbound Internet calls for CEL checks altogether.

Hopefully, these tips will assist others who might encounter Citrix SSL VPN challenges in very restricted use cases.