Introduction and Background

A customer upgrading from CVAD 2203 LTSR CU3 to CU4 Update 1 on Windows Server 2019 encountered broken start menus post-updates with Citrix Profile Management (CPM/UPM) as the profile management solution. The experience would also impact intermittently users who logged back into non-upgraded VDAs.

This issue has been reported in the past in multiple scenarios in Server 2016, 2019, and 2022. Some causes of this issue were alleged to have been fixed years ago, but forums are peppered with cases of this occurring in the past couple of years (Ref 1, Ref 2). In short, something is causing corruption of roaming Start Menus post-upgrade.

Environment:

Windows Server 2019
Multi-Session Citrix 2203 LTSR VDA – both CU4 and CU4 Update 1 tested with the same result
Hosted Shared Desktops (aka Multi-Session)
Citrix Profile Management 2203 LTSR matching CU with best practice configurations on CPM settings in place with the exception of accelerated folder mirroring
Microsoft Folder Redirection in use

Troubleshooting

Troubleshooting occurred on Citrix VDAs upgraded from CU3 to CU4 Update 1.

Microsoft Folder Redirection

The environment did have AppData (Roaming) and Start Menu redirected per legacy environment configurations. In testing with a pilot user group, however, redirecting these profiles back into the local user profile (and confirming their redirection was completed via UPMConfigCheck script and event logs) had no effect on the issue. These days, redirecting these folders is ill-advised due to various reasons from frequent calls to AppData to potential issues with Start Menu corruption.

Redirecting Start Menu back into the profile was implemented for all users, but due to the significant login delay resulting from redirecting AppData back into the user profile (in this case nearly 10 minutes added to next login), this configuration was left-as is as there was no net benefit, not even in combination with CPM modifications.

Citrix Profile Management

CPM automatic configuration was tested from its default to disabling the feature and manually configuring the CPM sync and Folder Mirroring settings for Windows Server as per CPM best practices configuration and despite confirming settings were in place, this yielded no results.

Accelerated Mirroring was enabled for a pilot set of users and on second logins, this did in fact resolve the issue and restore the Start Menu to proper functionality, and is an official Citrix recommendation to use. It has has evidence of resolving the issue for others.

However, this was not a viable fix in this situation. The creation of those containers results in a 200MB VHDX per user profile (which in many instances was already well under 50MB) plus an initial 40MB for the delta VHDX disk for any active Citrix user. Multiplying this by the number of users, this presented an untenable solution, one that would have caused a storage outage without prior planning and capacity expansion. Further, the content of this VHDX in our case was a meager 800KB. So 200MB per user to fix a sub-1MB per user problem. Difficult to rationalize. Later releases of CPM have a disk compaction capability, but unknown if this would have rectified this concern, but furthermore, this is a strict LTSR environment.

Resolution

In the end, an old fix was “the” fix. The resolution, or rather, workaround in our case was to implement the fix noted in CTX239584 by forcing a user-level Group Policy Preference in “Update” mode to reset the Immersive Shell cache at each logon. This did not require significant architectural changes, nor noticeably impeded functionality nor login times for users.

Note that it is not necessary to implement the fix as a logon script as per the articles as the use of logon scripts in general are both legacy and frowned upon due to their inherent problems. Implemnting via GPO through a user-level GPP was sufficient.

This resolved the issue both for users logging into CU4 upgraded systems, and those who might be logging into a CU4 system and back into a CU3 system during the upgrade cycle including repeated login testing.

Michael Shuster

Michael is Ferroque's founder and a noted Citrix authority, overseeing operations and service delivery while keeping a hand in the technical cookie jar. He is a passionate advocate for end-user infrastructure technology, with a rich history designing and engineering solutions on Citrix, NetScaler, VMware, and Microsoft tech stacks.