Introduction and Background
Recently, we have been working with a customer conducting a Citrix ADC migration. In this particular scenario, the customer was most comfortable orchestrating the migration by staging VIPs on the destination appliances using bogus IPs, and on cutover using scrips to change IPs on the source appliances to bogus IPs, and re-IPing the VIPs on the new appliances to the correct IPs. In this scenario, we effectively staged all configurations on the appliances, with the exception of temporary IPs on the VIPs.
We ran into an anomalous error that for a while, we could not make heads or tails of. The customer’s StoreFront VIP on HTTPS could not be IP’d to the correct address via GUI and not even CLI, citing “IP already in use”.
The configuration files were scoured, absolutely zero traces of this IP in use elsewhere beyond the HTTP 80 VIP. Re-IPing the HTTP 80 VIP temporarily still did not resolve the issue. We had removed any trace of the IP from the configuration file, yet we still could not configure the VIP with the correct IP. To make things more aggravating, we could use the IP on any port but 443.
After doing some digging, it came to our attention that behind the scenes, the ADC sets up probes for addresses listed in the session policies, if any are configured on the appliance. In our case, we had session policies referencing VIPs on the appliance for StoreFront. As DNS resolution was live on the appliance, it had established a monitoring pattern we guess, against the IP and port yet to be migrated off the old appliance. On account of this, it would not allow our SSL VIP to be IP’d correctly.
To work around this “chicken before the egg” problem, we temporarily scraped the URLs out of our session policies. This freed up the IP and port in question for us to successfully migrate the VIP. Once migrated, we adjusted the session policy back to its original configuration.
After resolving the issue in question, I’d checked with peers within the consulting organization at Citrix and although rare, it has come up before in the field. Our circumstance due to the order in which we staged the configurations, and the fact a session policy was dependent on a VIP VIP also hosted on the appliance, was to blame.
Hopefully, this article speeds up resolution time for others who might experience such an issue in the middle of a migration.
Michael Shuster is Ferroque Systems’ Chief Architect and noted Citrix authority. A passionate virtualization and digital workspaces advocate, he has designed, engineered, or otherwise advised clients on Citrix, VMware, and Microsoft technology platforms across the globe.