VMware Cross vCenter vMotion migration with separate SSO domains
I recently deployed a new vCenter 6.5 VCSA due to my adoption of a new hyper-converged stretched cluster. Similar to mine, there would be scenarios where you would need to deploy a new vCenter appliance rather than upgrading/migrating an older vCenter server.
Now that I have built and configured my new VCSA and configured my new hyper-converged stretched cluster, I needed to migrate my VMs from my old vCenter 6.0 u3e server to my new VCSA 6.5u2c appliance.
The immediate options that came to mind was either using a VM replication solution or VM live migration using VMotion. I had previously completed a similar exercise using VM replication as my source vCenter server was running v5.0 so VM live migration was not an option. As VM replication requires a small downtime of the VM whilst it is being migrated, it was difficult getting outage windows for each VM from the application owners and as with most organisations, there would be some old legacy VMs that nobody owns or knows what application runs on them. Due to the reasons mentioned, that VM migration project took months to complete.
Therefore, from previous experience, I preferred the live VM migration option. However, both source and destination vCenter/VCSA had different PSC servers. Also, connecting both vCenters in enhanced linked mode would not be possible which is the requirement for cross vCenter vmotion via the vSphere Web Client.
VMware KB 2106952 provides a good list of the requirements for cross vCenter migrations. The main point to note from that article mentions:
When using the vSphere APIs/SDK, both vCenter Server instances may exist in separate vSphere Single Sign-On domains. Additional parameters are required when performing a non-federated cross vCenter Server vMotion. The Move-VM cmdlet as of PowerCLI 6.5 supports both federated and non-federated Cross vCenter Server vMotion.
The main points to note from the KB:
- If migrating from vCenter 6.0 to VCSA 6.5, the minimum source vCenter version needs to be v6.0u3.
- PowerCLI 6.5 R1 is required.
- Both source and target must be synced to NTP.
- vSphere Enterprise Plus is required for cross vCenter vmotion.
In addition, I also found the following requirements:
- Distributed switches must run the same version.
- Although VMware KB 2126851 mentions the fix of running a minimum v5.5 DVS in the source vCenter, this solution did not work in my case. I had to upgrade the source DVS to 6.0 and recreate the target DVS from v6.5 to v6.0.
- vMotion vmkernel interfaces must be reachable between source and destination ESXi hosts.
- The following ports needs to be allowed on any firewalls between the source and destination ESXi hosts.
- TCP/UDP 902
- TCP 8000
Most implementation runs vmotion between hosts in a local site or network. As such, the vmotion network can be a non-routable network as there is normally Layer-2 network connectivity between the vmotion vmkernel interfaces between the ESXi hosts. I had the same issue as the vmotion network now needs to be routed for cross vCenter vmotion to work. A quick workaround is to remove the dedicated vmkernel interface used for vmotion (which is best practice) and temporarily allowing vmotion on the management vmkernel interface for the ESXi hsots.
As a reference, the following Powershell script can be used for cross vCenter vmotion:
$sourceVC = "source vCenter name or IP" $destVC = "destination vCenter name of IP" $switchname = "destination distributed switch" $datastorename = "destination datastore" $pgname = "destination portgroup" $vmname = "source VM name" $desthost = "destination ESXi host" $sourceVCConn = Connect-VIServer -Server $sourceVC $destVCConn = Connect-VIServer -Server $destVC $vm = Get-VM $vmname -Server $sourceVCConn $networkAdapter = Get-NetworkAdapter -VM $vm -Server $sourceVCConn $destination = Get-VMHost $desthost -Server $destVCConn $destinationPortGroup = Get-VDPortgroup -VDSwitch $switchname -Name $pgname -Server $destVCConn $destinationDatastore = Get-Datastore $datastorename -Server $destVCConn Move-VM -VM $vm -Destination $destination -NetworkAdapter $networkAdapter -PortGroup $destinationPortGroup -Datastore $destinationDatastore
For those familiar with VMware Flings, there is also a Cross vCenter Workload Migration Utility written in Java available at https://labs.vmware.com/flings/cross-vcenter-workload-migration-utility. This utility provides a GUI for mass migrations rather than using Powershell scripts.