I'm looking to do what I'm calling a swing-box upgrade, where I bring in a box whose sole purpose is for upgrades. I'd like to temporarily bring in a brand new server into our 2.5.3 ESX farm, and it gets to do all the vmfs upgrades, etc. and I can rebuild/reload all the original permanent servers at the same time rather than doing an upgrade of the ESX servers.
I want to shut down all the guests on one vmfs filesystem, have the swing-box upgrade the vmfs filesystem and upgrade the guests. I can then at my leisure rebuild the original ESX server and when completed vmotion the guests from the swing-box to the rebuilt original ESX server. Repeating the process for all the ESX servers in our environment, until they are all complete and I then remove the swing box. Another thought I had was for each server that I could take it down upgrade the host/vmfs/guest all at once, then on the backend take it out of service and reload it... but I don't like the number of moving parts involved (me just being paranoid)
The problem I'm having is that after I upgrade the vmfs filesystem, I am able to register them to a 3.x host because their .vmx config file is incorrect they are unable to be started, upgraded, etc; nor am I able to put them on a 2.5.x host because the vmfs has been upgraded. Maybe that's just the way it is and I'm stuck, but wondering if people have any ideas. If you aren't quite sure what I'm meaning please let me know.
(I've thought about the live migration, but we have ~4.5TB worth of vmdk files and it would take an eternity to move the data to new luns at the speed I've seen in testing 1GB/minute)