This is a common practice for rotating old physical servers coming off lease, or when moving VM based management servers to a new operating system.
There are some generic instructions on TechNet here: https://technet.microsoft.com/en-us/library/hh456439.aspx however, these don’t really paint the whole picture of what all should be checked first. Customers sometimes run into orphaned objects, or management servers they cannot delete because the MS is hosting remote monitoring activities.
Here is a checklist I have put together, the steps are not necessarily enforced in this order… so you can rearrange much of this as you see fit.
- Install new management server(s)
- Configure any registry modifications in place on existing management servers for the new MS
- Patch new MS with current UR to bring parity with other management servers in the management group.
- If you have gateways reporting to old management servers, install certificates from the same trusted publisher on the new MS, and then use PowerShell to change GW to MS assignments.
- Inspect Resource pools. Make sure old management server is removed from any Resource pools with manual membership, and place new management servers in those resource pools.
- If you have any 3rd party service installations, ensure they are installed as needed on new MS (connector services, hardware monitoring add-ons.
- If you have any hard coded script or EXE paths in place for notifications or scheduled tasks, ensure those are moved.
- If you run the Exchange 2010 Correlation engine – ensure it is moved to a new MS.
- If you use any URL watcher nodes hard coded to a management server – ensure those are moved to a new MS. (Web Transaction Monitoring)
- If you have any other watcher nodes – migrate those templates (OLEDB probe, port, etc.)
- If you have any custom registry keys in place on a MS, to discover it as a custom class for any reason, ensure these are migrated.
- If you have any special roles, such as the RMSe – migrate them.
- Ensure the new MS will host optional roles such as web console or console roles if required.
- Migrate any agent assignments in the console or AD integration.
- Ensure you have BOTH management servers online for a considerable time to allow all agents to get updated config – otherwise you will orphan the agents until they know about the new management server.
- If you perform UNIX/LINUX monitoring, these should migrate with resource pools. You will need to import and export SCX certs for the new management servers that will take part in the pool.
- If you use IM notifications, ensure the prerequisites are installed on the new MS.
- Ensure any new management servers are allowed to send email notifications to your SMTP server if it uses an access list.
- If you have any network devices, ensure the discovery is moved to another MS for any MS that is being removed.
- If you are using AEM, ensure this role is reconfigured for any retiring MS.
- If you are using ACS and the collector role needs to be migrated, perform this and update the forwarders to their new collector.
- If you have customized heartbeat settings for the management server, ensure this consistent.
- If you have any agentless monitored systems (rare) move their proxy server.
- If you were running a hardware load balancer for the SDK service connections – remove the old management servers and add new ones.
- Review event logs on new management servers and ensure there aren’t any major health issues.
- Uninstall old management server gracefully.
- Delete management server object in console if required post-uninstall.
If you have any additional steps you feel should be part of this list – feel free to comment.
Hi Kevin,
For UNIX/Linux monitoring, assuming that I’ve deployed new MS servers with OS 2016, what will happen to all those unix and linux agents which have already installed the SCX certificates that were signed by the ‘old’ management servers? What will happen if the old MS servers are to be removed and the new ones being enforced?
Just export the old management servers certs and save them before you destroy them. That way – any new MS can import/trust the certs that did the signing.
Hi Kevin,
Thanks for this!
We are in the process of a data centre exit and moving servers to Azure. As our SCOM environment sits in this DC, we now have to plan to “lift and shift” so to speak, our SCOM 2016 environment. Personally, I would prefer to build out a new SCOM 2019 environment, but we don’t have time before the DC exit to do that.
Have you done this before? This post covers a lot of this, but we will also need to move the DW and DB.
IP Addresses will change, and possibly also the server names, but we may have some power to retain those.
Would appreciate any advice or pointers to achieve this in the best possible way
Thanks
Andy
Also, don’t forget about reporting!
https://kevinholman.com/2017/01/23/i-retired-a-management-server-now-scom-reporting-doesnt-work/
Hello Kevin,
I have created a new SCOM environment and would like to move the gateway server to the new MS server.
You mentioned above to move gateways to a new management server, that we use powershell to change GW to MS assignments. What specific command scripts do I use in order to accomplish it. I reviewed your blog on “Assigning Gateways and Agents to Management Servers using PowerShell”. Yet, I am not clear with how to move gateways.
Any advice and help is greatly appreciated
I ran across an interesting situation recently. I had one management server, that we had to install in a remote location just for monitoring two physical servers, we ended up virtualizing these servers, and no longer had the issues we had with the physical servers, so I no longer needed this extra management server. For some reason, every agent had this management server as the first failover management server in the failover management server list. After removing the management server, a few of the servers retained this configuration. And were no longer visible under “Agent Managed” view, even though the primary management server for each was exactly the same as every other server. a few of these servers that retained this information are on isolated networks, where we only allow port 5723, so I was not able to connect to them via PowerShell to run the SCOM commands to change the failover management servers, I was basically stuck. I am attempt to re-introduce this management server I removed and then try and make sure that no servers have it as a failover management server, so it can again be removed without causing issues. Everything I read said that I only had to make sure that no servers had it as the primary management server, which none did.
Has anyone ever run into this situation? If so, are there any solutions to prevent this situation?
The SCOM management servers obviously know which management server is trying to communicate with an agent, I am curious why there is no utility to change that only in the SCOM DB, when an agent is not reachable. All of the commands I have seen that handle the Failover Management server list require those commands to be able to reach the actual server you want to make the changes for. What are we supposed to do when that is not possible?
Hello Kevin,
We need to migrate to Win2K19 OS on our existing SCOM 2016,WK216 Infra. Planning to add 2 New MS with SCOM 2016 to the management group and then do the in-place upgrade first on SCOM to 2019, later OS to Win2K19. At last, remove the Old SCOM 2016 MS from Mgmt Group.
Kindly let us know is this plan correct and is the recommended procedure or not.