Monitoring clustered applications can be a little tricky.
Clusters contains virtual, abstract objects, that could be hosted by one or more nodes. We need to ensure we always monitor the clustered resource, no matter where it is running.
We cannot simply target discovery and monitoring to the nodes, because by design the clustered resource will only exist on one node, then all other nodes would generate alerts.
In SCOM, for every clustered resource group that contains an IP Address and a Network Name, we will discover the network name as a Windows Computer object. We will use this “virtual” Windows Computer as the host for our clustered application class.
Essentially – this will be an MP that contains two discoveries – one lightweight discovery for the “Seed” class, then one more granular script based discovery to find only your clustered application class instances.
We start with a simple seed class discovery. The purpose of the seed class is to discover all nodes and virtual computers that “could” host the resource. In this example, I will use a simple registry discovery, and ensure Remoteable = true. Remoteable allows the nodes to discover for the virtual computers. Try to use something specific to your application, such as the existence of a specific service registry for this discovery. This seed discovery will target “Windows Server” class. The reason for this, is that we want to pass a property of the Windows Server Class “IsVirtualNode” to our seed class. This Windows Server class already has a property to know if the object is a cluster virtual object (true) or not (empty).
Next, we will use a script based discovery, and *target* the seed class instances. One of the first things we will pass to the script as a parameter, is the IsVirtualNode property. The script will only return discovery data for virtual objects. Then it will filter further, finding only the instance of your application, through a name, a file, a service, or however you choose to discover it.
Here is the MP example:
The class definitions:
<TypeDefinitions> <EntityTypes> <ClassTypes> <ClassType ID="Demo.MyClusteredApp.Seed.Class" Accessibility="Public" Abstract="false" Base="Windows!Microsoft.Windows.LocalApplication" Hosted="true" Singleton="false" Extension="false"> <Property ID="IsVirtualNode" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" /> </ClassType> <ClassType ID="Demo.MyClusteredApp.Clustered.Class" Accessibility="Public" Abstract="false" Base="Windows!Microsoft.Windows.LocalApplication" Hosted="true" Singleton="false" Extension="false"> <Property ID="ClResourceGroupName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" /> <Property ID="ClResourceName" Type="string" AutoIncrement="false" Key="false" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" /> </ClassType> <ClassType ID="Demo.MyClusteredApp.ComputersAndWatchers.Group" Accessibility="Public" Abstract="false" Base="SCIG!Microsoft.SystemCenter.InstanceGroup" Hosted="false" Singleton="true" Extension="false" /> </ClassTypes> </EntityTypes> </TypeDefinitions>
The seed discovery:
In this example, I am using a very simple seed discovery, such as the existence of the print spooler service. Normally, you’d want something specific to your application:
<Discovery ID="Demo.MyClusteredApp.Seed.Class.Discovery" Enabled="true" Target="Windows!Microsoft.Windows.Server.Computer" ConfirmDelivery="false" Remotable="true" Priority="Normal"> <Category>Discovery</Category> <DiscoveryTypes> <DiscoveryClass TypeID="Demo.MyClusteredApp.Seed.Class"> <Property PropertyID="IsVirtualNode" /> </DiscoveryClass> </DiscoveryTypes> <DataSource ID="DS" TypeID="Windows!Microsoft.Windows.FilteredRegistryDiscoveryProvider"> <ComputerName>$Target/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</ComputerName> <RegistryAttributeDefinitions> <RegistryAttributeDefinition> <AttributeName>SeedRegKeyExists</AttributeName> <Path>SYSTEM\CurrentControlSet\Services\Spooler</Path> <PathType>0</PathType> <!-- 0=regKey 1=regValue --> <AttributeType>0</AttributeType> <!-- 0=CheckIfExists (boolean) 1=treat data as string 2=treat data as INT--> </RegistryAttributeDefinition> </RegistryAttributeDefinitions> <Frequency>14400</Frequency> <ClassId>$MPElement[Name="Demo.MyClusteredApp.Seed.Class"]$</ClassId> <InstanceSettings> <Settings> <Setting> <Name>$MPElement[Name="Windows!Microsoft.Windows.Computer"]/PrincipalName$</Name> <Value>$Target/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</Value> </Setting> <Setting> <Name>$MPElement[Name="System!System.Entity"]/DisplayName$</Name> <Value>$Target/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</Value> </Setting> <Setting> <Name>$MPElement[Name="Demo.MyClusteredApp.Seed.Class"]/IsVirtualNode$</Name> <Value>$Target/Property[Type="Windows!Microsoft.Windows.Server.Computer"]/IsVirtualNode$</Value> </Setting> </Settings> </InstanceSettings> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery Type="Boolean">Values/SeedRegKeyExists</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value Type="Boolean">true</Value> </ValueExpression> </SimpleExpression> </Expression> </DataSource> </Discovery>
The script:
In this example, I first check IsVirtualNode = true, then if so, continue to find instances of my clustered application. You will need to customize this part for each app you wish to discover.
#================================================================================= # Discover Clustered Application # This discovery runs against a seed class target to find a specific clustered app # # Author: Kevin Holman # v1.0 #================================================================================= param($SourceId,$ManagedEntityId,$ComputerName,$IsVirtualNode) # Manual Testing section - put stuff here for manually testing script - typically parameters: #================================================================================= # $SourceId = '{00000000-0000-0000-0000-000000000000}' # $ManagedEntityId = '{00000000-0000-0000-0000-000000000000}' # $ComputerName = "computername.domain.com" # $IsVirtualNode = $true #================================================================================= # Constants section - modify stuff here: #================================================================================= # Assign script name variable for use in event logging. # ScriptName should be the same as the ID of the module that the script is contained in $ScriptName = "Demo.MyClusteredApp.Clustered.Class.Discovery.ps1" $EventID = "9051" #================================================================================= # Starting Script section - All scripts get this #================================================================================= # Gather the start time of the script $StartTime = Get-Date #Set variable to be used in logging events $whoami = whoami # Load MOMScript API $momapi = New-Object -comObject MOM.ScriptAPI #Log script event that we are starting task $momapi.LogScriptEvent($ScriptName,$EventID,0,"`n Script is starting. `n Running as ($whoami). `n ComputerName: ($ComputerName) `n IsVirtualNode: ($IsVirtualNode)") #================================================================================= # Discovery Script section - Discovery scripts get this #================================================================================= # Load SCOM Discovery module $DiscoveryData = $momapi.CreateDiscoveryData(0, $SourceId, $ManagedEntityId) #================================================================================= # Begin MAIN script section #================================================================================= #Only continue if this is a cluster virtual IF (!($IsVirtualNode)) { # Log an event to end discovery $momapi.LogScriptEvent($ScriptName,$EventID,0,"`n IsVirtualNode = false. Do not return discovery data.") } ELSE { # Log an event to continue discovery $momapi.LogScriptEvent($ScriptName,$EventID,0,"`n IsVirtualNode = true. `n We will continue discovery on this cluster object. `n ComputerName: ($ComputerName)") # ====== BEGIN your custom filter for clustered app ====== # Now we need to create a filter so that only the app we are looking for is discovered # This might be using a cluster powershell filter, WMI query, or looking for a file or process or however makes sense for your app # This example will discover any cluster resource with a NetworkName object type, that has a DNS name value that matches the target computer as an example only # Import cluster PS module Import-Module FailoverClusters #Get all the cluster resources on this cluster [array]$ClResources = Get-ClusterResource | where {$_.ResourceType.Name -eq "Network Name"} FOREACH ($ClResource in $ClResources) { #Get the Cluster Resource Name [string]$ClResourceName = $ClResource.Name #Get the NetBIOS name from the ComputerName passed as a param to the script [string]$ComputerNameSplit = ($ComputerName.Split("."))[0] #Get the DNS name from the cluster network name object $ClDNSNameObj = $ClResource | Get-ClusterParameter -Name DnsName [string]$ClDNSName = $ClDNSNameObj.Value #Get only the NetBIOS name for comparison [string]$ClDNSNameSplit = ($ClDNSName.Split("."))[0] IF ($ComputerNameSplit -eq $ClDNSNameSplit) { #Discover stuff [string]$ClResourceGroupName = $ClResource.OwnerGroup.Name $momapi.LogScriptEvent($ScriptName,$EventID,0,"`n Adding discovery data for: `n ComputerName () `n Cluster Resource Name: ($ClResourceName) `n Cluster Resource Group Name: ($ClResourceGroupName)") $instance = $DiscoveryData.CreateClassInstance("$MPElement[Name='Demo.MyClusteredApp.Clustered.Class']$") $instance.AddProperty("$MPElement[Name='Windows!Microsoft.Windows.Computer']/PrincipalName$", $ComputerName) $instance.AddProperty("$MPElement[Name='System!System.Entity']/DisplayName$", $ComputerName) $instance.AddProperty("$MPElement[Name='Demo.MyClusteredApp.Clustered.Class']/ClResourceName$", $ClResourceName) $instance.AddProperty("$MPElement[Name='Demo.MyClusteredApp.Clustered.Class']/ClResourceGroupName$", $ClResourceGroupName) $DiscoveryData.AddInstance($instance) } } # ====== END your custom filter for clustered app ====== } # Return Discovery Items Normally $DiscoveryData # Return Discovery Bag to the command line for testing (does not work from ISE) # $momapi.Return($DiscoveryData) #================================================================================= # End MAIN script section # End of script section #================================================================================= #Log an event for script ending and total execution time. $EndTime = Get-Date $ScriptTime = ($EndTime - $StartTime).TotalSeconds $momapi.LogScriptEvent($ScriptName,$EventID,0,"`n Script Completed. `n Script Runtime: ($ScriptTime) seconds.") #================================================================================= # End of script
Now, as you can see – this gets pretty complicated to write for each clustered application.
However, it doesn’t have to be complicated. We can wrap all this up into a Fragment, and reuse it over and over! I am adding example fragments to my fragment library: https://github.com/thekevinholman/FragmentLibrary
These fragments will make it super easy to reuse this code sample, and just tweak it for your specific clustered apps.
Class.And.Discovery.ClusteredApp.RegistrySeed.mpx
Combo.Class.Discovery.ClusteredApp.RegistrySeed.ComputerWatcherGroup.Views.Folder.mpx
When I use the combo fragment, I can see my work pay off instantly. I only need to provide 3 pieces of information:
The CompanyID, the AppName, and the RegKeyPath for the SEED class. That’s it!
Then I can literally import this into SCOM. Of course, you will want to customize the sample script first, but all the MP heavy lifting is done.
I have a nice prebuilt folder and state views:
I have discovered instances of my SEED class based on the registry:
I have discovered instances of my clustered applications as well, including properties about the cluster resource and group they are from:
This makes something pretty hard, really easy to use. 3 simple pieces of input information, customize the script, and you are done.
To see more about MP fragments, check out the basics:
I feel like I’m missing something here. Great seed-based discovery example, especially the ability to discover resources hosted within the cluster. However, it’s not really “monitoring” anything. We’d still need to target a service or application monitor at the discovered “Clustered.Class” object, right?
Would it be possible to put together a fragment that goes the whole way, discovering and monitoring a cluster-hosted service?
You are correct – this is just the class and discovery structure. From there – you can target monitoring workflows to the cluster class. This is kind of the point of fragments – lots of easy to add-on components. So now that you have this, just bring in a fragment using Visual Studio or Silect MP Author Pro, for service monitoring, and target your Cluster class you just created. It takes less than 60 seconds to do.
Hello Kevin,
the discovery works fine. After that I tried to point the monitoring to the cluster resource and it works fine at the beginning. But after the cluster switch to the other node, the monitor will not switch over to the other node. Do I have to turn down the interval for the cluster resource discovery?
Kind regards
I’ll have to play with that. When I wrote this I tested it and it worked fine. What version of SCOM and SCOM agent?
Hi Guys, could you tell me how about the problem in SCOM 2019 with Windows Server 2012 R2 cluster monitoring ? Looks like SCOM 2019 has a problem with proper discovery of the clusters and cluster respource (agentless monitoring).
Have not heard anything about this. What’s the problem?
Hey guys, I have just installed SCOM 2019 and the Cluster MP. I have 4 clusters all of which were discovered but only 2 of which say they are ‘monitored’. I don’t see what the difference is between the monitored and non-monitored clusters. The nodes of the clusters are all monitored and ‘proxy agent’ is enabled on all cluster members. Any idea? This is my first go around with SCOM 😉
This is normal, and has to do with what resource groups get discovered, monitored, and roll up their health state. There must not be any discovered resource groups on one of those clusters? What view shows “not monitored”? If you are looking at Windows Computers that’s normal. If you are look at a “Cluster State” view that’s probably not.
Thanks Kevin for the quick reply! I am looking under Microsoft Windows Cluster>Cluster>Cluster state. Two of the clusters say ‘Healthy’ and two say ‘not monitored’ I notice that the cluster names DO appear under Admin>Agentless Managed…and under their they are marked health, but i assume that area is not cluster aware.
Hmmm, that view should show as “monitored”. That view has objects of “Windows Cluster”. It should have health rollup from any and all resource groups in the cluster, and it should have health rollup from the operating systems for the nodes.
Make sure you have the appropriate OS MP’s installed and that you have an operating system instance discovered for the nodes.
Open health explorer for the “not monitored” ones and remove the filter, and see if there are child items.
Make sure you have enabled agent proxy for discovery.
In my case SCOM 2019, Microsoft Windows Cluster>Cluster>Cluster state all are healthy.
But under Admin>Agentless Managed are showing as Not monitored.
Proxy is enabled for all the agents but still the status is same.
Windows Cluster Management Library and Management monitoring are imported for 2012, 2012 R2 and 2016 MP’s .
Please suggest if i am missing any thing here.
Wellllll. I disabled filters one of the ‘not-monitored’ clusters and their are no child items (not checked or with info anyway) both nodes in this two node cluster are monitored and healthy and are windows server 2016 (same as my two ‘good’ clusters) Each node has ‘allow this agent to act as proxy. This wasnt enabled during the initial discovery but i’m assuming thats ok since the other 2 came up fine after some time. ‘Cluster Service State’ gives me all green lights for the nodes in all 4 clusters if thats significant?
Hi Kevin, Default SQL cluster monitoring dosen’t work even after setting service state check to false in the overrides. Do you have any comments on that.
What exactly does that mean?
Hi Kevin, we are trying to monitor two clusters which have the same name but are on different domains:
CLUSTER1.domain1.com
– NODE1.domain1.com
– NODE2.domain1.com
CLUSTER1.domain2.com
– NODE3.domain2.com
– NODE4.domain2.com
It looks like SCOM has only discovered one cluster object but thinks all four nodes are part of the same cluster. I assume this is because the name of the cluster object isn’t the FQDN?
Is this expected behaviour, and is there any way around it?
Hi Kevin, Thanks for the detailed steps. I was able to create the MP and also the discoveries are also working fine, Now I need to monitored the Cluster services for the Application. Any help or the article to continue work on creating monitors for clustered application resources and Services.
Hi Kevin, Any help for adding the monitors for monitoring the cluster resources for the discovered application cluster would of great help. There is a requirement to monitor the cluster resources including the application Services which are reporting in cluster manager as online and raise an alert if any resource is not online.
Hi Kevin, Just wondering, Would you just create a monitor for a service to target the discovered clustered class directly…
or in the discovery section below would you need to include a second additional discovery code of a service along with the Cluster Resource discovery code?
# ====== BEGIN your custom filter for clustered app ======
The reason I asked is I imported a monitor fragment and targeted the cluster discovery class but the class remains Healthy regardless if the services are stopped on either node?