Menu Close

How to generate an alert and make it look like it came from someone else

This capability has been around forever, but I have never seen it documented.  This is a really cool way to generate alerts as if they came from other agents, but target a different agent.

Suppose a scenario:  You have a client/server application (such as a backup program) where a central server logs all the events about success or failed jobs from clients.

This is scenario – we could simply generate alerts targeting the central server, and reading the event log, and bubble up the broken client name from the logs, into the alert.  The challenge becomes, what if some agents are test, or dev, and some are prod?  What if we have already put in place “tiering” of servers by groupings, and we use this to filter which alerts from which servers get ticketed?

There is actually a way to target one instance of a class with a workflow, but to generate alerts as if they came from a different instance of a different class, EVEN if that instance is a different agent altogether!

Let me demonstrate:

The most common write action for generating alerts in rules, is System.Health.GenerateAlert, which is the one commonly used in every Alert Generating rule you typically come across.  It is documented here:  https://msdn.microsoft.com/en-us/library/ee809352.aspx

HOWEVER – there is another write action you can use:  System.Health.GenerateAlertForType.

This is documented here:  https://msdn.microsoft.com/en-us/library/jj130310.aspx  While we document the modules and a sample XML example, we don’t really give much guidance anywhere on use cases.

This is a really cool write action, which allows us to generate alerts “on behalf” of a different object type, or even a different object type from a different computer!  Let me show the difference:

A typical System.Health.GenerateAlert looks like this:

<WriteAction ID="GenerateAlert" TypeID="Health!System.Health.GenerateAlert"> <Priority>1</Priority> <Severity>1</Severity> <AlertMessageId>$MPElement[Name="Demo.Rule.AlertMessage"]$</AlertMessageId> <AlertParameters> <AlertParameter1>$Data/EventDescription$</AlertParameter1> </AlertParameters> </WriteAction>

As you can see – very simple.  It sets the priority and severity of the alert, references the Alert Message ID (which is the alert name and description configuration) and contains any alert parameters we want to use in the display output (in this case, Event Description is very common).

 

Now, see the System.Health.GenerateAlertForType:

<WriteAction ID="GenerateAlertForTypeWA" TypeID="Health!System.Health.GenerateAlertForType"> <Priority>1</Priority> <Severity>1</Severity> <ManagedEntityTypeId>$MPElement[Name="Example.Client.Class"]$</ManagedEntityTypeId> <KeyProperties> <KeyProperty> <PropertyId>$MPElement[Name="Windows!Microsoft.Windows.Computer"]/PrincipalName$</PropertyId> <IsCaseSensitive>false</IsCaseSensitive> <Value>servername.fqdn.local</Value> </KeyProperty> <KeyProperty> <PropertyId>$MPElement[Name="Example.Client.Class"]/ClientName$</PropertyId> <IsCaseSensitive>false</IsCaseSensitive> <Value>servername.fqdn.local</Value> </KeyProperty> </KeyProperties> <AlertMessageId>$MPElement[Name="Demo.Rule.AlertMessage"]$</AlertMessageId> <AlertParameters> <AlertParameter1>$Data/EventDescription$</AlertParameter1> </AlertParameters> </WriteAction>

The key section here is <ManagedEntityTypeId> and then some <KeyProperties>

In the <ManagedEntityTypeId> we need to reference the CLASS that we want the alert to appear as it is coming FROM.

Then, in the <KeyProperties> we need two sections:

The first key property is mapping the Windows Computer principal name to the fqdn of the agent we want the alert to “appear to be from”.  This part is easy.

The second key property is mapping the SAME fqdn, to a matching property on the CLASS we referenced in <ManagedEntityTypeId>, or a parent base class that has the key property defined.

The second key property is the tough one.  The criteria for this to work (from my testing) is that we MUST have a class with a key property first, and that key property MUST be the fqdn of the agent/server for each instance (or whatever value we are “matching” on.

In most of my classes I create, I don’t create key properties.  Key properties aren’t required unless I have a class that will discover multiple instances on the same healthservice (agent).  For stuff I do – this is rarely the case.  However, it is EASY to create a key property for your custom classes, and many Microsoft classes already have key properties.  The big “gotchya” here is that in order to generate an alert for another instance of a class (not the targeted instance), the class we specify MUST have a key property defined for this to work.

So – I simply added a key property of “ClientName” to my custom class, and then to discover it, all I have to do is add some simple code to the discovery which maps the hosting Windows Computer principal name to the property.

Ok…. I know…. I probably lost a lot of you up to this point….. but it is easier to just do it, than it is to understand it.  That’s why I will post my XML examples at a link below.  Smile

 

Here is an example of me adding a custom key property to my custom class:

<ClassType ID="Example.AlertFromAnotherInstance.Client.Class" Accessibility="Public" Abstract="false" Base="Windows!Microsoft.Windows.LocalApplication" Hosted="true" Singleton="false" Extension="false"> <Property ID="ClientName" Type="string" AutoIncrement="false" Key="true" CaseSensitive="false" MaxLength="256" MinLength="0" Required="false" Scale="0" /> </ClassType>

And here is part of the discovery that I will use to map “ClientName” to the hosting Windows Computer principal name:

 

<Discovery ID="Example.AlertFromAnotherInstance.Client.Class.Discovery" Enabled="true" Target="Windows!Microsoft.Windows.Server.OperatingSystem" ConfirmDelivery="false" Remotable="true" Priority="Normal"> <Category>Discovery</Category> <DiscoveryTypes> <DiscoveryClass TypeID="Example.AlertFromAnotherInstance.Client.Class" /> </DiscoveryTypes> <DataSource ID="DS" TypeID="Windows!Microsoft.Windows.FilteredRegistryDiscoveryProvider"> <ComputerName>$Target/Host/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</ComputerName> <RegistryAttributeDefinitions> <RegistryAttributeDefinition> <AttributeName>ClientExists</AttributeName> <Path>SOFTWARE\Demo\Client</Path> <PathType>0</PathType> <AttributeType>0</AttributeType> </RegistryAttributeDefinition> </RegistryAttributeDefinitions> <Frequency>86400</Frequency> <ClassId>$MPElement[Name="Example.AlertFromAnotherInstance.Client.Class"]$</ClassId> <InstanceSettings> <Settings> <Setting> <Name>$MPElement[Name="Windows!Microsoft.Windows.Computer"]/PrincipalName$</Name> <Value>$Target/Host/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</Value> </Setting> <Setting> <Name>$MPElement[Name="System!System.Entity"]/DisplayName$</Name> <Value>$Target/Host/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</Value> </Setting> <Setting> <Name>$MPElement[Name="Example.AlertFromAnotherInstance.Client.Class"]/ClientName$</Name> <Value>$Target/Host/Property[Type="Windows!Microsoft.Windows.Computer"]/PrincipalName$</Value> </Setting> </Settings> </InstanceSettings>

 

So – now all I need to do it write a rule, and use our new write action.

You can write the event rule like my example will do using the console, or any other tool, then simply modify the write action section in XML.

Here is my simple rule:

 

<Rule ID="Example.AlertFromAnotherInstance.Server.Event.Rule" Enabled="true" Target="Example.AlertFromAnotherInstance.CentralServer.Class" ConfirmDelivery="false" Remotable="true" Priority="Normal" DiscardLevel="100"> <Category>EventCollection</Category> <DataSources> <DataSource ID="Microsoft.Windows.EventCollector" TypeID="Windows!Microsoft.Windows.EventCollector"> <ComputerName>$Target/Host/Property[Type="Windows!Microsoft.Windows.Computer"]/NetworkName$</ComputerName> <LogName>Application</LogName> <AllowProxying>false</AllowProxying> <Expression> <And> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery Type="UnsignedInteger">EventDisplayNumber</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value Type="UnsignedInteger">999</Value> </ValueExpression> </SimpleExpression> </Expression> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery Type="String">PublisherName</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value Type="String">TEST</Value> </ValueExpression> </SimpleExpression> </Expression> </And> </Expression> </DataSource> </DataSources> <WriteActions> <WriteAction ID="GenerateAlertForTypeWA" TypeID="Health!System.Health.GenerateAlertForType"> <Priority>1</Priority> <Severity>2</Severity> <ManagedEntityTypeId>$MPElement[Name="Example.AlertFromAnotherInstance.Client.Class"]$</ManagedEntityTypeId> <KeyProperties> <KeyProperty> <PropertyId>$MPElement[Name="Windows!Microsoft.Windows.Computer"]/PrincipalName$</PropertyId> <IsCaseSensitive>false</IsCaseSensitive> <Value>$Data/Params/Param[1]$</Value> </KeyProperty> <KeyProperty> <PropertyId>$MPElement[Name="Example.AlertFromAnotherInstance.Client.Class"]/ClientName$</PropertyId> <IsCaseSensitive>false</IsCaseSensitive> <Value>$Data/Params/Param[1]$</Value> </KeyProperty> </KeyProperties> <AlertMessageId>$MPElement[Name="Example.AlertFromAnotherInstance.Server.Event.Rule.AlertMessage"]$</AlertMessageId> <AlertParameters> <AlertParameter1>$Data/Params/Param[1]$</AlertParameter1> <AlertParameter2>$Data/EventDescription$</AlertParameter2> </AlertParameters> </WriteAction> </WriteActions> </Rule>

 

The rule is simple.  It simply looks in the Application Event log for an event ID 999, with a event source of “TEST”.  If found, run the write action.  If you scroll down, you can see the write action part, which I will explain:

 

In my rule, I am targeting the workflow to run on the “Server” class.  However, in my write action, I want the alert generated by instances of the “Client” class.  So on my <ManagedEntityTypeId> line, I am using Example.AlertFromAnotherInstance.Client.Class which is my client class ID.

Next, I map the key property for Windows Computer (Principal Name) to the machine I want to appear to generate the alert.  In this case, the name of the affected machine is in Param 1 of my test event, so I am mapping whatever name is in Param1 of the event to generate the alert.

Next, I map the key property of my custom class to the SAME FQDN value.

That’s it!

 

In this example – I create an event on my “Server”, and param 1 of the event will have the name of the client I want the alert to come from:

image

 

Note:  in the above image – the event was logged on a Server named “STORAGE.opsmgr.net” but param1 contained a name of “RD01.opsmgr.net”.

As long as RD01.opsmgr.net hosts an instance of my “Client” class, an alert will be generated as if it came from this server:

 

image

 

 

If you want to test my example XML out in your own environment, simply create some reg keys to be the “Server” and the “Client” instances to be discovered:

HKEY_LOCAL_MACHINE\SOFTWARE\Demo\Server

HKEY_LOCAL_MACHINE\SOFTWARE\Demo\Client

 

The example management pack is available for download at:

 https://github.com/thekevinholman/AlertFromAnotherInstance


8 Comments

  1. CyrAz

    Hi Kevin,

    That’s a really nice trick that I already have used quite a few times.

    But I would now want to do kind of the same thing with a monitor… I’m already pretty sure what I have in mind can’t be achieved, but let’s give it a try :
    – The software I want to monitor runs on multiple active/active servers
    – The monitoring data is obtained through a REST API running locally on every server. Every server returns all the monitoring data for the whole environment.
    – I have no issue running Discoveries locally on every server but still only instantiating only one object of each type for the whole environment in SCOM using unhosted classes, key properties etc.

    Now what I would like to achieve is to have a monitor targeted at a unhosted class, but still running locally on every server… Do you know if/how that can be done?

    The only options I can think of so far are :
    – First discover “local instances” of my objects on every server, then a “clustered instance” containing these local instances ( using a ClassAndRelationshipSnapshotDataMapper discovery) . I would then target the actual monitors at the local instances level and roll them up to the clustered instance using dependencies monitors, and have these generate the alerts; but that wouldn’t allow me to keep Alert’s context…
    – Same thing as previous option except don’t generate alerts from the monitors at all and use an alerting rule based on System.Health.GenerateAlertForType to generate the alerts at the “clustered instance” level.

    Thanks for any idea you could have 🙂

    CyrAz

    • Kevin Holman

      I like to keep things simple. I’d use a PowerShell timedscript rule, which would have the RESTAPI stuff in the datasource, then a writeaction using GenerateAlertfor type. That way you only get one Alert…. just like this example….. it will just have a very high repeat count as each local instance will trigger. You are correct – the other way is to use an unhosted object similar to how we do AD Site health, monitors on the local instances, using a health rollup policy from contained instances, but the issue there becomes lack of context in the monitor that you roll health up to.

      You CAN create an unhosted object, that is managed by agents – examples here: https://kevinholman.com/2014/01/19/how-to-create-workflows-that-wont-go-into-maintenance-mode/

      However, while that is what you said – I think you meant you want an object managed by the management servers (a singleton object representing your application) but having targeted workflows run on the agents. No way to do that, I am aware of. You could make a “representation” of that model, by creating events, then collecting those events, then driving monitor state of the singleton object by running a query to the DB and finding events of a specific type, and get context from the event. That’s kinda ugly and I would not want to implement that on a large scale.

  2. Scott

    Is it possible to just use a data property returned by a PowerShell DS containing the FQDN and match it to the FQDN of the Microsoft.Windows.Computer?

    i.e. I want to run a script on a management server or perhaps a watcher node, and return alert data related to a monitored Windows computer from a REST API, and then subsequently generate an alert related to the Windows computer. I don’t want/need to create any classes, I just need a way to generate an alert for the computer based on what is returned when a foreign system is queried.

    • Scott

      Well, it wouldn’t take the xml, but all that appears to be needed here is to use Windows Computer as the managed entity type id and one keyproperty that matches where the FQDM matches a data field in the return, and it works!

  3. Scott

    On second thought… yes this works:

    $MPElement[Name=”Windows!Microsoft.Windows.Computer”]$

    $MPElement[Name=”Windows!Microsoft.Windows.Computer”]/PrincipalName$
    false

  4. KM

    I have trouble getting this work, tried the example MP , enabled all the registery but still do not see an alert in console for any of the event i trigger….any suggestion will be helpful…

  5. Graham

    Hi Kevin, hoping you might reply with a bit of guidance. I have created a MP as per following this guide https://ds.squaredup.com/blog/how-to-create-100s-of-tcp-monitors-in-minutes-with-scom/ which I have adapted to monitor iMonitor Ports on SLES OES servers, works really well. The problem I face is the watcher node is a Windows Computer, (seems you can only have a Windows Computer Watcher node even if creating port monitors using the Console Template), and Alerts are generated against the Watcher Node Principal Name/Path in the Alert, so alerts appear to come from the Watcher Node. We redirect Alerts to Email/SMS based on the “computer to Support Team” relationship lookup in a SQL DB Table. The Linux Servers are managed by 18 different Support Teams around the State of Queensland. The Alerts come to the Team that manages the Windows Computer, in this case the us SCOM Team. We don’t send Alerts to a central Team that then allocates them out to the various Teams, Alerts go directly to the various Teams. Question, is it possible to use your scenario to redirect the Alerts to appear to come from the Linux Server as the Principal Name/Path in the Alert? I have read through your example but not sure where could start to achieve this in my current MP. I can email the MP if you like so you can see what it does and how it is constructed with the Class and Discovery and Alerts. Thank you in advance.

Leave a Reply

Your email address will not be published.