I recently had an interesting customer issue.
They wanted to collect some specific events for reporting. This was specific to NTLM authentications. The problem was – that these events could potentially FLOOD some servers, and they didn’t want to collect ALL the events, just a sample of unique events on a somewhat regular basis, so understand how often this happens.
SCOM is the perfect tool for this, because of the Consolidator Condition Detection Module. Let me explain.
Writing a SCOM event collection rule is simple and easy. But in this case – we need to “consolidate” these events, and only collect ONE event in a set time period. For this – we will add a single condition detection to our rule, which is the System.ConsolidatorCondition
This is very similar to the rules I demonstrated here:
The rule will look just like most event rules, it has a Microsoft.Windows.EventProvider datasource, and two write actions to collect the event to the OpsDB and DW. However, it will have a condition detection added.
Here is the XML for the rule:
<Rule ID="Demo.CollectConsolidatedEvent.Event999.Rule" Enabled="true" Target="Windows!Microsoft.Windows.Server.OperatingSystem" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100"> <Category>Custom</Category> <DataSources> <DataSource ID="DS" TypeID="Windows!Microsoft.Windows.EventProvider"> <ComputerName>$Target/Host/Property[Type="Windows!Microsoft.Windows.Computer"]/NetworkName$</ComputerName> <LogName>Application</LogName> <Expression> <And> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery Type="UnsignedInteger">EventDisplayNumber</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value Type="UnsignedInteger">999</Value> </ValueExpression> </SimpleExpression> </Expression> <Expression> <SimpleExpression> <ValueExpression> <XPathQuery Type="String">PublisherName</XPathQuery> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value Type="String">TEST</Value> </ValueExpression> </SimpleExpression> </Expression> </And> </Expression> </DataSource> </DataSources> <ConditionDetection ID="CD" TypeID="Demo.CollectConsolidatedEvent.Consolidate999ThenMapCD" /> <WriteActions> <WriteAction ID="CollectEventOpsDB" TypeID="SC!Microsoft.SystemCenter.CollectEvent" /> <WriteAction ID="CollectEventDWDB" TypeID="SCDW!Microsoft.SystemCenter.DataWarehouse.PublishEventData"/> </WriteActions> </Rule>
Notice the line for the condition detection:
<ConditionDetection ID=”CD” TypeID=”Demo.CollectConsolidatedEvent.Consolidate999ThenMapCD” />
This is a custom composite condition detection I wrote – which does two things:
1. It will consolidate all the events into a single Consolidated DataItem.
2. Since the event collection write actions require Microsoft.Windows.EventData as an input, and the Consolidator outputs System.ConsolidatorData, we need to add something called a “Mapper” which will convert one data type to another. In this case, I will be using the System.Event.GenericDataMapper You don’t need to understand the nitty gritty…. but basically I will be mapping data from the first module so it can be output as standard event data for the collection write actions to consume. Mappers are really cool – because they let us take data from ANYTHING and make it look like events, perf data, etc. We can hard-code mapper values, or we can pass then dynamic data from the previous module.
Here is the custom composite condition detection I used:
<ConditionDetectionModuleType ID="Demo.CollectConsolidatedEvent.Consolidate999ThenMapCD" Accessibility="Internal" Batching="false" Stateful="true" PassThrough="false"> <Configuration> </Configuration> <ModuleImplementation Isolation="Any"> <Composite> <MemberModules> <ConditionDetection ID="Consolidate" TypeID="System!System.ConsolidatorCondition"> <Consolidator> <ConsolidationProperties> <!-- If you want to consolidate ONLY events with matching identical parameters and collect multiple events when params are different, uncomment this section for your specific params --> <!-- <PropertyXPathQuery>Params/Param</PropertyXPathQuery> <PropertyXPathQuery>Params/Param</PropertyXPathQuery> <PropertyXPathQuery>Params/Param</PropertyXPathQuery> --> </ConsolidationProperties> <TimeControl> <WithinTimeSchedule> <Interval>30</Interval> <!-- set the time windows to look for events in seconds --> </WithinTimeSchedule> </TimeControl> <CountingCondition> <CountMode>OnNewItemNOP_OnTimerOutputRestart</CountMode> </CountingCondition> </Consolidator> </ConditionDetection> <ConditionDetection ID="EventMapper" TypeID="System!System.Event.GenericDataMapper"> <EventOriginId>$Target/Id$</EventOriginId> <PublisherId>$MPElement$</PublisherId> <PublisherName>$Data/Context/DataItem/PublisherName$</PublisherName> <Channel>$Data/Context/DataItem/Channel$</Channel> <LoggingComputer>$Data/Context/DataItem/LoggingComputer$</LoggingComputer> <EventNumber>$Data/Context/DataItem/EventNumber$</EventNumber> <EventCategory>$Data/Context/DataItem/EventCategory$</EventCategory> <EventLevel>$Data/Context/DataItem/EventLevel$</EventLevel> <UserName /> <Description>$Data/Context/DataItem/EventDescription$</Description> <Params /> </ConditionDetection> </MemberModules> <Composition> <Node ID="EventMapper"> <Node ID="Consolidate" /> </Node> </Composition> </Composite> </ModuleImplementation> <OutputType>System!System.Event.Data</OutputType> <InputTypes> <InputType>Windows!Microsoft.Windows.EventData</InputType> </InputTypes> </ConditionDetectionModuleType>
If you haven’t ever looks at one of these, it is pretty complex at first. I will do my best to break this down:
First – the rule has a datasource (Event Provider) that looks for Event 999 with source of TEST. That’s pretty simple.
Next – the rule passes this event data to the custom condition detection.
The Condition Detection first loads the consolidator module – to consolidate matching event data in a specified time window. In this case I used 30 seconds. This means the consolidator will start a timer upon the first event, and wait until the 30 second window has passed to output the single consolidated dataitem.
Next – the Mapper module can be hard coded for output of each event parameter, or we can pass the individual stuff from the consolidated dataitem, which I have done here. And example would be getting the event ID from the Consolidator – and passing it to the mapper:
The result of this – is that no matter how many events are spammed in the log, I will only collect a SINGLE event every 30 seconds, greatly reducing the number of events in the SCOM databases. Realistically, we might use times closer to one event per hour maximum if this is a seriously flooding event.
Here is how the event looks as collected. For the most part it appears just like normal eventdata, except it also includes information like how many times it occurred, and details on the time window:
Thank you for this blog. I use it quite a lot.
I was wondering if we can do the same for WMI collected events? I assume I would need to change the tag, but i’m not sure to what and if any additional changes needed. please advise.
Updated Download Link: https://github.com/thekevinholman/EventThenScript
I posted the wrong link. here is the correct one: https://github.com/thekevinholman/CollectConsolidatedEvents
Pingback:Deciding 'Event Collection vs. Alert' rule - Kevin Justin's Blog