Menu Close

What account will command channel notifications Run As in SCOM?

I get this question several times a year.  Let’s find out for sure.

Create a new Command Channel.

image

For the configuration settings, I will use the following, as this will allow me to write all common alert fields to a logfile:

image

Full Path of the command file:

%SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe

Command line parameters

-executionPolicy bypass -noprofile -file "C:\scripts\logalert.ps1" -AlertID "$Data[Default='NotPresent']/Context/DataItem/AlertId$" -AlertName "$Data[Default='NotPresent']/Context/DataItem/AlertName$" -AlertDescription "$Data[Default='NotPresent']/Context/DataItem/AlertDescription$" -Severity "$Data[Default='NotPresent']/Context/DataItem/Severity$" -ResolutionState "$Data[Default='NotPresent']/Context/DataItem/ResolutionState$" -ManagedEntityPath "$Data[Default='NotPresent']/Context/DataItem/ManagedEntityPath$" -ManagedEntityDisplayName "$Data[Default='NotPresent']/Context/DataItem/ManagedEntityDisplayName$" -ManagedEntityFullName "$Data[Default='NotPresent']/Context/DataItem/ManagedEntityFullName$" -ManagedEntity "$Data[Default='NotPresent']/Context/DataItem/ManagedEntity$" -CreatedByMonitor "$Data[Default='NotPresent']/Context/DataItem/CreatedByMonitor$" -WorkflowId "$Data[Default='NotPresent']/Context/DataItem/WorkflowId$" -DataItemCreateTimeLocal "$Data[Default='NotPresent']/Context/DataItem/DataItemCreateTimeLocal$" -WebConsoleUrl "$Target/Property[Type="Notification!Microsoft.SystemCenter.AlertNotificationSubscriptionServer"]/WebConsoleUrl$?DisplayMode=Pivot&AlertID=$UrlEncodeData/Context/DataItem/AlertId$"

Startup folder for the command line:

C:\scripts

 

In C:\scripts I place my script file “logalert.ps1”:

image

 

Logalert.ps1 contents:

param($AlertID, $AlertName, $AlertDescription, $Severity, $ResolutionState, $ManagedEntityPath, $ManagedEntityDisplayName, $ManagedEntityFullName, $ManagedEntity, $CreatedByMonitor, $WorkflowId, $DataItemCreateTimeLocal, $WebConsoleUrl) $ScriptName = "logalert.ps1" $EventID = "1234" $whoami = whoami $momapi = New-Object -comObject MOM.ScriptAPI #Log script event that we are starting task $momapi.LogScriptEvent($ScriptName,$EventID,0,"`nScript is starting. `nRunning as ($whoami). `nAlertID: ($AlertID) `nAlertName: ($AlertName) `nAlertDescription:($AlertDescription) `nSeverity:($Severity) `nResolutionState: ($ResolutionState) `nManagedEntityPath: ($ManagedEntityPath) `nManagedEntityDisplayName: ($ManagedEntityDisplayName) `nManagedEntityFullName: ($ManagedEntityFullName) `nManagedEntity: ($ManagedEntity) `nCreatedByMonitor: ($CreatedByMonitor) `nWorkFlowId: ($WorkflowId) `nDataItemCreateTimeLocal: ($DataItemCreateTimeLocal) `nWebConsoleURL: ($WebConsoleUrl)")

 

Now lets set up a subscriber for command channel alerts, and a subscription:

Create a new subscriber and call it “Command Channel Subscriber”

image

 

Add a new subscriber address, like so:

image

image

image

image

 

Next, add the subscriber to a subscription.  I will subscribe to my Test Alerts from my SCOM.Management pack:

image

image

 

Add the Command Channel Subscriber to the subscription:

image

 

Add the proper Channels:

image

 

And finish:

image

 

Next – lets review our resource pools. 

For testing, I like to change the Notifications pool to manual, and just put a single server in there temporarily.  This will ensure I know which server is running ALL notification.

image

 

Then I will trigger the Create Test Event task from the SCOM.Management pack:

image

 

Which will trigger a test Alert:

image

 

Now open the Operations Manager event log on the Management server in our resource pool, and we should see a 1234 Event, which was triggered by the command channel script.

image

 

I can see – that by default – the account used for Notifications is the same as the Management Server Action Account for this management server.

Now – lets configure the Notification RunAs account:

Create a new RunAs account for Notifications (this needs to be a real account in Active Directory):

image

image

 

Distribute this account to the Notifications Resource Pool:

image

 

Next, we must ASSOCIATE the RunAs account to the Notifications Account Profile.  Browse to the Profiles, and find the Notifications Account Profile:

image

 

Associate your Notifications RunAs account PROFILE with your newly created RunAs account, for all targeted objects:

image

 

Once this association is made, you will see a new MonitoringHost.exe process spun up on the Management server, under your new account:

image

 

All notifications will now execute as this account.  You might need to grant this account privileges on your management server for it to be able to Log On Locally, or Log On As A Service, depending on your SCOM version.  You can review the management server event log for errors which will show up if there is a configuration issue.

 

Now – go back and trigger a new alert

image

 

And check the log:

image

 

We can easily see the Notification Command Channel runs as our new Notification RunAs account.

 

Now lets review the XML to understand how or why this works:

When you create a Notification Subscription, you are actually creating a RULE in the unsealed Microsoft.SystemCenter.Notifications.Internal management pack.

Here is an example:

<Rule ID="Subscription8568d5a5_a6d3_4f45_a8fd_14d42eebe276" Enabled="true" Target="Notification!Microsoft.SystemCenter.AlertNotificationSubscriptionServer" ConfirmDelivery="false" Remotable="true" Priority="Normal" DiscardLevel="100"> <Category>Notification</Category> <DataSources> <DataSource ID="DS1" RunAs="SystemCenter!Microsoft.SystemCenter.DatabaseWriteActionAccount" TypeID="SystemCenter!Microsoft.SystemCenter.SubscribedAlertProvider"> <AlertChangedSubscription Property="Any"> <Criteria> <Expression> <SimpleExpression xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <ValueExpression> <Property>RuleId</Property> </ValueExpression> <Operator>Equal</Operator> <ValueExpression> <Value>8060f053-dc84-330d-c7d0-3c659f9b7ca7</Value> </ValueExpression> </SimpleExpression> </Expression> </Criteria> <ExpirationStartTime>03/05/2021 16:25:37</ExpirationStartTime> <PollingIntervalMinutes>1</PollingIntervalMinutes> <UserSid>S-1-5-21-3626055071-1639654894-2113106914-1104</UserSid> <LanguageCode>ENU</LanguageCode> <ExcludeNonNullConnectorIds>false</ExcludeNonNullConnectorIds> <RuleId>$MPElement$</RuleId> <TargetBaseManagedEntityId>$Target/Id$</TargetBaseManagedEntityId> <TimeZone>6801000000000000C4FFFFFF00000B0000000100020000000000000000000300000002000200000000000000|Central Standard Time</TimeZone> </AlertChangedSubscription> </DataSource> </DataSources> <ConditionDetection ID="CD1" TypeID="Microsoft.SystemCenter.Notification.Recipients"> <Recipients> <To> <DirectoryReference> <RecipientId>646e126c-c96e-479f-b555-b9df1386ab59</RecipientId> </DirectoryReference> <DirectoryReference> <RecipientId>187dfeec-6197-4ca5-9272-d6b0aa2c23f3</RecipientId> </DirectoryReference> </To> </Recipients> </ConditionDetection> <WriteActions> <WriteAction ID="WA0" TypeID="Cmd_b8d0e358_b9b8_468b_b7de_38ba028a6c7b" /> <WriteAction ID="WA1" TypeID="SMTPAction_6cb98307_eb31_4f10_b35a_31f1e7648807" /> </WriteActions> </Rule>

If you scroll down to the Write Actions, you can see our “Cmd_” write action which is the command channel.  Look further up in the XML of the MP and you can find this write action definition.  This Write Action is our CHANNEL.

<WriteActionModuleType ID="Cmd_b8d0e358_b9b8_468b_b7de_38ba028a6c7b" Accessibility="Public" Batching="false"> <Configuration /> <ModuleImplementation Isolation="Any"> <Composite> <MemberModules> <WriteAction ID="NotificationAction" TypeID="Notification!Microsoft.SystemCenter.Notification.CommandAction"> <RecipientProtocol>Cmd_b8d0e358_b9b8_468b_b7de_38ba028a6c7b</RecipientProtocol> <ApplicationName>%SystemRoot%\system32\WindowsPowerShell\v1.0\powershell.exe</ApplicationName> <WorkingDirectory>c:\scripts</WorkingDirectory> <CommandLine>-executionPolicy bypass -noprofile -file "C:\scripts\logalert.ps1" -AlertID "$Data[Default='NotPresent']/Context/DataItem/AlertId$" -AlertName "$Data[Default='NotPresent']/Context/DataItem/AlertName$" -AlertDescription "$Data[Default='NotPresent']/Context/DataItem/AlertDescription$" -Severity "$Data[Default='NotPresent']/Context/DataItem/Severity$" -ResolutionState "$Data[Default='NotPresent']/Context/DataItem/ResolutionState$" -ManagedEntityPath "$Data[Default='NotPresent']/Context/DataItem/ManagedEntityPath$" -ManagedEntityDisplayName "$Data[Default='NotPresent']/Context/DataItem/ManagedEntityDisplayName$" -ManagedEntityFullName "$Data[Default='NotPresent']/Context/DataItem/ManagedEntityFullName$" -ManagedEntity "$Data[Default='NotPresent']/Context/DataItem/ManagedEntity$" -CreatedByMonitor "$Data[Default='NotPresent']/Context/DataItem/CreatedByMonitor$" -WorkflowId "$Data[Default='NotPresent']/Context/DataItem/WorkflowId$" -DataItemCreateTimeLocal "$Data[Default='NotPresent']/Context/DataItem/DataItemCreateTimeLocal$" -WebConsoleUrl "$Target/Property[Type="Notification!Microsoft.SystemCenter.AlertNotificationSubscriptionServer"]/WebConsoleUrl$?DisplayMode=Pivot&amp;AlertID=$UrlEncodeData/Context/DataItem/AlertId$"</CommandLine> <TimeoutSeconds>2147483647</TimeoutSeconds> <RequireOutput>false</RequireOutput> </WriteAction> </MemberModules> <Composition> <Node ID="NotificationAction" /> </Composition> </Composite> </ModuleImplementation> <InputType>Notification!Microsoft.SystemCenter.Notification.RecipientsData</InputType> </WriteActionModuleType>

 

This CHANNEL uses the Microsoft.SystemCenter.Notification.CommandAction write action from the Microsoft.SystemCenter.Notifications.Library, as seen in the XML above.

If we review that Microsoft.SystemCenter.Notification.CommandAction write action XML, we can see the RUNAS association where this workflow WILL use the Notifications RunAs account profile:

<WriteActionModuleType ID="Microsoft.SystemCenter.Notification.CommandAction" Accessibility="Public" RunAs="Microsoft.SystemCenter.Notification.NotificationActionAccount" Batching="false"> <Configuration> <IncludeSchemaTypes> <SchemaType>System!System.CommandExecuterSchema</SchemaType> </IncludeSchemaTypes> <xsd:element name="RecipientProtocol" type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema" /> <xsd:element name="ApplicationName" type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema" /> <xsd:element name="WorkingDirectory" type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema" /> <xsd:element name="CommandLine" type="xsd:string" xmlns:xsd="http://www.w3.org/2001/XMLSchema" /> <xsd:element name="TimeoutSeconds" type="xsd:integer" xmlns:xsd="http://www.w3.org/2001/XMLSchema" /> <xsd:element name="RequireOutput" type="xsd:boolean" xmlns:xsd="http://www.w3.org/2001/XMLSchema" /> </Configuration> <ModuleImplementation Isolation="Any"> <Composite> <MemberModules> <ConditionDetection ID="RecipientFilter" TypeID="Microsoft.SystemCenter.Notification.RecipientFilter"> <Protocol>$Config/RecipientProtocol$</Protocol> <Multicast>false</Multicast> </ConditionDetection> <WriteAction ID="Endpoint" TypeID="System!System.CommandExecuter"> <ApplicationName>$Config/ApplicationName$</ApplicationName> <WorkingDirectory>$Config/WorkingDirectory$</WorkingDirectory> <CommandLine>$Config/CommandLine$</CommandLine> <TimeoutSeconds>$Config/TimeoutSeconds$</TimeoutSeconds> <RequireOutput>$Config/RequireOutput$</RequireOutput> </WriteAction> </MemberModules> <Composition> <Node ID="Endpoint"> <Node ID="RecipientFilter" /> </Node> </Composition> </Composite> </ModuleImplementation> <InputType>Microsoft.SystemCenter.Notification.RecipientsData</InputType> </WriteActionModuleType>

Specifically this part, on the first line:

image

 

That is what tells this workflow to load as the account associated to that RunAs profile, if one is associated.

 

Lastly, remember, if you are going to use Command Channel Notifications (which I generally do not recommend)…. these have a scaling issue.  By default, SCOM will only allow 5 simultaneous responses (5 alerts with the command channel subscribing to them) in a single run, and will drop the rest.  If this happens, you might see the following event:

Log Name:      Operations Manager
Source:        Health Service Modules
Date:          8/25/2021 7:48:16 AM
Event ID:      21410
Task Category: None
Level:         Warning
Keywords:      Classic
User:          N/A
Computer:      SCOM1.opsmgr.net
Description:
The process could not be created because the maximum number of asynchronous responses (5) has already been reached, and it will be dropped.
 
Command executed:    “C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe” -executionPolicy bypass -noprofile -file “C:\scripts\logalert.ps1” -AlertID “{addd5be7-b31c-4a0c-911d-cfc3d7bd2a5f}” -AlertName “Test Alert on Test Event 100” -AlertDescription “This is a test alert fired by event ID 100:  \n Event Description:  \n This is a Test event 100” -Severity “0” -ResolutionState “0” -ManagedEntityPath “SCORCH16.opsmgr.net” -ManagedEntityDisplayName “SCORCH16.opsmgr.net” -ManagedEntityFullName “SCOM.Management.Class:SCORCH16.opsmgr.net” -ManagedEntity “{efeeade0-eb50-e531-96f8-2887251c429e}” -CreatedByMonitor “false” -WorkflowId “{8060f053-dc84-330d-c7d0-3c659f9b7ca7}” -DataItemCreateTimeLocal “8/25/2021 7:48:16 AM” -WebConsoleUrl “http://SCOM1/OperationsManager?DisplayMode=Pivot&AlertID=$UrlEncodeData/Context/DataItem/AlertId$”
Working Directory:    c:\scripts

One or more workflows were affected by this. 

Workflow name: Subscription8568d5a5_a6d3_4f45_a8fd_14d42eebe276
Instance name: Alert Notification Subscription Server
Instance ID: {E07E3FAB-53BC-BC14-1634-5A6E949F9230}
Management group: SCOM

 

And an alert will be generated:

image

 

 

To work around this you need to set a Registry key/value to allow for the maximum number of asynchronous responses to alerts,which can be set as high as 100:

 

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Modules\Global\Command Executer\AsyncProcessLimit

Create a new REG_DWORD value.  Valid options are from 5 to a maximum of 100.

image

 

This will allow 100 scripts (or command channel responses) to execute simultaneously.  Take care that 100 of these will not harm the resources of your management server, and understand that if more than 100 alerts arrive (in a single minute) that need this response run, any over the first 100 will be dropped.  They will not queue and will never have the command channel run.  It is important to understand this limitation of using the command channel, especially for something like a ticketing system integration.

9 Comments

    • Kevin Holman

      I don’t know any way to do that. We generally recommend moving to Orchestrator or use the product connector framework at that point, not command channels.

  1. Dimitri

    I have absolutely no way to make it work. I checked all forums from MS and others. The script simply is not starting.
    No matter what I put in command line, nothing runs.
    On some MS forum they say the startup folder for the command line must be the path for powershell.exe
    In some other they say it’s the path of the script to be run.
    I copied the configuration you suggested, but still nothing happens. Is there a way to debug or at least to trace what happens ? I cannot see anything also in event log.

    Thanks

  2. Kailasnath Aanchery

    Hi Kevin

    We have new installation of SCOM 2022. I have created a command subscription but the command (a powershell script) is not getting triggered when the alert is logged(thal service heartbeat failure). To test, I created an email subscription for the same alert and that works. The powershell execution policy is proper too. I am able to manually run the script but the scom subsription would not trigger it.

  3. Dimitrov

    Hi Kevin,

    The service is not starting with the run as account user.

    i always get the error: Microsoft.EnterpriseManagement.HealthService.Modules.Notification.SmtpNotificationException: The SMTP server requires a secure connection or the client was not authenticated. The server response was: 5.7.57 SMTP; Client was not authenticated to send anonymous mail during MAIL FROM. Smtp status code ‘MustIssueStartTlsFirst’.

    How can i fix this?

  4. Marius Ene

    When using the command channel , the Web Console link is not working, in the sense that SCOM variables for $UrlEncodeData/Context/DataItem/AlertId$ does not get resolved so it redirects you to the actual alert. The URL looks like this:
    “https://scomserver.domain.com/OperationsManager/#/monitoring/drilldown/alert/$UrlEncodeData/Context/DataItem/AlertId$”. Is there any fix for this?

  5. Fursel

    Anyone of you tried to use Azure CLI with Azure Communication Service to send SMS from SCOM.
    https://learn.microsoft.com/en-us/cli/azure/communication/sms?view=azure-cli-latest#az-communication-sms-send

    We have issue with powershell script when running it via triggered scom alert as it is hanging on the az coomunication sms send and not doing anything, hangs also powershell process. When running script with Action Account directly from powershell it runs without an issue and sms is send, same when running it as task from task scheduler with same action account which is set for notification sms is send.

    Maybe there is some limitation with scom running command as powershell which can be bypassed ? I noticed that $env:HOMEPATH doesnt return anything with scom running powershell as it does with normal powershell.

  6. Pingback:Create ServiceNow incidents from SCOM - Kevin Justin's Blog

Leave a Reply

Your email address will not be published.