Menu Close

Monitoring SQL Agent Jobs in SCOM

This article covers SQL Agent job monitoring in the current SQL Version Agnostic Management pack, version 7.0.20.0 and later.

 

I will cover three conceptual areas:

  • Out of the box experience and defaults
  • SQL Agent only monitoring (monitoring all jobs at the agent level)
  • SQL Agent Job monitoring (optionally discovering and monitoring agent jobs on an individual level)

 

Out of the box, we don’t discover or monitor individual SQL agent jobs by default.  What we do is discover the SQL Agent object:

 

image

 

Out of the box, we monitor for Service Running status, Last Job Run Status, and Job Duration:

 

image

 

Last Run Status – checks all jobs on the SQL Agent every 10 minutes, and if ANY job didn’t complete successfully this monitor will stay in a warning state.  This does not generate an alert because there is an override to disable alerts to control noise.  If you want this level of monitoring, you need to override Alerting back to enabled.

Job Duration – checks all jobs on the SQL Agent every 10 minutes, and if ANY job takes longer than 60 minutes it will set a warning state, or 120 minutes for a critical state.  Alerting is also disabled via override.  If you want this level of monitoring, you need to override Alerting back to enabled. 

 

There are also several rules that target the “SQL Agent” object, that primarily look in the Application event log for issues/errors related to the SQL Agent:

 

image

 

Notice that the “A SQL job failed to complete successfully” is disabled out of the box.   This was done to reduce out of the box noise…. as many customers have terribly monitored/maintained SQL environments and have so many jobs failing that this was almost un-actionable.  If you want to be alerted to ANY SQL Agent job failure of any kind – then you might consider enabling this rule via override.  I would recommend this if you aren’t going to discover and monitor individual SQL agent jobs (covered later in this article).

 

Ok – that covers the out of the box defaults, and SQL Agent level monitoring.  Now, what if I need to go deeper, and have the ability to override settings for individual SQL Agent jobs?  You will notice that in the view for SQL Agent Job State – there are no objects:

 

image

 

This is by design, because we don’t discover SQL Agent jobs individually out of the box.  This is because customers have potentially tens of thousands of individual SQL Agent jobs, and discovering these is optional because it can increase load on the monitored agent, and increase instance space and monitoring load in SCOM.

That said, there are scenarios where you might wish to enable this.  Perhaps you have SQL Agent jobs that you don’t want to monitor at all, or others that have especially long run-times that need unique thresholds (like SQL backups or maintenance on VERY large DB’s)  In this case – we will want to enable the discovery to discover SQL Agent jobs as discovered objects in SCOM.

To enable the object discovery, go to Authoring Pane, select Object Discoveries.  In the upper right corner, select Change Scope, then View All, Clear All.  Search for “Agent Job”, and select “MSSQL on Windows:Agent Job” 

 

image

 

This will show you the discovery we are looking for:  “MSSQL on Windows: Discover SQL Server Agent Jobs

 

image

 

Notice this is disabled by default.  Create an override, “For all objects of class: MSSQL on Windows: Agent” and set Enabled = True.  Save these overrides to your custom “Overrides – SQL” management pack.

 

image

 

This discovery runs every 4 hours by default, so within 4 hours you should see your SQL Agent job state view populated with your individual SQL Agent jobs.  However, you will likely see agent jobs showing up within a few minutes after making the change, if it was never enabled before.  (**Hint – to speed this process up in a lab scenario, bounce the Microsoft Monitoring Agent service on your agents – as this will force a discovery to run on service startup)

 

image

 

Now – lets talk about what we monitor by default for discovered SQL Agent jobs.  There are a total of two monitors.  Last Run Status, and Job Duration:

 

image

 

Last Run Status – monitor checks every 10 minutes, and will remain Yellow (Warning state) for any job whose last run was not a success.  This gives you a nice “real time” view of the unhealthy jobs that need some attention.

Job Duration – monitor runs every 10 minutes, and looks for jobs that have exceeded a run-time threshold.  The default is 60 minutes (warning) and 120 minutes (critical). 

Note*** – On BOTH of the above monitors, alerting is disabled by default to control noise.  The out of the box configuration is to show state only.  You must override each monitor if you want alerts.

In the following example – I am creating an override for my SQL agent job that performs Index Operations, setting the thresholds higher and enabling alerting….. and saving it to my “Overrides – SQL” management pack:

 

image

 

As you can see, this gives you SQL Agent job by job granularity.

You could easily also create groups of SQL agent jobs based on group name…. if you wanted to treat all jobs with the same or similar names the same from an override perspective.  This will additionally ease the burden of maintaining overrides using a dynamic group based on criteria of the job:

 

image

 

See my other SQL MP How-To posts:

https://kevinholman.com/2019/06/12/how-to-transition-to-the-sql-version-agnostic-mp/

https://kevinholman.com/2016/08/25/sql-mp-run-as-accounts-no-longer-required/

https://kevinholman.com/2020/01/31/how-to-exclude-sql-express-edition-from-scom-monitoring/

5 Comments

  1. Orchyl salmasan

    Hi Kevin,
    Regarding “Job Duration” monitor, Is there any way to append the actual job duration of the job on the alert description?

    • Kevin Holman

      That would not be possible, because most of the time, the job is still running. This monitor triggers when any individual job is running over the threshold. It doesn’t wait until the job complete, so it cannot give you an actual duration.

  2. Venkatesh

    Hello Kevin,

    We have the SQL MP version 7.015.0 in our environment. Recently we had a request from 1 of our client asking to monitor their SQL agent jobs. This particular SQL box is a 2019 one. Is there any option to monitor the SQL agent job of a SQL 2019 agent with the existing MP. I did override the object discovery MSSQL on Windows: Discover SQL Server Agent Jobs” for this specific object which failed too. Would like to know other possibilities apart from installing the Agnostic pack.

    • Kevin Holman

      7.0.15.0 is the SQL Version Agnostic MP.

      However, the first version to support SQL 2019 is 7.0.20.0 and this is documented in the MP Guide. You need to keep your MP’s up to date to monitor the latest versions of anything.

Leave a Reply

Your email address will not be published. Required fields are marked *