Reset Monitor using SCOM 2012 and Orchestrator – A must have Runbook

There is at least one Runbook each SCOM responsible must have. Are you wondering which? Well, I am sure you are always aware of what kind of alert you are closing. Is this alert beeing generated by a rule or by a monitor. If it is generated by a rule you just can close the alert and you don’t have to care about anything. But if the alert is being generated by a monitor you always facing the problem that if you just close the alert itself your monitor will often stay in a critical or warning state. A good practice to close alerts from monitors is to fix the problem itself in your IT infrastructure and then go to the health explorer and reset the monitor. After that the alert will also close.

A similar problem occurs in a more advanced situation. Let’s assume System Center Operations Manager is generating incident tickets in System Center Service Manager through the connector. After the ticket has been closed in Service Manager the alert in Operations Manager will also close. But what if the alert has been generated by a monitor? Well, Service Manager does not reset the monitors in Operations Manager.

For both situation you could build a Runbook in System Center Orchestrator Smiley.

It looks like that….simple huh?

image

O.k. let me explain. The first activity “Monitor Alert” is constantly monitoring the alerts for updates. If an alert is updating its status from “New” to “Closed” e.g. you close an alert using the SCOM console or Service Manager closes the alert. Each time the Runbook will start. The “Monitor Alert” activity will pass the ID from the alert to the“Reset Monitor” activity and then run/trigger the script activity. This script will reset the monitor which has generated the alert. Easy? Yes.

Imagine it does not matter if you are closing the alert or Service Manager will close the alert in both situations if the alert was generated by a monitor the Runbook will do its work.

The “Monitor Alert” activity looks like this…

image

…and the “Reset Monitor” activity and script which I slightly modified from here looks like this…

image

Script Code:

# Import Operations Manager Module and create Connection
Import-Module OperationsManager;
New-SCOMManagementGroupConnection scom01;
# Get alert
$alertid=”Put here published data ID from “Monitor Alert” acitivity]“;
$alerts = Get-SCOMAlert | where { $_.id -eq $alertid};
foreach ($alert in $alerts)
{
# Get IDs
$mrid = $alert.monitoringruleid;
$mcid = $alert.monitoringclassid;
$moid = $alert.monitoringobjectid;
# Get Objects
$monitor = Get-SCOMMonitor | where {$_.id -eq $mrid};
$monitoringclass = Get-SCOMClass | where {$_.id -eq $mcid};
$monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
# Reset Monitor
$monitoringobject | foreach{$_.ResetMonitoringState($monitor)};
};

Make sure you modify line 5 with the published data!

Now every time the monitor gets reset properly and you don’t have to care about it anymore.

Cool, SCOM and SCORCH better together!

About Stefan Roth

Consultant
This entry was posted in Orchestrator, Script. Bookmark the permalink.

44 Responses to Reset Monitor using SCOM 2012 and Orchestrator – A must have Runbook

  1. Kevin Greene says:

    Nice one Stefan – such a handy runbook :)

    Kevin.

    • scomfaq says:

      Thank’s Kevin, there is more stuff comming…

      Stay tuned :)

      Stefan

      • Kevin Greene says:

        Stefan,

        I’ve been playing around with Orchestrator over the last few days and went to implement your runbook above.

        When I either test the runbook or simply just run it, I get a failure on the ‘Run .Net Script’ action with an error alert like this:

        File C:\Program Files\System Center Operations Manager 2012\Powershell\OperationsManager\OperationsManager.psm1 cannot be loaded because the execution of scripts is disabled on this system. Please see “get-help about_signing” for more details.

        Now, I know you might be thinking that I haven’t set my execution policy to either ‘RemoteSigned’ or ‘Unrestricted’, but this was the first thing that I did on both the SCOM and SCOrch servers and when I run ‘get-executionpolicy’ on both servers, it comes back with ‘Unrestricted’. I’ve tried rebooting both servers and also recreating the runbook again.

        Any ideas for someone whose PoSh Kung Fu is pretty weak?

        Thanks!

      • scomfaq says:

        Hi Kevin,

        Cool, great hearing from you.

        Try to start the Powershell in “C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe” on your Orchestrator server and set there also Set-Executionpolicy unrestricted ;).

        Let me know if it worked.

        Regards,

        Stefan

  2. Kevin Greene says:

    Stefan, that sorted the problem out straight away – I should’ve known it was something as simple as that – was trying to debug the issue a little too deep when I hadn’t thought about the WOW64 PowerShell.exe instance :)

    I’ll know the next time I have this issue for sure now!

    Thank you so much for the rapid response and excellent Orchestrator runbooks – it’s making my ramp-up on it much easier!

    Kevin.

  3. Scorch Lover says:

    Stefan,
    Thanks for the great post as this will truly be helpful. HOwever, I’m having the following issue using SCOM 2012 and Orchestrator 2012. Using the following script;
    # Import Operations Manager Module and create Connection
    Import-Module OperationsManager;
    New-SCOMManagementGroupConnection ;
    # Get alert
    $alertid=”{Id from “Monitor Alert”}”;
    $alerts = Get-SCOMAlert | where { $_.id -eq $alertid};
    foreach ($alert in $alerts)
    {
    # Get IDs
    $mrid = $alert.monitoringruleid;
    $mcid = $alert.monitoringclassid;
    $moid = $alert.monitoringobjectid;
    # Get Objects
    $monitor = Get-SCOMMonitor | where {$_.id -eq $mrid};
    $monitoringclass = Get-SCOMClass | where {$_.id -eq $mcid};
    $monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
    # Reset Monitor
    $monitoringobject | foreach{$_.ResetMonitoringState($monitor)};
    };

    I get a SUCCESSFUL run when using the RUNBOOK tester, but when actually running the script I get the following error in my Orchestrator Log History;

    Run.NET script – FAILED
    Error Summary Text
    “The term ‘New-SCOMManagementGroupConnection’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.”

    Any idea why this is not working for me? Typing the same commands from my shell both x86 and x64 versions work fine???

    • Scorch Lover says:

      Update: also see this in event log.

      C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe
      2544

      Microsoft.EnterpriseManagement.Core.Cmdlets, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35

      System.DllNotFoundException: Unable to load DLL ‘MOMBIDldr.dll’: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
      at Bid.internalInitialize()

      • scomfaq says:

        Hi,

        I have seen this error before in other situations.

        Sometimes it helps to copy the ‘MOMBIDldr.dll’ to copy into the mentioned directory here C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe. But please test it before going into production.

        Regards,

        Stefan
        .

    • scomfaq says:

      Hi Scorch Lover :),

      You need to go to “C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe” and open Powershell.

      Set the executionpolicy to unrestricted by typing Set-ExecutionPolicy Unrestricted

      I think after that it should work.

      Regards,

      Stefan

      • Scorch Lover says:

        Thanks Stefan, I had already done that. What I just found out is that Orchestrator ‘requires’ a reboot after installing the OpsMgr shell/console even though it does not ‘require’ a reboot :) Guess it can’t load the .dll without the reboot. Just rebooted and it worked like a champ. Thanks again for the great article, look forward to seeing more in the future. Hope this helps someone else!

  4. Stan says:

    I get this warning “The requested name is valid, but no data of the requested type was found”. Anyone have any ideas?

  5. Andy says:

    Hi Stefan

    Hello i recive also the Warning/error above in the log history. I also recive the error Bad numeric constant: 80.
    any ideas?

    Andy

    • scomfaq says:

      Hi Andy

      Hmm, did you copy/paste my code? If so make sure all special characters e.g. -, etc. are what they suppose to be. Sometimes if you copy code from the web, it changes characters, and the code is unusable.

      Try to execute the code just in a Powershell command window, maybe you will see more what it is all about.

      Cheers,

      Stefan

      • Andy says:

        Hi Stefan
        Now it works. It seems that i have not replaced all the characters. Now I have it make with find/replace.

        Thanks for the Blog!
        Andreas

  6. gilad greenfeld says:

    Hi Stefan,
    Thanks for the great post as this will truly be helpful.

    My question is:
    After the monitor alert capture an alert the Runbook is finish.
    I want to continue monitoring new alert.
    What would be the best way to handle it?
    What happen if we got two alert at the same time (we have a big environment – around 600 servers) ?

    Thanks a lot for your great job.
    Gilad

    • scomfaq says:

      Hi Gilad

      Your welcome, glad I could help.

      The first activity of the runbook is already constantly runnig and “monitoring” for new alerts comming in. Unless you are stopping the runbook there is no need to do anything. The “Monitor Alert” activity is always “on”…

      Orchestrator will handle this for you, all new alerts will be processed separately you don’t have to be concernt about it. I am using this runbook also in a environment of ~600 servers.

      Conclusion: Just build it and test it, there is nothing that can go wrong….

      I hope this helps

      Stefan

  7. gilad greenfeld says:

    Hi Stefan,

    I’m using the Runbook and we see something strange.
    you may be able to help me with that.
    I see in the log that the trigger for the monitoring alert is activate every few second.
    i don’t now if there is a problem in SCOM or in ORCHESTRATOR but its cause us a very heavy traffic.
    i also see that we got the same alert every few second.
    i wrote the output to the log.

    This is example of the log:

    <1The monitor has been initialized for the first time or it has exited maintenance mode>
    <1The monitor has been initialized for the first time or it has exited maintenance mode>

    us you can see we got the same alert every ten second. (its only example of 2 alert but we got a lot of alerts)

    Do you know what is the problem?
    Thanks for your help,
    Gilad

  8. Pingback: Create a Orchestrator Runbook to reset SCOM objects to “healthy” « IT Hassle

  9. klaus says:

    Hi,
    Thanks for this useful runbook example! I implemented it yesterday but encountered a problem:
    Each time an alert is processed by the runbook (Powershell) and is resetted, the “Monitor Alert” discovers it again and again (as for SCORCH, it seems to be an updated alert). I’ve tried to add the filter “MonitoringObjectHealthState” does not equal “Healthy”, but it did not work as expected. Do you have any recommendations?
    By the way, I haven’t used the “Run .NET Activity” but the “Execute PS Script” from Codeplex IP, as I wanted to run it as a different user than the SCORCH service account.
    Thanks in advance!

    • scomfaq says:

      Hi Klaus,

      I guess I need to look at this problem (?) and get back to you.

      Cheers,

      Stefan

      • Matthias says:

        Hi!

        I managed to fix Klaus’ problem. I added an extra activity to update a customfield in the alert. This results in the parameter LastModifiedBy of the alert beeing edited to the account that SCO uses to connect to SCOM.

        If you then add “LastModifiedBy does not equal … (the same account)” to the Monitor Alert filter, SCO will discover the alert only once.

        This offcourse only counts if the alert doesn’t get closed by SCO in the first place (with that account).

        Regards,
        Matthias

      • scomfaq says:

        Hi Matthias

        Thanks for your comment.

        Well this could work or just write another kind of “mark” into the customfield and check this “mark” as an additional paramater on the Monitor Alert Parameter.

        Cheers,

        Stefan

  10. Pingback: Orchestrator 2012: Reset SCOM 2012 monitor for closed alert | SystemCenterTipps

  11. Pingback: bad OM Performance because State Change Events are not groomed out - Mihai Sarbulescu's System Center Blog - Site Home - TechNet Blogs

  12. Martin says:

    Hi Matthias, Stefan,

    can you post the lines you added to the script? About LastModifiedBy….. I run in the same problem.

    Regards,

    Martin

  13. Martin says:

    Hi Matthias, Stefan,

    I could configure the script like you said, i added to the alert a CustomField: Closed by Orchestrator
    The only thing I Need to figure out is that i have a lot of warnings with following value:

    Cannot validate argument on parameter ‘Class’. The argument is null or empty. Supply an argument that is not null or empty and then try the command again.

    $monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
    I guess the error is an empty value at $monitoringclass.

    Can you help me! Maybe we can Change the script to say if $Monitoringclass empty go to next….

    Regards,
    Martin

    • scomfaq says:

      Hi Martin

      Try to re-type the pipe character it might be a copy / paste error which causes the command to fail. If it doesn’t work, re-type the other characters.

      Cheers,

      Stefan

  14. Pingback: Excellent runbook for SCOM 2012 – In my head...

  15. Jeff Patton says:

    I ran into the problem that Scorch Lover encountered and copying the .dll file didn’t do anything for me. I wound up finding a solution in the Ops forums on Microsoft.

    http://social.technet.microsoft.com/Forums/systemcenter/en-US/8081ffdd-84ce-46de-ba52-7955bbc10cc0/powershell-write-action-module-using-om12-cmdlets#cc737110-aba2-4fe8-bdd5-6cb09552814f

    I imported the module using the full path to the .psm1

  16. Arne says:

    Hi,
    I ran into the following problem. I get an error: Invalid language type specified. Could someone help?

    Thanks,
    Arne

  17. Arne says:

    If I ran “C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe” I got this error. Now I changed the .Net Script back to PowerShell and the error is gone.

  18. klaus says:

    Arne,
    I had the same issue. The problem in my case is/was that the executed powershell ran in an environment where the OperationsManager module was not registered and I was therefore not able to do import-module OperationsManager. However, I created a powershell script which I put on all Management Servers (c:\users\public\documents\scripts\scomps.ps1) and which is “included” in all powershell command channels
    scomps.ps1:
    Import-Module “C:\Program Files\System Center 2012\Operations Manager\Powershell\OperationsManager\OperationsManager.psm1″
    . “C:\Program Files\System Center 2012\Operations Manager\Powershell\OperationsManager\Startup.ps1″
    . “C:\Program Files\System Center 2012\Operations Manager\Powershell\OperationsManager\Functions.ps1″

    and my command channel is as followed:

    Full path to cmd:
    C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

    CMD parameters (example where I set a special resolution state to grab it via Orchestrator):
    . c:\users\public\documents\scripts\SCOMPS.ps1; Get-SCOMAlert -Id (‘$Data/Context/DataItem/AlertId$’) | Set-SCOMAlert -ResolutionState 13;

    Startup folder:
    C:\Windows\System32\WindowsPowerShell\v1.0\

  19. Scott says:

    Has anyone seen the issue for the line
    $monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid}
    that results in high memory consumption and never completes when the object is an interface on a network device? I see the following event id 26373 on the SCOM management server:
    The query resulted in 230773 rows.
    This may cause high memory consumption, and affect server performance.
    Consider using buffered mode for this query.

    • _skold says:

      We see this aswell. We recieve alot of Event 26373, OpsMgr SDK Service
      ___
      The query resulted in 182678 rows.
      This may cause high memory consumption, and affect server performance.
      Consider using buffered mode for this query.
      TypeId: Sytem.Entity
      Criteria:
      ___
      Any thoughts?

      • Scott says:

        I resolved it by using this modification for the monitoringobject variable
        $monitoringobject = Get-SCOMMonitoringobject -id $moid

  20. Michiel Aubertijn says:

    Stefan,

    This is an excellent runbook!
    I have added a little thing in our situation. This runbook reset also monitors for alerts that were closed by the system. We don’t need to reset this ones as well so I added to the scomalerts def:
    | where {$_.resolvedby -ne “System”}

  21. Pingback: SCOM 2012 – MAS Tool V 1.0 | stefanroth.net

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s