There is at least one Runbook each SCOM responsible must have. Are you wondering which? Well, I am sure you are always aware of what kind of alert you are closing. Is this alert beeing generated by a rule or by a monitor. If it is generated by a rule you just can close the alert and you don’t have to care about anything. But if the alert is being generated by a monitor you always facing the problem that if you just close the alert itself your monitor will often stay in a critical or warning state. A good practice to close alerts from monitors is to fix the problem itself in your IT infrastructure and then go to the health explorer and reset the monitor. After that the alert will also close.
A similar problem occurs in a more advanced situation. Let’s assume System Center Operations Manager is generating incident tickets in System Center Service Manager through the connector. After the ticket has been closed in Service Manager the alert in Operations Manager will also close. But what if the alert has been generated by a monitor? Well, Service Manager does not reset the monitors in Operations Manager.
For both situation you could build a Runbook in System Center Orchestrator 
.
It looks like that….simple huh?
O.k. let me explain. The first activity “Monitor Alert” is constantly monitoring the alerts for updates. If an alert is updating its status from “New” to “Closed” e.g. you close an alert using the SCOM console or Service Manager closes the alert. Each time the Runbook will start. The “Monitor Alert” activity will pass the ID from the alert to the“Reset Monitor” activity and then run/trigger the script activity. This script will reset the monitor which has generated the alert. Easy? Yes.
Imagine it does not matter if you are closing the alert or Service Manager will close the alert in both situations if the alert was generated by a monitor the Runbook will do its work.
The “Monitor Alert” activity looks like this…
…and the “Reset Monitor” activity and script which I slightly modified from here looks like this…
Script Code:
# Import Operations Manager Module and create Connection
Import-Module OperationsManager;
New-SCOMManagementGroupConnection scom01;
# Get alert
$alertid=”Put here published data ID from “Monitor Alert” acitivity]“;
$alerts = Get-SCOMAlert | where { $_.id -eq $alertid};
foreach ($alert in $alerts)
{
# Get IDs
$mrid = $alert.monitoringruleid;
$mcid = $alert.monitoringclassid;
$moid = $alert.monitoringobjectid;
# Get Objects
$monitor = Get-SCOMMonitor | where {$_.id -eq $mrid};
$monitoringclass = Get-SCOMClass | where {$_.id -eq $mcid};
$monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
# Reset Monitor
$monitoringobject | foreach{$_.ResetMonitoringState($monitor)};
};
Make sure you modify line 5 with the published data!
Now every time the monitor gets reset properly and you don’t have to care about it anymore.
Cool, SCOM and SCORCH better together!
Nice!
Nice one Stefan – such a handy runbook 🙂
Kevin.
Thank’s Kevin, there is more stuff comming…
Stay tuned 🙂
Stefan
Stefan,
I’ve been playing around with Orchestrator over the last few days and went to implement your runbook above.
When I either test the runbook or simply just run it, I get a failure on the ‘Run .Net Script’ action with an error alert like this:
File C:Program FilesSystem Center Operations Manager 2012PowershellOperationsManagerOperationsManager.psm1 cannot be loaded because the execution of scripts is disabled on this system. Please see “get-help about_signing” for more details.
Now, I know you might be thinking that I haven’t set my execution policy to either ‘RemoteSigned’ or ‘Unrestricted’, but this was the first thing that I did on both the SCOM and SCOrch servers and when I run ‘get-executionpolicy’ on both servers, it comes back with ‘Unrestricted’. I’ve tried rebooting both servers and also recreating the runbook again.
Any ideas for someone whose PoSh Kung Fu is pretty weak?
Thanks!
Hi Kevin,
Cool, great hearing from you.
Try to start the Powershell in “C:WindowsSysWOW64WindowsPowerShellv1.0powershell.exe” on your Orchestrator server and set there also Set-Executionpolicy unrestricted ;).
Let me know if it worked.
Regards,
Stefan
Stefan, that sorted the problem out straight away – I should’ve known it was something as simple as that – was trying to debug the issue a little too deep when I hadn’t thought about the WOW64 PowerShell.exe instance 🙂
I’ll know the next time I have this issue for sure now!
Thank you so much for the rapid response and excellent Orchestrator runbooks – it’s making my ramp-up on it much easier!
Kevin.
Kevin,
Great, that I could help.
Enjoy!
Stefan
Stefan,
Thanks for the great post as this will truly be helpful. HOwever, I’m having the following issue using SCOM 2012 and Orchestrator 2012. Using the following script;
# Import Operations Manager Module and create Connection
Import-Module OperationsManager;
New-SCOMManagementGroupConnection ;
# Get alert
$alertid=”{Id from “Monitor Alert”}”;
$alerts = Get-SCOMAlert | where { $_.id -eq $alertid};
foreach ($alert in $alerts)
{
# Get IDs
$mrid = $alert.monitoringruleid;
$mcid = $alert.monitoringclassid;
$moid = $alert.monitoringobjectid;
# Get Objects
$monitor = Get-SCOMMonitor | where {$_.id -eq $mrid};
$monitoringclass = Get-SCOMClass | where {$_.id -eq $mcid};
$monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
# Reset Monitor
$monitoringobject | foreach{$_.ResetMonitoringState($monitor)};
};
I get a SUCCESSFUL run when using the RUNBOOK tester, but when actually running the script I get the following error in my Orchestrator Log History;
Run.NET script – FAILED
Error Summary Text
“The term ‘New-SCOMManagementGroupConnection’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.”
Any idea why this is not working for me? Typing the same commands from my shell both x86 and x64 versions work fine???
Update: also see this in event log.
C:Windowssystem32WindowsPowerShellv1.0powershell.exe
2544
Microsoft.EnterpriseManagement.Core.Cmdlets, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
System.DllNotFoundException: Unable to load DLL ‘MOMBIDldr.dll’: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
at Bid.internalInitialize()
Hi,
I have seen this error before in other situations.
Sometimes it helps to copy the ‘MOMBIDldr.dll’ to copy into the mentioned directory here C:Windowssystem32WindowsPowerShellv1.0powershell.exe. But please test it before going into production.
Regards,
Stefan
.
Hi Scorch Lover :),
You need to go to “C:WindowsSysWOW64WindowsPowerShellv1.0powershell.exe” and open Powershell.
Set the executionpolicy to unrestricted by typing Set-ExecutionPolicy Unrestricted
I think after that it should work.
Regards,
Stefan
Thanks Stefan, I had already done that. What I just found out is that Orchestrator ‘requires’ a reboot after installing the OpsMgr shell/console even though it does not ‘require’ a reboot 🙂 Guess it can’t load the .dll without the reboot. Just rebooted and it worked like a champ. Thanks again for the great article, look forward to seeing more in the future. Hope this helps someone else!
I get this warning “The requested name is valid, but no data of the requested type was found”. Anyone have any ideas?
Hi Stan
Could you please explain where and how you get this warning.
Thanks,
Stefan
You have to replace scom01 in the line “New-SCOMManagementGroupConnection scom01;” with the name of your own Mgmt-Server.
Hi Stefan
Hello i recive also the Warning/error above in the log history. I also recive the error Bad numeric constant: 80.
any ideas?
Andy
Hi Andy
Hmm, did you copy/paste my code? If so make sure all special characters e.g. -, etc. are what they suppose to be. Sometimes if you copy code from the web, it changes characters, and the code is unusable.
Try to execute the code just in a Powershell command window, maybe you will see more what it is all about.
Cheers,
Stefan
Hi Stefan
Now it works. It seems that i have not replaced all the characters. Now I have it make with find/replace.
Thanks for the Blog!
Andreas
Hi Stefan,
Thanks for the great post as this will truly be helpful.
My question is:
After the monitor alert capture an alert the Runbook is finish.
I want to continue monitoring new alert.
What would be the best way to handle it?
What happen if we got two alert at the same time (we have a big environment – around 600 servers) ?
Thanks a lot for your great job.
Gilad
Hi Gilad
Your welcome, glad I could help.
The first activity of the runbook is already constantly runnig and “monitoring” for new alerts comming in. Unless you are stopping the runbook there is no need to do anything. The “Monitor Alert” activity is always “on”…
Orchestrator will handle this for you, all new alerts will be processed separately you don’t have to be concernt about it. I am using this runbook also in a environment of ~600 servers.
Conclusion: Just build it and test it, there is nothing that can go wrong….
I hope this helps
Stefan
I got it.
Thanks for your help.
Its working and running 🙂
Gilad
Hi Stefan,
I’m using the Runbook and we see something strange.
you may be able to help me with that.
I see in the log that the trigger for the monitoring alert is activate every few second.
i don’t now if there is a problem in SCOM or in ORCHESTRATOR but its cause us a very heavy traffic.
i also see that we got the same alert every few second.
i wrote the output to the log.
This is example of the log:
<1The monitor has been initialized for the first time or it has exited maintenance mode>
<1The monitor has been initialized for the first time or it has exited maintenance mode>
us you can see we got the same alert every ten second. (its only example of 2 alert but we got a lot of alerts)
Do you know what is the problem?
Thanks for your help,
Gilad
Hi,
Thanks for this useful runbook example! I implemented it yesterday but encountered a problem:
Each time an alert is processed by the runbook (Powershell) and is resetted, the “Monitor Alert” discovers it again and again (as for SCORCH, it seems to be an updated alert). I’ve tried to add the filter “MonitoringObjectHealthState” does not equal “Healthy”, but it did not work as expected. Do you have any recommendations?
By the way, I haven’t used the “Run .NET Activity” but the “Execute PS Script” from Codeplex IP, as I wanted to run it as a different user than the SCORCH service account.
Thanks in advance!
Hi Klaus,
I guess I need to look at this problem (?) and get back to you.
Cheers,
Stefan
Hi!
I managed to fix Klaus’ problem. I added an extra activity to update a customfield in the alert. This results in the parameter LastModifiedBy of the alert beeing edited to the account that SCO uses to connect to SCOM.
If you then add “LastModifiedBy does not equal … (the same account)” to the Monitor Alert filter, SCO will discover the alert only once.
This offcourse only counts if the alert doesn’t get closed by SCO in the first place (with that account).
Regards,
Matthias
Hi Matthias
Thanks for your comment.
Well this could work or just write another kind of “mark” into the customfield and check this “mark” as an additional paramater on the Monitor Alert Parameter.
Cheers,
Stefan
Hi Matthias, Stefan,
can you post the lines you added to the script? About LastModifiedBy….. I run in the same problem.
Regards,
Martin
Hi Matthias, Stefan,
I could configure the script like you said, i added to the alert a CustomField: Closed by Orchestrator
The only thing I Need to figure out is that i have a lot of warnings with following value:
Cannot validate argument on parameter ‘Class’. The argument is null or empty. Supply an argument that is not null or empty and then try the command again.
$monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid};
I guess the error is an empty value at $monitoringclass.
Can you help me! Maybe we can Change the script to say if $Monitoringclass empty go to next….
Regards,
Martin
Hi Martin
Try to re-type the pipe character it might be a copy / paste error which causes the command to fail. If it doesn’t work, re-type the other characters.
Cheers,
Stefan
Has this issue been resolved? The script works fine for me and then I will see 3 or 4 of these warnings.
Thanks,
Gregg
I ran into the problem that Scorch Lover encountered and copying the .dll file didn’t do anything for me. I wound up finding a solution in the Ops forums on Microsoft.
http://social.technet.microsoft.com/Forums/systemcenter/en-US/8081ffdd-84ce-46de-ba52-7955bbc10cc0/powershell-write-action-module-using-om12-cmdlets#cc737110-aba2-4fe8-bdd5-6cb09552814f
I imported the module using the full path to the .psm1
Hi,
I ran into the following problem. I get an error: Invalid language type specified. Could someone help?
Thanks,
Arne
Where do you get this error?
Cheers,
Stefan
If I ran “C:WindowsSysWOW64WindowsPowerShellv1.0powershell.exe” I got this error. Now I changed the .Net Script back to PowerShell and the error is gone.
Arne,
I had the same issue. The problem in my case is/was that the executed powershell ran in an environment where the OperationsManager module was not registered and I was therefore not able to do import-module OperationsManager. However, I created a powershell script which I put on all Management Servers (c:userspublicdocumentsscriptsscomps.ps1) and which is “included” in all powershell command channels
scomps.ps1:
Import-Module “C:Program FilesSystem Center 2012Operations ManagerPowershellOperationsManagerOperationsManager.psm1”
. “C:Program FilesSystem Center 2012Operations ManagerPowershellOperationsManagerStartup.ps1”
. “C:Program FilesSystem Center 2012Operations ManagerPowershellOperationsManagerFunctions.ps1”
and my command channel is as followed:
Full path to cmd:
C:WindowsSystem32WindowsPowerShellv1.0powershell.exe
CMD parameters (example where I set a special resolution state to grab it via Orchestrator):
. c:userspublicdocumentsscriptsSCOMPS.ps1; Get-SCOMAlert -Id (‘$Data/Context/DataItem/AlertId$’) | Set-SCOMAlert -ResolutionState 13;
Startup folder:
C:WindowsSystem32WindowsPowerShellv1.0
Has anyone seen the issue for the line
$monitoringobject = Get-SCOMMonitoringobject -class $monitoringclass | where {$_.id -eq $moid}
that results in high memory consumption and never completes when the object is an interface on a network device? I see the following event id 26373 on the SCOM management server:
The query resulted in 230773 rows.
This may cause high memory consumption, and affect server performance.
Consider using buffered mode for this query.
We see this aswell. We recieve alot of Event 26373, OpsMgr SDK Service
___
The query resulted in 182678 rows.
This may cause high memory consumption, and affect server performance.
Consider using buffered mode for this query.
TypeId: Sytem.Entity
Criteria:
___
Any thoughts?
I resolved it by using this modification for the monitoringobject variable
$monitoringobject = Get-SCOMMonitoringobject -id $moid
Stefan,
This is an excellent runbook!
I have added a little thing in our situation. This runbook reset also monitors for alerts that were closed by the system. We don’t need to reset this ones as well so I added to the scomalerts def:
| where {$_.resolvedby -ne “System”}
Hi Michiel
Thank you for your input!
Cheers,
Stefan
The Script can be faster and easier: The Runbook starts the Powershell-Script once for each Alert.
So, you don’t need the for-each-loop. All used Commandlets accept Id as Parameter so you don’t need to querry all Objects and Pipe it to where {$_Id -eq …). With this enhancements no MonitoringClass-Instance is needed. And last but not least Get-SCOMMonitringobject returns only one Instance. Again there is no for-each needed.
# Import Operations Manager Module and create Connection
Import-Module OperationsManager;
New-SCOMManagementGroupConnection hgf-int-vm-273
# Get alert
$alert = Get-SCOMAlert -Id “`d.T.~Ed/{A28FF5FE-996F-4211-84A8-16836A554471}.Id`d.T.~Ed/”
# Get IDs
$mrid = $alert.monitoringruleid
$moid = $alert.monitoringobjectid
# Get Objects
$monitor = Get-SCOMMonitor -Id $mrid
$monitoringobject = Get-SCOMMonitoringobject -Id $moid
# Reset Monitor
$monitoringobject.ResetMonitoringState($monitor)
Stephan … in the above script is “`d.T.~Ed/{A28FF5FE-996F-4211-84A8-16836A554471}.Id`d.T.~Ed/” just the published alert ID from the Monitor Alert Activity ?
Hi
Yes.
Cheers,
Stefan
Hey Stephan ! Thanks for the awesome work ! … is that id on $alert = …. just the published data from the monitor alert activity ?
Hello Stefan,
thanks for this helpful Runbook. I have one Question:
I am getting the error that my user has not enough rights to do the resolve. It’s my SCO User. I don’t finde the possibility to change the user. So should I give the SCO user rights at SCOM?
Regards, Doreen
Great Runbook!
But one thing I have noticed when running this against Dependency Monitors is that the script has problem reseting the Health.
Tried to run this script and manually put in the AlertID and got this answer
3A9369B9-BA14-A5E0-DF81-7079B0255079
The monitor belonging to instance of the selected entity
was forced to reset its state. If monitor is unit monitor, it will contribute with
‘healthy’ state to its aggregate rollup monitor, eventually causing whole instance
health state recalculation. When monitor is not unit monitor, it forces reset of
every leaf unit monitor that rolls up its state thru this monitor. This task has no
effect on selected dependency rollup monitor.
Hi Stefan Roth,
I implemented your runbook and it runs, but it runs ervery 10 seconds. I don’t have so much monitores to be closed. What can I check?
Hi Stefan,
Excellent post . One thing i have not under stood You are using Monitoring ID to reset .How to implement this run book for all Monitors in SCOM ?