SCOM 2012 – Linux Two-State Monitor with “Script in Script”

If you are a Windows guy and you haven’t touched Linux so far implementing SCOM will force you to get hands on *nix systems. Here I would like to provide a cool, little way how to overcome a limitation of the Unix/Linux Shell Command Two (or Three) State Monitor.

This two state monitor allows you to call a shell script or a one-line command sequence (using pipeline operators). That means you just can call one command “one-liner” using the pipe symbol “|” e.g.  ls –l /tmp |  wc  –l . This example will count the files/directories in the /tmp directory. The “ls –l” command is similar to the Windows “dir” command and then the output is sent to the “wc –l” command which counts the words by line (wc=word count).  But the real world is that most scripts on the Linux side are not just one-liner. A Linux guy might creates a script or he might asks you if you can execute a script which calls another script in Linux. Sounds complicated? No, I show you…

In the /tmp directory I created two text files countfile.sh and runcount.sh (don’t get confused about the .sh ending, these are just plain text files).

image

The countfile.sh has two lines…

  • #!/bin/bash => Which shell executes this script
  • ls –l  /tmp | wc –l | bc => Will count the files in the /tmp directory. I just added the “bc” command which is used to convert to an integer value. But for this example you would not need it.

image

The runcount.sh file has also two lines…

image

Notice here the line…

. /tmp/countfile.sh => This line calls the countfile.sh file AND returns the output in the same shell. The “.” (dot) makes this possible, if you don’t use it the command would execute the countfile.sh in a separate shell and you would not be able to catch the value.

Next we need to make these scripts executable and readable. How do we do that? We set the permission of the files to read and execute using the “chmod” command. You must set these permissions or you won’t be able to run the scripts.

image

You can no check if it works by executing the /tmp/runcount.sh script…

image

We have 20 files and directories in the /tmp directory! Cool!

Now lets build the monitor…

image

Give it a name and choose a class in my case “SUSE Linux Enterprise Computer”…

image

For this testing I just create choose to run every 30 seconds (choose a higher interval for production! e.g. 30 minutes)…

image

Next you need to provide the shell command…

image

If there are more than 10 files in the directory throw an error alert…

image

And if there are equal or less than 10 files the monitor will be healthy…

image

We leave it the way it is…

image

Adjust the alert settings to your needs…

image

Notice here I added the line

$Data/Context///*[local-name()=”StdOut”]$

in the description field. This contains the output from the script in our case the file/folder count.

After a short time you receive an error if the threshold is reached…

image

and the alert properties…

image

In this example I showed you how to

  • create simple shell scripts
  • how to call a shell script from a shell script
  • how to use the “script in script” in SCOM

This will help you to overcome the “one-liner” limitation and the limitation to just execute one script.

If you are in the situation where you need to monitor Linux systems, I always try to force the Linux guys to build all the logic into their scripts and just return the values  of the monitored state. On the SCOM side I am just calling the script and make the corresponding mapping to their scripts. E.g. if the script output is “0” and means unhealthy and “1” means healthy I map this to the two state monitor. You also could you words like “NOK” or “OK” for unhealthy and healthy state.

I hope you find this useful Smile

About Stefan Roth

Consultant
This entry was posted in Script, Xplat. Bookmark the permalink.

10 Responses to SCOM 2012 – Linux Two-State Monitor with “Script in Script”

  1. Great blog you have here – very informative. Glad I stumbled onto it.

  2. 施 安宏 says:

    Hi , can i detect a log file and run command, not use timer ? thx.

  3. Arthur says:

    Hello Stefan,

    I have followed your post for another script bash. The monitor is added but the state stay green on this context “The monitor has been initialized for the first time or it has exited maintenance mode”. And never change status …

  4. Prakash Bhimji says:

    Hi Stefan,

    I wonder if you can please advise, i have followed your article (Which is great by the way), but i am not receiving any SCOM Alerts. if i run the command on the Solaris Server it return a value of 9 (which is correct), my Error Expression is set to alert if there are more than 5 files in the folder. the taget group is set to Solaris 10 Computers. is there something that i am missing?

    Kind Regards
    Prakash

  5. cloudwrap says:

    Good post – nice to see someone really getting to grips with Linux monitoring. It’s becoming more important in the enterprise space and this kind of work adds credibility to the value of SCOM in that mix.

    • scomfaq says:

      Hi,

      Thank’s for your comment. As Microsoft drives more and more into this direction I think it is very important to have a common understanding of “both” worlds.

      Cheers,

      Stefan

  6. user_feo says:

    I want to setup something similar for one of our linux server, but I want to know if there are any error in a log. I have used a script and actual command but it does not return an error when I test it.
    command: tail -n30 /tmp/log.070113 |awk ‘/error/’
    Any suggestions?

  7. Fabian says:

    Hello,

    is there any way to debug a monitor? My Script works fine and SCOM is able to execute it as i defined a Task to test this and everything works fine. Still my monitor doesn’t generate an event, even though it should. So my question is, how can I debug my scom monitor? I want to see what it does excactly to solve my problem.

    Greetings,

    Fabian

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s