SCOM – Certificate Missing Enhanced Key Usage EventID 20050

missing

If you want to monitor a server which does not belong to a domain you need to use a certificate, which has special requirements. You will find many posts how to handle SCOM certificates using a Microsoft PKI on the internet. An example is the detailed post from Tyson Paul. One of the essential requirements for the certificates is to provide the Enhanced Key Usage properties for Client Authentication (OID 1.3.6.1.5.5.7.3.2) and Server Authentication (OID 1.3.6.1.5.5.7.3.1). If you do not provide these properties you will receive an error in the Operations Manager event log…

image

A problem you could face in the real world is, that some customers won’t allow you to create the certificates for SCOM and they might have “generic” certificates for other use cases. Usually YOU provide the request file and provide the configuration for the certificates. Under certain circumstances this might not be the case. This means, that you might certain properties will be missing on the certificate itself. In case of SCOM, you can add the missing properties on the certificate. Just go to the Details of the certificate after you imported it into your computer. Click Edit Properties and select the purpose in the dialog, like this…

image

Having this option in place, let’s you successfully monitor the workgroup servers.

This will probably save you some headache 🙂 .

SCOM 2016 – Upgrade Notes from the Field

upgrade

Upgrading from SCOM 2012 R2 to SCOM 2016 is theoretically no such big deal. BUT sometimes you could face issues at the customer’s infrastructure, which force you to take some extra hurdle. This post should give you a high level overview of different migration scenarios and additionally some pitfalls you could meet upgrading to SCOM 2016.

High level upgrade path

There are 3 ways to upgrade a SCOM 2012 R2 environment.

1. Side-by-side migration (“Slow Motion”)

image

  • This is probably the way which has almost no risks, but takes a long time to finish and has a consequence that you loose old data. Why is this? You install a brand new SCOM 2016 management group, having brand new databases (OperationsManager / OperationsManagerDW / OperationsManagerAC). If needed you also install separate Web Console, Reporting and if needed the ACS role also on a dedicated (management) server. I think the best option is to install all these SCOM 2016 roles on a brand new Windows Server 2016 server and the databases on SQL Server 2016. This way you have the latest and greatest technologies available and you are armed for the next couple of years. Having this in place you are able to dual-home (multi-homing) the agent which is sending data to both management groups SCOM 2012 R2 and SCOM 2016. There is a good article on TechNet Wiki how to configure multi-homing if you have multiple AD forests or here if you have agents deployed in the same AD forest. As soon you have the new management group up and running you need to migrate all management packs, channels, subscriptions, overrides, roles etc. There are ways to export and import this stuff, but I recommend if you are choosing this upgrade path, then I would start configuring SCOM from scratch. Especially creating new overrides and documenting them will give you a chance to have a well configured and documented SCOM environment. One huge advantage of this upgrade path is, that you are able to upgrade to new versions of existing management packs, implementing new management packs and testing them thoroughly with no impact on your production SCOM environment until you switch management group and turn on notifications. This approach has also few disadvantages:
  • It takes usually a long time to finish this migration.
  • There are 2 management groups to maintain.
  • The amount of work to tune the management packs should not be underestimated.
  • Dual-homing an agent could lead to some more stress on the agent server.

2. SCOM In-place only upgrade (“Big Bang”)

image

  • If you decide to go for an in-place upgrade you are taking a much faster but also “risky” path, which needs more pre-work, testing and in case of failures also some plans to revert the changes using backups and/or VM snapshots. An in-place upgrade is in theory not that much of a problem and also fully supported by Microsoft. The first step is to run the SCOM 2016 setup on a management server which will discover the roles on the management server and upgrade the server itself and also the SCOM databases to SCOM 2016. If you managed to successfully upgrade the first management server / management group, then you go for the next management server, ACS Collector, Gateways, Console, Web Console and Reporting Server. As soon you have upgraded all components you are all done. Sounds easy, but believe me, there are plenty of things that could fail. This approach has also few disadvantages:
  • Because you upgrade SCOM only, the operating system stays the same. Of course you could theoretically in-place upgrade the operating system as well, but I really don’t encourage you to do so. If you need to upgrade SCOM and the operating system as well, please check the next upgrade option.
  • All your SCOM configurations bad or good will stay. If your management group is badly configured it will stay badly configured – an upgrade won’t change anything.
  • You need to check if the management packs work with SCOM 2016, especially third party or community MP’s. Please ask the vendor BEFORE you start the upgrade.
  • Make sure you meet the system requirements for SCOM 2016 .
  • Remember there are also 3rd party connectors in SCOM which might are not supported by SCOM 2016.

3. SCOM In-place upgrade and OS upgrade (“Big Bang++”)

image

  • If you decide to go for an in-place upgrade and you also want to upgrade the operating system to Windows Server 2016 in your environment, then this is an elegant way to achieve this goal. The risks are the same as “in-place only” upgrade but in addition you need to have a good plan how to switch the SCOM agents and ACS Forwarders to the new management servers. Before you start upgrading, make sure you have new Windows Server 2016 servers installed, which will become the new management servers. Step 1 is to run in-place upgrade on an “old” SCOM 2012 R2 management server (make sure it meets SCOM 2016 system requirements). If this is finished upgrade the other SCOM 2012 R2 management servers to SCOM 2016 and also ACS Collector, Gateways, Console, Web Console and Reporting Server. Step 2 if your management group is upgraded successfully install SCOM 2016 management servers on the fresh installed Windows Server 2016 servers. Depending on your SCOM environment, but if you have ACS installed, you could also install ACS Collector on a additional dedicated SCOM 2016 management server running on Windows Server 2016. Step 3 move the Windows / Linux agents, ACS Forwarders to the new management servers / ACS Collector. Step 4 uninstall the “old” management servers from the management group. If you have Web Console and/or Reporting installed you could simply uninstall the features from the “old” SCOM servers and reinstall it on new Windows Server 2016 server pointing to the SCOM 2016 deployment. I recommend uninstalling Reporting and Web Console BEFORE you upgrade the management group. This scenario has the same problems as an “in-place only” scenario but additionally you have to be aware of few more things:
  • Switching the Windows / Linux agents or ACS Forwarders to the new management servers could take some time and depending on the amount of “clients” Step 3 needs to be planned carefully.
  • If you don’t have the agents controlled by the SCOM Console you need to prepare some PowerShell scripts for moving the agents / ACS Forwarders to the new management servers.
  • Remember to install certificates for Linux  or Windows agents monitoring on the new management servers.
  • Remember to set the SPNs for the new management servers.
  • If you changed settings (Registry)  on your old management servers, check if you need to make these settings on your new management servers as well.

Continue reading

SCOM 2016 – System.Data.SqlClient. SqlException, Exception Error Code: 0x80131904 Login failed for user

NoClue

Sometimes there are things you won’t understand. I tried to install a new SCOM 2016 management server in a virgin Azure VM (Windows Server 2016). The necessary SQL Server 2016  was placed on another Azure VM (Windows Server 2016). Well this is nothing special and I have installed SCOM a gazillion times, BUT now I faced this error…1

…of course I checked the setup log as suggested and I found this error…

[21:17:34]:    Error:    :Exception running sql string [NOT DEFINED]: Threw Exception.Type: System.Data.SqlClient.SqlException, Exception Error Code: 0x80131904, Exception.Message: Cannot open database “OperationsManager” requested by the login. The login failed.
Login failed for user ‘RETURNONE\Stefan’.
[21:17:34]:    Error:    :StackTrace:   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, DbConnectionPool pool, String accessToken, Boolean applyTransientFaultHandling)
    at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions

I double checked the permissions for my account, so it has sysadmin permission on the SQL Server but repeating the installation ended up in the same error. After a while I found the solution, I unregistered and registered msiexec program on the SCOM server like this…

image

…the I executed the setup again (make sure you always use elevation ;))and the installation succeeded. Thanks to Abu-Obaid who delivered the solution, all credits to him. I hope this will save you some time!

SCOM – Editing SPN Quickly

Edit

If you are dealing with SCOM, you know that there is a lot to install and configure before it runs smoothly. One step during the installation process, is to configure the SPN (Service Principal Names) in Active Directory. In fact you need to set SPNs per SCOM management server and if you are hosting the web console on a dedicated server you also need to set an SPN (and Kerberos constraint delegation) correctly, so authentication will work properly. But how should the SPNs look like? Well, Kevin Holman (Microsoft) published  while ago an awesome post how they should look like. It will answer probably most of your questions. If not, just drop me a comment 🙂 .

There are different ways to mess with SPNs settings. First born tool is SetSPN.exe, which has been around for a while and can be considered “classic”. A more modern way of doing SPN registration is to use PowerShell of course. In terms of SCOM, if you are using a domain account for System Center Data Access Service then you could use PowerShell cmdlet Set-ADUser to  to register SPNs. It would look like this…

Get-ADUser -Filter 'Name –eq "[DAS Account]"' | Set-ADUser -ServicePrincipalNames  @{Add="MSOMSdkSvc/[MgmtServerFQDN]"}
Get-ADUser -Filter 'Name –eq "[DAS Account]"' | Set-ADUser -ServicePrincipalNames  @{Add="MSOMSdkSvc/[MgmtServerNetBIOS]"}

But if you more into GUI’s or you need to troubleshoot quickly, it might be faster to use the Active Directory Users and Computers console. You need to turn on Advanced Features

image

…check the Attribute Editor on your System Center Data Access account and select the servicePrincipalName property…

image

There you have a quick and nice overview, what has been configured on you service account. In addition you are able to add and remove obsolete SPN’s. Hope this helps :).

SCOM / OMS – MP University 2017 Recording

Sielct

Yes, Silect did it again! Few days ago Silect Software provided MP University 2017, an online event packed with sessions from well known names like Kevin Holman, Brian Wren and Aditya Goda from Microsoft, Marnix Wolf from Didacticum and Mike Sargent from Silect. What I like about this event is, that it is not marketing instead the sessions are packed with very deep content of MP authoring and as it seems to start touching OMS as well Smile. If you missed this event I encourage you to watch the recordings online on Youtube.

MP Authoring Basics and Silect MP Author

 

MP Authoring using Fragments

Continue reading

OMS – OMS, is it SCOM in the cloud?

on-premise-vs.-cloud

I can recall many instances whilst attending conferences and talking with customers or colleagues whereby misunderstandings have caused a significant amount of confusion.

“Operations Management Suite is SCOM in the cloud”

This is a one that has been doing the rounds lately, but it is correct? To answer the question we need to do a bit of digging into the past. André Malraux once said,

“Who wants to read in the future, must scroll in the past.”.

System Center Operations Manager (SCOM) was and is the Microsoft monitoring solution for homo- and heterogeneous IT environments. SCOM was originally developed by NetIQ, then purchased in 2000 by Microsoft. It carries with it a 17-year evolution, which started when the product was called Microsoft Operations Manager (MOM). In 2007 «MOM» was completely rewritten on a flexible and extensible framework SCOM was born. The development has continued ever since and the latest available version is SCOM 2016.
About 6 years ago, Microsoft began to experiment with System Center Advisor, an agent-based assessment and best practice analyzer solution based in the cloud. It provided the ability to analyze different workloads such as Windows operating system, SQL Server, Active Directory and Hyper-V components, detect changes to IT infrastructure, and propose Microsoft best practices from in the form of alarms. Between 2012 and 2013 the range of supported technologies was extended to include Exchange, SharePoint and Lync. Initially a separate solution, it quickly became integrated into SCOM 2012 SP1 by means of a connector. The newly generated information retrieved from Azure became available both on-premise within SCOM and in the cloud through System Center Advisor extension. By SCOM 2012 R2, the connector came pre-bundled as part of the suite. In 2014 System Center Advisor was transformed, gone was the Silverlight-based web application and in came a new HTML 5 based web app with a host of new capabilities. This meant that the Best Practice Analyzer System Center Advisor could be integrated into a new product called Azure Operational Insights, the range of capabilities for which could be greatly expanded by the use of so-called Intelligence Packs (IP). The following packs were released as part of the initial deployment:

  • Configuration Assessment
  • Malware Assessment
  • Capacity Planning
  • Change Tracking
  • Log Management
  • SQL Assessment
  • System Update Assessment

A new key feature acted like a cloud-based «data pot» whereby data was collected using an agent and could be analyzed with a PowerShell-like syntax within Azure Operational Insights Search Data Explorer.  A connection to SCOM was also ensured by a SCOM connector. In addition, the Operational Insights product is now the foundation for today’s current Operations Management Suite (OMS). Operational Insights Search Data Explorer is called Azure Log Analytics and Intelligence Packs are called solution (packs).
Since we now know the background of both products, I would like to juxtapose their facts, in order to be able to answer the question objectively.

Concept
SCOM consists of an extensible hierarchical object model. This means that components that are to be monitored in SCOM can be discovered (Discovery) by means of management packs (XML files) and placed into a hierarchy (service model) using relationships. Sensors (monitors) can move a subordinate object, into a healthy state, or into a faulty (unhealthy) state and visually represent it. The health state can be passed to its parent object (rollup). This model is described as a health model and has many advantages as well as certain disadvantages.

OMS works with so-called flat data, this means the data exists as data records in a large data pot. There are no objects or relationships among the collected data. For example, solution 1 collects disk information from computer X. At the same time solution 2 collects information on the same disk, BUT there is no relationship nor knowledge of the status between the disk data from solution 1 and solution 2. OMS does not (yet) have any service model and therefore also no health model.

Continue reading

SCOM / SCSM – Retrieve Decrypted RunAs Account Credentials

password-ftr

I am not sure if you have seen it, but Richard Warren from nccgroup has figured out, how to decrypt the RunAs account credentials in SCOM. The problem up to now was, that there was no official way to retrieve the encrypted credentials from SCOM. There is just one DLL to use, which offers the decrypt method. He has written a EXE and a PowerShell script on Github . I know there are always two sides of the medal. In this case an evil and a good way of using this knowledge. I think I don’t have to talk about the evil way, instead I would like to talk about its benefit.

Richard Warren has used it for SCOM RunAs accounts, but if you think about it Service Manager (SCSM), which is based on the same framework, therefore I was curious if this approach also works for SCSM. In fact it did! Why is this awesome? Well, think about it. We are able to “securely” store credentials in SCSM (or SCOM) using RunAs accounts. Now we are able to retrieve those credentials easily. Because I do a lot of automation in SCSM using service requests and itnetX PowerShell activities I always had some trouble to store credentials in a save manner. There are many ways to do so, like exporting the credentials into XML (Export-CliXML) , using certificates , encrypting the credentials using a key and store it somewhere like here or maybe you could store the credentials in SMA and retrieve it using PowerShell. Whatever method you are going to use, you will end up with more or less problems. The best approach would be, to store the credentials on the system where you need it (SCSM) and the SCSM administrator can manage these accounts without to dig into PowerShell code or certificates etc. Therefore RunAs accounts are a perfect way for storing credentials.

Because of that, I have used Richard’s sample, modified the code a bit to be able to use it on SCOM and SCSM and also return proper output. The PowerShell module will return the a credential hash table. You need to execute the module on the SCOM or SCSM management server and the only parameter you need to provide is the SCOM RunAs account display name like in this example.

In SCOM the RunAs account looks like this…

image

…and if you use the PowerShell module it works like this…

image

You can download the module from PowerShell Gallery . Be aware of the fact, that you need permission to access the database and management server.

Continue reading