SCOM Maintenance Mode Scheduler

Veeam just announced 4.0 of the “Management Pack for VMware”, also known as NWorks:

Version 4.0 includes comprehensive end-to-end monitoring from ESX Cluster, to ESX Host, to Virtual Machine and to Applications inside the VM. The nworks/Veeam MP now provides integration of application and service data with the VI3 topology. This breakthrough functionality enables a fully integrated picture, with end-to-end monitoring from ESX Servers through to the enterprise applications, such as SQL Server, Active Directory and Microsoft Exchange running inside virtual machines.

Version 4.0 also includes new features such as CIM SMASH hardware sensor status and alerts, graphical representations of data stores, flexible collection intervals, VirtualCenter 2.5 performance counters, and more!

Included with the Management Pack are:

• A full set of customizable Rules/Monitors for monitoring performance and events
• An automatically created Topology Diagram to provide a visual overview of your virtual environment
• Integration with Microsoft alerting & notification mechanisms
• Integration with Microsoft Reporting Warehouse

The cool thing about this solution is that is doesn’t rquire agents on ESX or any kind of helper for that matter. Veeam “Management Pack for VMware” relies on the VMware API/SDK, which assures that 4.0 is also fully supported for ESXi.

The nworks MP supports the following integrations:

Microsoft Integrations

• MOM 2005 SP1
• Systems Center Operations Manager 2007 SP1

VMware Solutions

• ESX Server 2.X and 3.X, including ESXi
• VirtualCenter 2.0.X and 2.5

I recently ran across and issue where our notification subscriptions were getting disabled every 30 minutes.  The strange thing was that only about half of our subscriptions were being disabled and they were the same subscriptions every time.  I tried re-enabling them with both with Powshell as well as the GUI and had the same result, subscriptions kept being disabled.  After digging through event logs I found this warning.
 

Log Name:      Operations Manager
Source:        Health Service Modules
Event ID:      11452
Task Category: None
Level:         Warning
 

Validate alert subscription data source module encountered an alert subscription data source with configuration that has gone out of scope. Disabling the alert subscription data source module.

Alert subscription name: Subscriptionaca6a276_e5a9_446b_9751_0ea539168e41

One or more workflows were affected by this. 
Workflow name: Microsoft.SystemCenter.ValidateAlertSubscription

The problem turned out to be that we recently cleaned up the SCOM Admins user group.  One of the users removed from the group had created half of the subscriptions.  By putting the user back in the SCOM Admins group and re-enabling the subscriptions the problem was solved.  But we really didn’t want this user in the SCOM Admins group as he had moved on to a different role. 

So why was this happening?    When a subscription is created the user who created the subscriptions SID is associated with that subscription.  There is a workflow that checks every half hour for SIDs no longer valid.  They could be invalid because their accounts access that had been removed, or possibly because the account has been disabled or deleted.

To fix it long term I had to export the “Microsoft.SystemCenter.Notifications.Internal” management pack.  This management pack is unsealed and contains all subscriptions.

Inside the management pack I searched for one of the subscriptions that were being disabled and one that was wasn’t.  I then replaced the SID of the bad subscription with the SID of the good subscription.

After doing replacing the SIDs I re-imported the management pack and re-enabled all subscriptions and the problem was solved for good.

Here is an example of one of the SIDs I had to replace.

Features Include

Enhanced application performance and availability across heterogeneous platforms
. Delivers monitoring across Windows, Linux and Unix servers-all through a single console.
. Extends end to end monitoring of distributed applications to any workload running on Windows, Unix and Linux platforms.
. Maximize availability of virtual workloads with integration with System Center Virtual Machine Manager 2008.

Improved management of applications in the data center
. Delivers on the scale requirements of URL monitoring of your business.
. Meet agreed service levels with enhanced reporting showing application performance and availability.
. More efficient problem identification and action to resolve issues.

Increased speed of access to information and functionality to drive management
. Faster load times for views and results.
. Improved and simplified management pack authoring experience

Download from MS Connect LINK

I needed to schedule a URL into Maintenance Mode this week and couldn’t find any scripts or examples so I created my own.

param($rootMS,$urlName,$minutes,$comment,$reason)

Add-PSSnapin "Microsoft.EnterpriseManagement.OperationsManager.Client" -ErrorVariable errSnapin;

Set-Location "OperationsManagerMonitoring::" -ErrorVariable errSnapin;
new-managementGroupConnection -ConnectionString:$rootMS -ErrorVariable errSnapin;
set-location $rootMS -ErrorVariable errSnapin;

$URLWatcher = (Get-MonitoringClass -name Microsoft.SystemCenter.WebApplication.Perspective) | Get-MonitoringObject | where {$_.DisplayName -eq $urlName}

$startTime = [System.DateTime]::Now
$endTime = $startTime.AddMinutes($minutes)

"Putting URL into maintenance mode"
New-MaintenanceWindow -startTime:$startTime -endTime:$endTime -monitoringObject:$URLWatcher -comment:$comment -Reason:$reason

 
Usage:

C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe URLMaintenanceModeV4.ps1 -rootMS: `SCOMRMS1′ -urlName: ‘MSN Website Checker’ -minutes:45 -comment: ‘My Comment’ -reason: ‘PlannedOther’

Download: URLMaintenanceModeV4.zip

SCOM Remote Maintenance Mode Scheduler is a GUI based tool that lets administrators easily schedule maintenance mode for a server or group of servers inside System Center Operations Manager 2012

Download for 2012: SCOMRemoteScheduler2.0_R2.zip

Instructions:  SCOMRemoteMaintenanceModeScheduler2.0.pdf

New Features in version 2.0 include

• Ability to schedule a daily maintenance mode.
• New Show Scheduled Tasks Dialog.
• Added feature to see all details for scheduled jobs.
• Added feature to delete scheduled jobs
• Fixed Minor bugs

Here is what the new Show Scheduled Tasks Dialog looks like

I just noticed this today that Vmware Workstation 6.5 was released on Tuesday this week. 

My Vista Media Center is running 24 x 7 recording HD and SD TV shows.  The core 2 duo procs sit at pretty low utilization so earlier this year I installed Vmware Workstation as well as put an extra 2 gigs of RAM and a second SATA drive in the desktop.  This makes for a great SCOM test environment without slowing down my laptop or having a dedicated server.

Some of the more notable new feature in Workstation 6.5 are

Unity mode — Integrate your favorite guest applications with your host. Open the application window, enter Unity mode, and the Workstation window is automatically minimized. The guest application windows look just like host application windows, but with color-coded borders.

More powerful record/replay of VM execution activity — Easily enable this powerful debugging tool, which records full system behavior, including all CPU and device activity. You can now insert markers while creating or playing back a recording and quickly navigate to these markers during replay. You can also browse a recording to replay from any spot.

Folder sharing and dragging, dropping, copying, and pasting text and files — Transferring files between virtual machines or between host and guests using these features is up to 25 percent faster.

Networking performance using NAT — You will see significant improvements in networking performance if a virtual machine is configured to use NAT. Performance is ten times better than in the previous release in some cases.

I/O performance — A new asynchronous I/O manager boosts performance on Windows hosts under heavy I/O loads.

http://www.vmware.com/download/ws/

The new management pack has a bunch of new rules and most importantly doesn’t cause the WMI errors like previous versions.

Download: My Custom Dell MP without SNMP

After importing the new Management Pack I immediately started getting errors on my Root Management Server and Management Server.  Not the good kind of errors you would expect, (like errors telling you there is a hardware issue).  The kind of errors that tell you that your SCOM infrastructure is now hosed!

Here were some the errors I received
*****************************************************************************************************************************
Data Warehouse failed to enumerate database components to be deployed. Failed to enumerate Data Warehouse components for deployment. The operation will be retried.
Exception ‘SqlException’: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
One or more workflows were affected by this.
Workflow name: Microsoft.SystemCenter.DataWarehouse.Deployment.Component
*****************************************************************************************************************************
Data Warehouse managed object type synchronization process failed to write data to the Data Warehouse database. Failed to store data in the Data Warehouse. The operation will be retried.
Exception ‘SqlException’: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
One or more workflows were affected by this.
Workflow name: Microsoft.SystemCenter.DataWarehouse.Synchronization.TypedManagedEntity
*****************************************************************************************************************************
Event ID: 31569

Description:
Report deployment process failed to request management pack list from Data Warehouse. The operation will be retried.
Exception ‘SqlException’: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.

Workflow name: Microsoft.SystemCenter.DataWarehouse.Deployment.Report

****************************************************************************************************************************
Health Service Unloaded System Rule(s)
Alert raised by monitor when system rules have been unloaded by the Health Service.
Event ID:      4000
A monitoring host is unresponsive or has crashed.  The status code for the host failure was 2164195371.
*****************************************************************************************************************************

After doing some digging I found there was a fix for the Data Warehouse errors.  Installing KB954643 immediately fixed the Data Warehouse errors.

The only thing left was the Event ID 4000 – A monitoring host is unresponsive or has crashed.  The status code for the host failure was 2164195371.  So how do we fix these errors?  I found KB951526.  Problem was I already installed this fix and was still having the issues.

So now how do we fix the problem?  The old management pack didn’t cause these issues so why is the new one? 
So I decided to export both the old and new management pack using Borris’s powershell script to see what changed is this new version of the Dell MP.

After evaluating the two management packs in the Authoring Console it was apparent that the new management pack contains tons of new SNMP monitors that I don’t need.  Also my experience after talking to people in the community is that the SNMP provider is not very scalable or robust.   So I set all of SNMP related discoveries to disabled and removed all rules related to SNMP.  After doing that and re-importing the management pack the 4000 errors are gone and everything seems to be working much better.  I have posted my customized management pack at the beginning of this post.

Microsoft began a major virtualisation push late yesterday, with the introduction of new virtualisation tools and by making its core hypervisor product free of charge. The new Hyper-V Server 2008 would be available via the web at no charge, Microsoft said in a statement. The download, which will go live in 30 days, supports virtual machines running Windows or Linux. The software comprises Hyper-V itself, plus a cut-down Server Core installation of Microsoft’s Windows Server 2008, and as such is a stand-alone version of the virtualisation solution.

Microsoft had previously said it would charge US$28 for the standalone Hyper-V product. As expected, Microsoft announced its new System Center Virtual Machine Manager (SVCMM) 2008, which will also be released within 30 days.

The software allows companies to centrally manage their virtual infrastructure, whether their virtual networks run on Microsoft’s Windows Server 2008 Hyper-V, Server 2008 Hyper-V or Virtual Server 2005 R2; or on VMware Virtual Infrastructure 3. Also yesterday, Microsoft said it had demonstrated that for the first time it could run live migration in a virtualised environment.

Full story: ZDNet News

PDF Version

Something that has always puzzled me with MOM 2005 and now System Center Operations Manager 2007 is that when I create a dynamic group of computers and one of the computers goes offline I don’t get a heartbeat.  This seems like something that should work “out of the box “as all other monitors are dependent upon the server being up.  Last year I posted a tool called the “watchanator” that addressed this issue.  The “watchanator” worked well but was complicated to setup and needed to be run every time the dynamic group changed.  Well now I have a better solution.  This new solution may seem a little complicated as well first but is really very easy if you follow my simple steps.

1.     Create the dynamic group, using the group wizard. 

a.  Give your dynamic group a name and select the unsealed management pack you want to store the group in.

b.  On the Explicit Members tab click next.

c.  On the Dynamic Members tab click  and create the formula you want to use to create you dynamic group.  I used a simple one that dynamically includes all of my exchange servers.

d.  From the groups window, right click to verify that your dynamic group includes the computers you want.

As you can see the dynamic group only contains windows servers and does not contain the heartbeat object(or agent watcher as it is know in scom)

2.     Export the Management pack

3.   Open up the Management pack in any xml editor.  I am using visual studio

4.  Search for <MembershipRules>.  The membership rules make up the dynamic group.  As you can see the first membership rule between <MembershipRule> and </MembershipRule> contains the formula that creates my dynamic group.

5.  Now we need to add the code to include the health watchers.  Open up watchers.xml available at https://www.scom2k7.com/downloads/Watchers.xml and copy the xml code. 

Here is the code.

<MembershipRule>

 <MonitoringClass>$MPElement[Name=”SystemCenter!Microsoft.SystemCenter.HealthServiceWatcher”]$</MonitoringClass>

<RelationshipClass>$MPElement[Name=”MicrosoftSystemCenterInstanceGroupLibrary6062780!Microsoft.SystemCenter.InstanceGroupContainsEntities”]$</RelationshipClass>

<Expression>

<Contains>              

<MonitoringClass>$MPElement[Name=”SystemCenter!Microsoft.SystemCenter.HealthService”]$</MonitoringClass>

      <Expression>

<Contained>         

<MonitoringClass>$MPElement[Name=”MicrosoftWindowsLibrary6062780!Microsoft.Windows.Computer”]$</MonitoringClass>

          <Expression>

            <Contained>

              <MonitoringClass>$Target/Id$</MonitoringClass>

            </Contained>

          </Expression>

        </Contained>

      </Expression>

    </Contains>

  </Expression>

</MembershipRule>

6.     Paste this code after the first </MembershipRule> and before </MembershipRules> then save the file.

*Note if you choose another class other than Microsoft.Windows.Computer to create you dynamic group you will have to replace the line from watchers.xml with the matching line in your dynamic group membership rule.

<MonitoringClass>$MPElement[Name=”MicrosoftWindowsLibrary6062780!Microsoft.Windows.Computer”]$</MonitoringClass>

7.   Save the MP and re-import the Management Pack into SCOM

8.   Go back to your group and right click View group members

9.  The health watchers may up to 20 seconds to populate.  Push F5 a couple of times to refresh the screen.

Now when we create a subscription to this group and a server in the group goes offline we will now get the heartbeat alert.

You download the sample management pack at https://www.scom2k7.com/downloads/Exchange.Servers.xml

Feature Summary

• DHCP 2003/2008
  • Native Systems Center Operations Manager 2007 MP
  • Windows Server 2008 Support
  • Scope & Superscope Address Availability Health
  • Updated Health Logic
  • New Reports
  • Updated Knowledge Base Information
  • Coexists with the included DHCP 2000 MP
• DHCP 2000
  • Converted Systems Center Operations Manager 2007 MP
  • Coexists with new native DHCP 2003 / 2008 MP

Note: This package includes library MP, 2000 and 2003 MPs, and DHCP 2008 MP. DHCP 2008 MP can only be installed on System Center OpsMgr 2007 SP1.