ConfigMgr Performance Baseline the Easy Way

Hi, Jonas here!

Or as we say in the north of Germany: “Moin Moin!”
I am a Customer Engineer based in Germany and I recently played a bit with Azure Log Analytics (now called ) and the on-premises monitoring capabilities and I want to show you, how easy it is to monitor your own MECM / ConfigMgr environment.

TL;DR

If you ever saw performance problems in your ConfigMgr environment and if you are interested in an easy way to create a ConfigMgr performance baseline, this is the right article for you. I am explaining the concept and setup and provide a list of useful performance counters in the: “List of useful performance counters” section below.
Analyzing the gathered data will be a topic for a next article though, but you will be able to output something like shown in the picture below:

jonasoh_0-1597184346469.png

A baseline as a starting point:

When you encounter a performance issue in you environment you might not be able to tell if that is just the result of a normal increase in usage and you simply need more CPU or RAM or if what you encounter is an anomaly were you need to find the root cause to be able to get back to normal without the extra need for resources.

A baseline can help you answer this very basic question and then act before the real problem starts.

So, if you have gathered data for about a month or so and you see something like this:

log000-perf1-1.PNG

You have your baseline with some ups and downs, and you should be able to tell, if the next performance decrease is just a normal increase in usage (as shown in the left picture below) or an anomaly which you need to analyze and fix (as shown in the right picture below). (A very simplified graphic visualization)

jonasoh_2-1597184346507.png

The setup options:

As mentioned before, I am using /Azure Log Analytics to monitor my on-premises environment and Log Analytics gives me two options to send data to the Log Analytics service.

You can either send the data directly from the monitoring agent (running on the local machine) to Azure Log Analytics (see option 1 below) or use the Azure Log Analytics Gateway if not every machine has direct access to the internet (see option 2 below).

You can find the documentation here: LINK

Option 1: Direct connection:

jonasoh_3-1597184346513.png

Option 2: Log Analytics Gateway:

jonasoh_4-1597184346520.png

Log Analytics agent setup:

Since my machines have direct access, I simply installed the Log Analytics agent on them and used option 1 (see above) to send performance data to my workspace and I followed the following steps:

Step 1: Create Log Analytics workspace

All you need is an Azure Subscription where you can create an Azure Log Analytics workspace. The steps are documented here: Create workspace

Step 2: Install Log Analytics agent

Go to your new Log Analytics workspace and click on “Agents management” to download the Log Analytics agent to the machines you would like to monitor.

jonasoh_5-1597184346553.png

During the agent setup select “Connect the agent to Azure Log Analytics (OMS)” and click “Next”.

log005-b.PNG

On the next page you need to copy and paste the “Workspace ID” and the “Primary Key” from the “Agents management” site we used earlier to point the agent to the correct workspace. If you need to set up a proxy for the connection, you can do this via the “Advanced” button.

log006-b.png

Step 3: Verify connectivity

You can find the installed “Microsoft Monitoring Agent” in Control Panel under “System and Security”. (you will find multiple names for the agent in the documentation)

log008-b.png

On the “Azure Log Analytics (OMS)” tab you can verify the successful connection to you Log Analytics workspace:

log009-b.png

If you go back to the “Agents management” site in your Log analytics workspace, you can click on “Go to logs” to verify the successful connection of your agents.

log010.PNG

You should be redirected to the “Logs” section, where the following query should output your connected machines.

Keep in mind that it can take a minute for the first data to show up and you might need to click on “Get started” if you see the page for the first time:

log011.PNG

If not, go to “Logs” and run the following query:

Heartbeat

| where OSType == ‘Windows'

| summarize arg_max(TimeGenerated, *) by SourceComputerId

| sort by Computer

| render table


Defining the baseline

All you have to do is to add the needed performance counters and let the agent gather the data.

Go to “Advanced settings” in your Log Analytics workspace“ and click on “Data”, “Windows Performance Counters” and add the counters you like with the plus sign:

log012.PNG

For most default counters you can simply use the search box next to the plus sign and add them, but what if we need to add some counters not in the list, the ConfigMgr counters for example?

I wrote a little script called “Get-PerfCounterList.ps1” to help you find the correct counter names and be able to easily copy and paste them into Log Analytics.

The script can be downloaded here: LINK

IMPORTANT: Run the script as an administrator on the machine you want to monitor with Log Analytics, otherwise only a subset of counters might be visible.

The output is a simple grid view showing you all the available performance counters on the machine.

And if you filter for “SMS” for example, you get a list of the ConfigMgr counters and the path you need for Log Analytics.

log013-b.png

All you have to do is to choose the counter you like to monitor and copy the path into the counter search field in Log Analytics and hit the plus sign to add the counter to the list.

If there are multiple instances available (as shown in the screenshot above via the green arrows) you can select multiple counters in the grid view and click on “OK” to get another grid view of those specific instances and the correct path names:

log014-b.png

The script is also helpful if you use a named instance to store your ConfigMgr SQL DB, because you then need the exact name of the performance counter.

Since I am using a SQL instance called “INST01”, my counters look like this for example: “MSSQL$INST01:Memory…” instead of “SQLServer:Memory…”:

log015-b.png

As mentioned before, simply copy and paste the counter you like into the search bar and click on the plus sign next to it:

log018.PNG

I added the “SMS Inbox(*)File Current Count” counter in my example and since the counter will only be refreshed every 15 minutes locally, I set the sample interval to 900 seconds.

Since the counter has 32 instances and each instance will have an entry in Log Analytics, the higher sample interval will limit the data which needs to be stored in Log Analytics.

(At the time of writing the maximum sample interval was 1800 seconds (30 minutes))

When you are done adding all the needed counters, click on “Save” and the counter configuration will automatically be sent to every connected agent.

Sizing estimates

The actual amount of data stored in Log Analytics depends on the sample interval per counter, the number of counters, the number of instances per counter and the number of agents sending data to the workspace.

Use the script mentioned above to see how many instances each counter has and check if each instance is needed or if just a subset is enough to get the baseline you need and adjust the sample interval to save space if needed.

You will find more details about that topic in the Log Analytics documentation:

Windows and Linux performance data sources in Azure Monitor

Azure Monitor pricing

List of useful performance counters

I will not explain every counter in detail, because that would be an article on its own, instead I will add some notes to some of them, if I feel that's important to the baseline for ConfigMgr.

It is also not a complete list of counters, but the list should give you the most useful data for your baseline.

Use the search term: “Windows Performance Counters Explained” to find resources about the counters and how they are helpful.

Operating System related:

LogicalDiskAvg. Disk sec/Read
LogicalDiskAvg. Disk sec/Write 
LogicalDiskCurrent Disk Queue Length 
LogicalDiskDisk Reads/sec
LogicalDiskDisk Transfers/sec 
LogicalDiskDisk Writes/sec 
Memory% Committed Bytes In Use 
MemoryAvailable Mbytes
MemoryPage Reads/sec
MemoryPage Writes/sec
AdapterBytes Received/sec
AdapterBytes Sent/sec 
InterfaceBytes Total/sec
Processor(_Total)% Processor Time 
SystemProcessor Queue Length 

related:

SQLServer:Access MethodsFull Scans/sec
SQLServer:Access MethodsIndex Searches/sec

SQLServer:Access MethodsIndex Searches/sec

SQLServer:Access MethodsTable Lock Escalations/sec

SQLServer:Access MethodsIndex Searches/sec

SQLServer:Buffer ManagerFree pages

SQLServer:Buffer ManagerLazy writes/sec

SQLServer:Buffer ManagerPage life expectancy

SQLServer:Buffer ManagerStolen pages

SQLServer:Buffer ManagerTarget pages

SQLServer:Buffer ManagerTotal pages

SQLServer:Databases(*)Log Growths
SQLServer:Databases(*)Log Shrinks

SQLServer:Locks(*)Number of Deadlocks/sec

SQLServer:Memory ManagerMemory Grants Outstanding

SQLServer:Memory ManagerMemory Grants Pending

SQLServer:Memory ManagerTarget Server Memory (KB)

SQLServer:Memory ManagerTotal Server Memory (KB)

SQLServer:Plan Cache(Object Plans)Cache Object Counts

SQLServer:Plan Cache(SQL Plans)Cache Object Counts

SQLServer:Plan Cache(Object Plans)Cache Pages

SQLServer:Plan Cache(SQL Plans)Cache Pages

SQLServer:SQL StatisticsBatch Requests/sec

SQLServer:SQL StatisticsSQL Compilations/sec

SQLServer:SQL StatisticsSQL Re-Compilations/sec

SQLServer:Wait Statistics(*)Memory grant queue waits

SQLServer:Wait Statistics(*)Network IO waits

SQLServer:Wait Statistics(*)Page latch waits

SQLServer:Wait Statistics(*)Wait for the worker

ConfigMgr related:

SMS Inbox(*)File Current Count            (will only be updated every 15 minutes locally)

SMS Outbox(*)File Current Count         (will only be updated every 15 minutes locally)

SMS AD Group DiscoveryDDRs generated/minute

SMS AD System DiscoveryDDRs generated/minute

SMS Discovery Data ManagerUser DDRs Processed/minute

SMS Inventory Data LoaderMIFs Processed/minute

SMS Software Inventory ProcessorSINVs Processed/minute

SMS Software Metering ProcessorSWM Usage Records Processed/minute

SMS State SystemMessage Records Processed/min

SMS Status Messages(*)Processed/sec

IIS related:

Web Service(*)Bytes Sent/sec

Web Service(*)Bytes Received/sec

Data visualization

To analyze the gathered data, got to your Azure Log Analytics workspace and click “Logs” (1).

The actual data is stored in the “Perf” table (2) under “LogManagement” and can be queried via the query window (3) using KQL (Kusto Query Language).

In my example, the output is a “timechart” (4), but it can be any type of output Azure Log Analytics is capable of.

The query I am using is just an example, because analyzing the data is a topic on its own and might be worth another article.
Other example queries can be found here: LINK

log016.PNG

Data preview:

By clicking on the preview icon (see below) you will get a result set with 10 entries back, which helps to explore the gathered data and to finetune your KQL queries:

jonasoh_18-1597184347203.png

Alerting and monitoring:

You can create alerts and send out notifications when certain criteria are met, like CPU is at 90% for the last 15 minutes, or a ConfigMgr inbox has over 1000 files in it for the last hour or so and you have the ability to use multiple reporting features to visualize the data like, Workbooks, Azure Portal Dashboards or PowerBI reports.

I will not explain those topics in this article, since each part would be a topic for its own article. Instead I provide you with a list of links to the documentation if you want to start right away:

Alerting: LINK

Azure Portal dashboards: LINK

Azure Monitor Workbooks: LINK

PowerBI reports: LINK

Conclusion:

Azure Log Analytics / Azure Monitor gives you an easy way of gathering performance data and building your ConfigMgr performance baseline and with KQL and the Alerting feature you have powerful tools to analyze the data and generate alerts when needed.

Have fun monitoring your environment and let me know what you think

Jonas Ohmsen

Microsoft Germany

Disclaimer:

This posting is provided “AS IS” with no warranties, and confers no rights

Resources:

Get-PerfCounterList.ps1
https://github.com/jonasatgit/scriptrepo/tree/master/General

Windows and performance data sources in Azure Monitor:

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-sources-performance-counters

Azure Monitor pricing:

https://azure.microsoft.com/en-us/pricing/details/monitor/

Create a Log Analytics workspace in the Azure portal:
https://docs.microsoft.com/en-us/azure/azure-monitor/learn/quick-create-workspace

Log Analytics agent overview:

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/log-analytics-agent

Azure Monitor:
https://docs.microsoft.com/en-us/azure/azure-monitor/platform/alerts-overview

Azure Portal dashboards:

https://docs.microsoft.com/en-us/azure/azure-monitor/learn/tutorial-logs-dashboards

Azure Monitor Workbooks:

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/workbooks-overview

PowerBI reports:

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/powerbi

 

This article was originally published by Microsoft's Azure SQL Database Blog. You can find the original article here.