Monitoring for an Azure Server Going Offline

My miniature schnauzer, Raven, is a smart and lively dog who loves to hunt for rodents in the yard. She has a keen sense of smell and can detect the slightest movement of her prey. She barks loudly to alert me whenever she finds a potential target and chases after it with all her speed. However, the rodents are too cunning and often escape to a tree or a hole in the ground before she can catch them. She then returns to me with a disappointed look on her face, hoping for a treat or a pat on the head.

is like Raven, but much more efficient and reliable. It can monitor your Azure servers and detect when they go offline in ~1 minute. It can also alert you via email, SMS, or webhook when something goes wrong, so you can take action to fix it. With , you can stay on top of your server's health and performance.

is a beneficial tool that has low costs for logs that are already in the tool. The main expenses for Azure Monitor come from ingesting the logs, so using the monitoring tool for data that is already there is a good way to help your enterprise reduce their spending. To illustrate how Azure Monitor can assist, an example of checking server availability is shown below.

Monitoring the availability of Azure servers can be very challenging, but Azure Monitor can help with that and make the task easier for the administrator. In the example below, I will demonstrate monitor and alert the administrator with an e-mail (or text message) when a server that is being monitored goes offline.

Requirements:

  • A Log Analytics Workspace (LAW) where servers send telemetry information.
  • The Azure Monitoring Agent installed on the server(s) to be monitored.
  • The server “Enabled” for monitoring, within Azure Monitor.
  • A Data Collection Rule.

There is the assumption that there is a LAW available to be used for this process. A LAW can be created if one doesn't exist.

Check to see if the Azure Monitoring Agent (AMA) is Installed

PaulBergson_0-1704993982946.png

  • Select “Extensions + applications” within “Settings”.
    • Look for the extension named “AzureMonitorWindowsAgent”.
      • If found, the AMA agent will be installed on this machine.

PaulBergson_1-1704993983024.png

Install the Azure Monitoring Agent (AMA) Manually

To install the AMA agent on an Azure VM without the use of an Azure policy, you can enable monitoring and create a Data Collection Rule (DCR). The manual process to install the agent, enable monitoring and create a DCR is covered later in this document.

Using Azure Policy to install the Azure Monitoring Agent (AMA) on All Newly Created Windows Servers

  • Browse to the Azure Policy blade.
  • Select Definitions.

PaulBergson_2-1704993982967.png

  • Click on the “Search” box.

PaulBergson_3-1704993982991.png

  • Select “Configure Windows to run Azure Monitor Agent using system-assigned managed identity”.
    • Note: The server must have system-assigned managed identity enabled on each virtual machine before the agent can be deployed.

PaulBergson_4-1704993983046.png

  • Click on the ellipse (the three dots at the end of the Name) and select “Assign”.

PaulBergson_5-1704993983398.png

  • Enter the correct “Scope”.
  • Enter any “Exclusions”, if needed.
  • Select “Next”.

PaulBergson_6-1704993982980.png

  • There should be no “Parameters”, so click “Next.

PaulBergson_7-1704993982987.png

  • Select “Next”
  • There is nothing to enter in the “Non-compliance message”.
  • Select “Next”.
  • Remediation is handled later.
  • Select Next.

PaulBergson_8-1704993983089.png

  • The policy definition should look similar to below.
  • Select “Create”.

PaulBergson_9-1704993982831.png

Create a Remediation task to install the Azure Monitoring Agent on Windows Servers that Exist but don't have the AMA Agent Installed

  • Within Policy select the “Assignments” blade.
    • In the “Search” box find the previously created policy and select it.

PaulBergson_10-1704993982883.png

  • Click on “Create remediation task”.

PaulBergson_11-1704993982859.png

  • Click on “Remediate”.

PaulBergson_12-1704993982862.png

  • Click on the “Remediation” tab.

PaulBergson_13-1704993982850.png

Any devices without the AMA agent should now be deploying/deployed. Clicking on the policy definition will articulate how many resources were remediated.

In the example below, there were no devices that required the agent.

PaulBergson_14-1704993982834.png

Enable Azure Virtual Machine Monitoring

In order to monitor servers in Azure monitor not only does the server have to have the AMA agent installed, but it also needs be Azure Monitor Insight “Enabled” and have a Data Collection Rule (DCR) on what should be collected.

  • Browse to Azure Monitor > Insights/Virtual Machines.
    • Select the “Not Monitored” tab.
      • Click on the “Enable” button.

PaulBergson_15-1704993982854.png

  • Click on the “Enable” button.

PaulBergson_16-1704993982806.png

Within the “Monitoring and Configuration” window:

PaulBergson_17-1704993983032.png

Note: Unless you have already created a rule specifically for “Alert Rules”, I recommend creating a new one.

  • Subscription – User selectable.
  • Data collection rule
  • Guest performance – Enabled
  • Processes and dependencies (Map) – Disabled
  • Log Analytics workspace – User selectable.
  • Click “Create”.

PaulBergson_18-1704993982984.png

PaulBergson_19-1704993983029.png

  • Click “Configure”

PaulBergson_20-1704993982988.png

  • Once the Azure Monitor – Insights Onboarding process has completed, this window can now be closed.
  • Refresh the Azure Monitor – Virtual Machines blade.
    • The new device can now be found in the “Monitored” blade.

PaulBergson_21-1704993983049.png

Verify the New Data Collection Rule was Created

  • Open a new browser tab and browse to Azure Monitor – Settings/Data Collection Rules.

The new Data Collection Rule can be seen.

PaulBergson_22-1704993982959.png

  • Close the newly opened tab.

Create an Azure Monitor Alert Rule

When creating an Azure Monitor Alert Rule, a Data Collection Rule (DCR) will need to be applied against any servers that should be monitored. To apply a DCR against an Azure vm it will need to have the AMA agent installed and running. Creating an Azure Alert Rule provides the ability to create a DCR and install the AMA agent.

In the example below, an alert rule will be created to create a notification if an Azure vm goes offline.

  • Open the Azure portal and browse to Azure Monitor.
    • Click on “Alerts”
      • Click on “Alert rules”

PaulBergson_23-1704993983044.png

  • Select “Create”

PaulBergson_24-1704993983126.png

This should bring up the “Alert rules” blade.

PaulBergson_25-1704993983042.png

  • Under the “Resource types” option select “Virtual Machines”

PaulBergson_26-1704993983016.png

  • Browse and select the resources to monitor.

PaulBergson_27-1704993983038.png

  • Select “Next: Condition >”.
  • Within the “Condition” tab, select “See all signals”.

PaulBergson_28-1704993982982.png

  • Select the “VM Availability Metric (Preview)”.
    • Click “Apply”.

PaulBergson_29-1704993982978.png

  • Ensure the following are configured:
    • Threshold = Static
    • Aggregation type = Average
    • Operator = Less than
    • Unit = Count
    • threshold value = 1
    • Check every = 1 minute
    • Lookback period = 1 minute

PaulBergson_30-1704993983009.png

  • Click “Next: Actions >”
  • On the “Actions” tab, select “+ Create action group”.
  • Complete the entries for:
    • Subscription
    • Resource group
    • Region = Global
    • Action group name = “EMail when a VM is unavailable”
    • Display name = “EMail for VM
  • Click “Next Notifications >”

PaulBergson_31-1704993982994.png

  • On the “Notifications” tab, click the drop-down arrow on the “Notification type” box.
    • Select Email/SMS message/Push/Voice”.

PaulBergson_32-1704993983021.png

  • Select the options for notification.
    • Email – checked.
    • SMS (Carrier charges may apply).
      • Phone number = 888 555-1212
  • Click “Ok”.
  • Name = “Notify Email”

PaulBergson_33-1704993983026.png

  • Click “Next: Actions >”.
  • Click “Next: Tags >”.
    • Enter any tags that are required.
  • Click “Next: Review + create >”.

PaulBergson_34-1704993982965.png

With the example of notification creation, you should receive both an e-mail and text message acknowledging you will be notified if this rule is triggered.

PaulBergson_35-1704993982996.png

PaulBergson_36-1704993982971.png

PaulBergson_37-1704993983001.png

Note: Under the “Contains actions” column, the actions to be taken are both email and messaging.

  • Click “Next: Details >”

The details tab gives the opportunity to configure the alert:

  • Severity
  • Rule Name
  • Alert Description

PaulBergson_38-1704993982976.png

  • Click on the “Advanced Options” to expand additional alerting options.
    • Check the box to “Automatically resolve alerts”.

PaulBergson_39-1704993983034.png

  • Click “Next: Tags >”
    • Enter any tags that are required.
  • Click “Next: Review + create >”
  • Click “Create”

PaulBergson_40-1704993983011.png

The new alert rule should now appear.

PaulBergson_41-1704993982999.png

Test the New Alert Rule

  • Within the Azure Monitor blade, select “Alerts”.
    • In the example below there are no Azure Monitors Alerts.

PaulBergson_42-1704993983039.png

  • Browse to the Azure “Virtual Machines” blade and stop a server.

PaulBergson_43-1704993983013.png

To show how the process works, I went into Azure Virtual Machines and stopped the server.

PaulBergson_44-1704993982914.png

Within the next minute or two, Azure Monitor should now notify you by email and messaging. In this tutorial only the email messaging will be displayed.

  • Notification of unavailability of the vm VM20019-DCR.

PaulBergson_45-1704993983036.png

  • Browse to the Azure Monitor > Alerts blade.
    • The new alert has fired.

PaulBergson_46-1704993982909.png

  • Clicking on the newly fired alert will bring additional details.

PaulBergson_47-1704993983003.png

The alert was configured (default) to auto-resolve, so if the server is re-started it will self “Resolve”.

  • Start the Azure vm back up.
    • After about 1 minute, the Azure Monitor alert will now display as resolved.

PaulBergson_48-1704993982969.png

An email should be sent about the new state of the alert.

PaulBergson_49-1704993982963.png

 

This article was originally published by Microsoft's Core Infrastructure and Security Blog. You can find the original article here.