Enrich your advanced hunting experience using network layer signals from Zeek

UPDATE : July 9, 2023

The article has been updated to include new signatures added for SSL, DNS and NTLM protocols.

UPDATE : May 22, 2023

On July 18, 2023, Microsoft will be deprecating a subset of signatures found in the “NetworkSignaturesInspected” action type of Advanced Hunting. With the recent integration of Zeek providing advanced protocol parsing capabilities, which result in better visibility into full network sessions compared to the raw packet bytes found in the “NetworkSignaturesInspected” action type of Advanced Hunting today, the effort to consolidate will provide a better overall experience for our customers by reducing the signatures that serve similar functions without the added benefits provided by the new Zeek alternative. For customers currently using the “NetworkSignaturesInspected” action type, here is a list of signatures that will be deprecated, referenced alongside their alternatives available in Advanced Hunting:

Steps you can take now:

Your organization might be using a “NetworkSignatureInspected” action type in your Advanced Hunting queries and custom detections. Particularly, you might be using a Signature Name that is going to be deprecated soon. Please update your queries with the new action types so that you can leverage this valuable data and avoid breaking your current custom detections.

An example of your old query:

DeviceNetworkEvents
| where ActionType == "NetworkSignatureInspected"
| extend AdditionalFields = todynamic(AdditionalFields)
| where AdditionalFields.SignatureName == "SSH"

Your new query:

DeviceNetworkEvents
| where ActionType == "SshConnectionInspected"

———–

In our previous blog about hunting for network signatures in Microsoft 365 Defender, we described how we used device discovery capabilities to capture some network event information in deeper detail and expose them in advanced hunting with the NetworkSignatureInspected action type. Since then we have made several developments, the most significant being the integration with Zeek. This release has expanded what is possible for generating network detections across Microsoft Defender for Endpoint. That announcement, shared examples of detections created for PrintNightmare and NTLM password spraying attempts.

Today, we would like to share a variety of Zeek-based events in advanced hunting that will help you expand your investigation, hunting, and detection capabilities for identifying and addressing network-layer anomalies across HTTP, SSH and ICMP protocols. Using the new Zeek events, we will demonstrate how to perform network threat hunting while also covering some of the MITRE ATT&CK Matrix.

Note: As the integration with Zeek continues to mature, more action types will gradually be released over time. With the Zeek integration only supported on Windows devices, these action types will surface for connections to and from Windows device.

To identify these action types in your tenant, look for the value ConnectionInspected in the ActionType field of the DeviceNetworkEvents table of advanced hunting. The extra information is stored in the AdditionalFields column as a JSON data structure and has the commonly known Zeek fields per event, which can be parsed. These field names are identical to those that Zeek uses, which are documented on Zeek's site. You can also check the Schema Reference flyout page on the advanced hunting pages to check for any new action types that were recently released.

Link to query

DeviceNetworkEvents
| where ActionType contains 'ConnectionInspected'
| distinct ActionType

The result of this query looks something like this:

Figure 1 – Sample result upon checking for ConnectionInspected in the ActionType table

The format of the action type will follow the [Protocol_Name]ConnectionInspected standard.

Inspecting HTTP connections

The HttpConnectionInspected action type contains extra information about HTTP connections, inbound or outbound. In cases where you click on an event of the HttpConnectionInspected action type, the page flyout will parse the additional fields and present them in a  format like the example below:

Figure 2 – Sample result of an HttpConnectionInspected action type

Below, you will find a complete list of fields that this action type can expose and the respective descriptions:

Let's look at a few examples of using the HttpConnectionInspected action type. In the first example, you want to look for rare user agents in the environment to identify potentially suspicious outbound web requests and cover the “T1071.001: (Application Layer Protocol) Web Protocols” technique.

Link to query

// Identify rare User Agent strings used in http conversations
DeviceNetworkEvents
| where ActionType == 'HttpConnectionInspected'
| extend json = todynamic(AdditionalFields)
| extend direction = tostring(json.direction), user_agent = tostring(json.user_agent)
| where direction == 'Out'
| summarize Devices = dcount(DeviceId) by user_agent
| sort by Devices asc

Suppose you have identified a suspicious-looking user-agent named “TrickXYZ 1.0” and need to determine which user/process/commandline combination had initiated that connection.  Currently, the HttpConnectionInspected events, as with all Zeek-related action types, do not contain that information, so you must execute a follow-up query by joining with events from  ConnectionEstablished action type. Here's an example of a follow-up query:

Link to query

// Identify usage of a suspicious user agent
DeviceNetworkEvents
| where Timestamp > ago(1h) and ActionType == "HttpConnectionInspected"
| extend json = todynamic(AdditionalFields)
| extend user_agent = tostring(json.user_agent)
| where user_agent == "TrickXYZ"
| project ActionType,AdditionalFields, LocalIP,LocalPort,RemoteIP,RemotePort, TimeKey = bin(Timestamp, 5m)
| join kind = inner (
DeviceNetworkEvents
| where Timestamp > ago(1h) and ActionType == "ConnectionSuccess"
| extend TimeKey = bin(Timestamp, 5m)) on LocalIP,RemoteIP,LocalPort,TimeKey
| project DeviceId, ActionType, AdditionalFields, LocalIP,LocalPort,RemoteIP,RemotePort , InitiatingProcessId,InitiatingProcessFileName,TimeKey

In another example, let's look for file downloads from HTTP, particularly files of executable and compressed file extensions to cover the “T1105: Ingress tool transfer” technique:

Link to query

// Detect file downloads
DeviceNetworkEvents
| where ActionType == 'HttpConnectionInspected'
| extend json = todynamic(AdditionalFields)
| extend direction= tostring(json.direction), user_agent=tostring(json.user_agent), uri=tostring(json.uri)
| where uri matches regex @".(?:dll|exe|zip|7z|ps1|ps|bat|sh)$"

The new HTTP action type will unlock a variety of possibilities for detection on this protocol. We  look forward to seeing the queries you come up with by sharing your contributions with the community.

Looking at SSH connections

The SshConnectionInspected action type will display information on SSH connections. While decrypting the entire SSH traffic is not possible, the cleartext part of the SSH session initiation can provide valuable insights. Let's look at the data found in the AdditionalFields section.

Figure 3 – Screenshot of additional fields that SshConnectionInspected generates.

The fields depend on the activity that was observed. Some of these fields might not appear depending on the connection. For example, if the client disconnected before completing the authentication, you will not have an auth_success field populated for that event..

Below, you will find a complete list of fields that this action type can expose and the respective descriptions:

Let's look at a few advanced hunting examples using this action type. In the first example, you want to look for potentially infected devices trying to perform “T1110: Brute-Force” against remote servers using SSH as an initial step to “T1021.004: Lateral Movement – Remote Services: SSH”.

The query below will give you a list of Local/Remote IP combinations with at least 12 failed attempts (three failed authentications on four sessions) of SSH connections in the last hour. Feel free to use this example and adapt it to your needs.

Link to query

// Detect potential bruteforce/dictionary attacks against SSH
DeviceNetworkEvents
| where ActionType == 'SshConnectionInspected'
| extend json = todynamic(AdditionalFields)
| extend direction=tostring(json.direction), auth_attempts = toint(json.auth_attempts), auth_success=tostring(json.auth_success)
| where auth_success=='false'
| where auth_attempts > 3
| summarize count() by LocalIP, RemoteIP
| where count_ > 4
| sort by count_ desc

In the next example, let's suppose you are looking to identify potentially vulnerable SSH versions and detect potentially unauthorized client software being used to initiate SSH connections and operating systems that are hosting SSH server services in your environment:

Link to query

// Identify Server/Client pairs being used for SSH connections
DeviceNetworkEvents
| where  ActionType == "SshConnectionInspected"
| extend json = todynamic(AdditionalFields)
| project Server = tostring(json.server),Client = tostring(json.client)
| distinct Server ,Client

Figure 4 – An example result with a short description of the different components

The results above describe breaking down the SSH banners to identify the different components. A short analysis of the banners shows that the server is Ubuntu 22.04, running OpenSSH version 8.9, and the client software is WinSCP version 5.21.3. Now, you can search these versions online to verify if they are vulnerable.

Note: The query above can be used to surface potential “T1046: Network Service Discovery” attempts, as attackers may try to search for unpatched or vulnerable SSH services to compromise.

Reviewing ICMP connections

The IcmpConnectionInspected action type will provide details about ICMP-related activity. The breadth of fields generated creates opportunities for some interesting detections. Here's an example of the human-readable view of the event as shown on the event flyout page

Figure 5 – Sample result of an IcmpConnectionInspected action type

 Below, you will find a complete list of fields that this action type can expose and the respective descriptions:

Let's explore a few examples of hunting queries that you can use to leverage the ICMP connection information collected by Defender for Endpoint.

In the first example, you wish to look for potential data leakage via ICMP to cover the “T1048: Exfiltration Over Alternative Protocol” or “T1041: Exfiltration Over C2 Channel” techniques. The idea is to look for outbound connections and check the payload bytes a device sends in a given timeframe. We will parse the direction, orig_bytes, and duration fields and look for conversations over 100 seconds where more than 500,000 were sent. The numbers are used as an example and do not necessarily indicate malicious activity. Usually, you will see the download and upload are almost equal for ICMP traffic because most devices generate “ICMP reply” with the same payload that was observed on the “ICMP echo” request.

Link to query

// search for high upload over ICMP
DeviceNetworkEvents
| where ActionType == "IcmpConnectionInspected"
| extend json = todynamic(AdditionalFields)
| extend Upload = tolong(json['orig_bytes']), Download = tolong(json['resp_bytes']), Direction = tostring(json.direction), Duration = tolong(json.duration)
| where Direction == "Out" and Duration > 100 and Upload > 500000
| top 10 by Upload
| project RemoteIP, LocalIP, Upload = format_bytes(Upload, 2, "MB"), Download = format_bytes(Download, 2, "MB"),Direction,Duration,Timestamp,DeviceId,DeviceName

Below is an example result after exfiltrating a large file over ICMP to another device on the network:

In the last example, you wish to create another hunting query that helps you detect potential Ping sweep activities in your environment to cover the “T1018: Remote System Discovery” and “T1595: Active Scanning” techniques. The query will look for outbound ICMP traffic to internal IP addresses, create an array of the targeted IPs reached from the same source IP, and display them if the same source IP has pinged more than 5 IP Addresses within a 10-minute time window.

Link to query

// Search for ping scans
DeviceNetworkEvents
| where ActionType == "IcmpConnectionInspected"
| extend json = todynamic(AdditionalFields)
| extend Direction = json.direction
| where Direction == "Out" and ipv4_is_private(RemoteIP)
| summarize IpsList = make_set(RemoteIP) by DeviceId, bin(Timestamp, 10m)
| where array_length(IpsList) > 5

Identifying the origin process of ICMP traffic can be challenging as ICMP is an IP-Layer protocol. Still, we can use some OS-level indications to narrow down our search. We can use the following query to identify which process-loaded network, or even ICMP-specific, binaries:

Link to query

DeviceImageLoadEvents
| where FileName =~ "icmp.dll" or FileName =~ "Iphlpapi.dll"

Inspecting SSL connections

The SslConnectionInspected action type contains extra information about SSL connections, inbound or outbound. In cases where you click on an event of the SslConnectionInspected action type, the page flyout will parse the additional fields and present them in a  format like the example below:

Figure 6 – Sample result of an SslConnectionInspected action type

Below, you will find a complete list of fields that this action type can expose and the respective descriptions ( sourced from Zeek SSL log documentation )

In the following example query, you wish to search https connections to external IP that use self-signed certificates :

Link to query

DeviceNetworkEvents
| where ActionType == "SslConnectionInspected"
| extend AdditionalFields = todynamic(AdditionalFields)
| extend issuer = tostring(AdditionalFields.issuer), direction = tostring(AdditionalFields.direction)
| where direction == "Out" and not(ipv4_is_private(RemoteIP))
| where AdditionalFields.issuer matches regex @"CN=S+$"

Inspecting DNS

The DnsConnectionInspected action type contains extra information about DNS connections, inbound or outbound. In cases where you click on an event of the DnsConnectionInspected action type, the page flyout will parse the additional fields and present them in a format like the example below:

Figure 7 – Sample result of an DnsConnectionInspected action type

Below, you will find a complete list of fields that this action type can expose and the respective descriptions (sourced from the Zeek DNS log documentation)

Using this action type we can explore and investigate the DNS traffic of each managed endpoint. In the following example we will try to identify potentially suspicious DNS traffic directed towards a DNS server that is not listed in the DNS server configuration of any managed endpoint:

Query Link

let knownDnsServers =
 DeviceNetworkInfo
 | where isnotempty( DnsAddresses)
 | mv-expand todynamic(DnsAddresses)
 | where DnsAddresses !in("::", "127.0.0.1","0.0.0.0")
 | distinct tostring(DnsAddresses);
DeviceNetworkEvents
| where ActionType == "DnsConnectionInspected"
| extend AdditionalFields = todynamic(AdditionalFields)
| extend Direction = tostring(AdditionalFields.direction), QueryType = tostring(AdditionalFields.qtype_name), Query = tostring(AdditionalFields.query), Answers = todynamic(AdditionalFields.answers)
| where Direction == "Out" and QueryType == "TXT" and RemoteIP !in(knownDnsServers) and isnotempty(Answers)

Inspecting NTLM traffic

For network connection with NTLM authentication we've create “NtlmAuthenticationInspected” action type. By clicking on the AdditionalFields the page flyout will parse the additional fields and present them in a format like the example below:

Figure 2 – Sample result of an NtlmConnectionInspected action type

Below, you will find a complete list of fields that this action type can expose and the respective descriptions (sourced from the Zeek NTLM log documentation)

Using using this action type we can track NTLM authentication that were observed on managed endpoints. In the following example we will try to identify multiple failed logons in short interval against internet-facing :

Query Link

let devices = DeviceInfo
|summarize arg_max(Timestamp, *) by DeviceId
| where IsInternetFacing
|distinct DeviceId;
DeviceNetworkEvents
| where ActionType == "NtlmAuthenticationInspected" and DeviceId in(devices)
| extend AdditionalFields = todynamic(AdditionalFields)
| extend Direction = tostring(AdditionalFields.direction),username = tostring(AdditionalFields.username), success = iff(isempty(AdditionalFields.success),  false, tobool(AdditionalFields.success))
| where isnotempty( username) and not(success) and Direction == "In"
| summarize Attempts = make_set(username) by SourceIP = LocalIP, bin(Timestamp, 10m),DeviceId
| where array_length(Attempts) > 3

More information

Understand which versions of the Microsoft Defender for Endpoint agent support the new integration here:

• What's new in Microsoft Defender for Endpoint | Microsoft Learn

Find out more details about the integration in our ZeekWeek 2022 presentations:

• ZeekWeek 2022 – Zeek for Windows: The Journey to Run on All Endpoints – Elad Solomon
• ZeekWeek 2022 – Zeek for Endpoint: Detection and Device Discovery – Boaz Wasserman

View the open-source contribution in Zeek's GitHub repository:

• Microsoft's Port of Zeek for Windows – zeek/zeek (github.com)

Previous announcements:

• Ignite:Microsoft Defender for Endpoint Announcements at Microsoft Ignite 2022
• Corelight:Zeek is Now a Component of Microsoft Windows (corelight.com)
• Zeek Project:A few questions and answers on Zeek and Windows

This article was originally published by Microsoft's Defender for Endpoint Blog. You can find the original article here.