Cloud Management Gateway – Inbound Rule for Port 8443

Hello! My name is Nandan Sheth, and I am a part of Microsoft's Customer Success Unit based out of Dublin, Ireland. I have been helping customers set up the Cloud Management Gateway for a few years now, but recently an organization with 40000+ users asked me a question that I haven't given much thought to. When you set up the Gateway using a Virtual Machine Scale Set, one of the resources created in Azure is the Security Group. The Security Group has an inbound rule for port 8443.

Why is this rule created and what is it needed for?

Figure 1: Network Security Group showing the inbound rule for port 8443.Figure 1: Security Group showing the inbound rule for port 8443.

After searching a bit, I realized that this information is not properly documented and unless you really start digging into all the configuration that is performed to create the Gateway, there's no way to explain this rule. Once you have had a dig around though, it's not so complicated after all. In this blog post, I want to try and explain the purpose of this inbound rule for port 8443.

Let's start with a review of the ports that are used to facilitate the Cloud Management Gateway connections.

The ports used for connections to the Cloud Management Gateway are documented on this link. Port 8443 is not documented at all.

Client

Protocol

Port

Server

Description

Service connection point

HTTPS

443

Azure

CMG deployment

CMG connection point (virtual machine scale set)

HTTPS

443

CMG service

Protocol to build CMG channel to only one VM instance

CMG connection point (virtual machine scale set)

HTTPS

10124-10139

CMG service

Protocol to build CMG channel to two or more VM instances

CMG connection point (classic cloud service)

-

10140-10155

CMG service

Preferred protocol to build CMG channel

CMG connection point (classic cloud service)

HTTPS

443

CMG service

Fall back protocol to build CMG channel to only one VM instance

CMG connection point (classic cloud service)

HTTPS

10124-10139

CMG service

Fall back protocol to build CMG channel to two or more VM instances

Client

HTTPS

443

CMG

General client communication

Client

HTTPS

443

Blob

Download cloud-based content

CMG connection point

HTTPS or HTTP

443 or 80

Management point

On-premises traffic, port depends upon management point configuration

CMG connection point

HTTPS or HTTP

443 or 80 / 8530 or 8531

Software update point

On-premises traffic, port depends upon software update point configuration

With a Virtual Machine Scale Set, the Cloud Management Gateway service can be scaled up or down as needed. When the Cloud Management Gateway service has a single instance, all connections from the Cloud Management Gateway Connection Point to the Cloud Management Gateway Service are set up using port 443. We can validate this by running netstat -aon on the server hosting the Cloud Management Gateway Connection Point.

You can identify the public IP address by clicking into the Virtual Machine Scale Set. It is displayed in the Overview section. I have the public IP address hidden for security measures. 

Figure 2: Overview of the Virtual Machine Scale Set to identify the public IP Address.Figure 2: Overview of the Virtual Machine Scale Set to identify the public IP Address.

netstat -aon shows connections to this IP address.

Figure 3: Output of netstat -aon with a single VM instance.Figure 3: Output of netstat -aon with a single VM instance.

If I increase the number of instances in my Cloud Management Gateway service, and run netstat -aon again, I see the following details being returned:

Figure 4: Figure 3: Output of netstat -aon with multiple VM instances.Figure 4: Figure 3: Output of netstat -aon with multiple VM instances.

From the Cloud Management Gateway Connection Point, the connections are now being set up to ports 10124 and 10125. What happens to these connections on the Azure side? How is the traffic processed from these public ports?

The inbound rule for port 8443 starts coming into play when we increase the number of instances in the Cloud Management Gateway service. Because we have multiple virtual machines in Azure now, traffic goes through the Network Load Balancer. This resource is also created during the Cloud Management Service setup. Let's take a look at the Network Load Balancer in Azure.

  • From the Azure portal, click into the resource group that hosts the Cloud Management Gateway service.
  • Then, click into the Load Balancer resource.
  • Once you are on the Load Balancer blade, click on Frontend IP Configuration.

Figure 5: Options under Network Load Balancer.Figure 5: Options under Network Load Balancer.

  • In the details for the Frontend IP Configuration, there should be a single configuration – loadBalancerFrontEndWeb. Click into this configuration.
  • You should now see all the load balancing rules, Network Address Translation pools and inbound Network Address Translation rules.

Figure 6: Network Load Balancer Frontend Configuration showing Load Balancing configuration.Figure 6: Network Load Balancer Frontend Configuration showing Load Balancing configuration.

  • As you increase the number of instances in your Cloud Management Gateway service, you will see that the number of inbound Network Address Translation rules increases. The screenshot above was taken for a Cloud Management Gateway service with two instances.
  • Natpoolfe2.0 to 2.x show that traffic to the public IP address on ports 10124 to 10139 is being translated to port 8443. The screenshot below shows details for natpoolfe2.0 which translates 10124 to 8443. Similarly, natpoolfe2.1 will translate 10125 to 8443 and so on.

Figure 7: Network Mapping between port 10124 and 8443.Figure 7: Network Mapping between port 10124 and 8443.

  • Natpoolfe.0 and natpoolfe.1 are unrelated and are used to translate traffic to port 3389 and are not relevant here.
  • We can look at the overall topology for the Cloud Management Gateway service and this is how we can explain the Network Security Group inbound rule for port 8443.

Figure 8: Network Topology of Cloud Management Gateway with description of all the components.Figure 8: Network Topology of Cloud Management Gateway with description of all the components.

It is important to note that the inbound rule on Network Security Group is required to ensure that this translated traffic is allowed on to the virtual network that the Cloud Management Gateway service is attached to. If this rule is deleted, connections to the virtual machines in the Virtual Machine Scale Set may fail. On the public IP address, there is no listener for port 8443. This can be further validated using a simple PowerShell command as shown below. Test-NetConnection on port 433, 10124, 10125… completes successfully. However, Test-NetConnection on port 8443 fails.

NandanSheth_8-1685262017918.png

For more details on Network Security Groups, refer to these links:

Azure network security groups overview | Microsoft Learn

Network security group – how it works | Microsoft Learn

Hopefully, this clarifies the use of the Network Security Group inbound rule for port 8443 in case it is something you are asked to investigate.

 

This article was originally published by Microsoft's Secure Blog. You can find the original article here.