TQA: Add-ClusterVirtualMachineRole fails with the error “The File Exists”

This time I was involved to identify a problem that stuck all the DR tests in a large enterprise, let me show you a description of the environment.



The Customer have a Production Farm (multiple HV clusters) on a Primary Datacenter, this farm have multiple LUNs (Logical Unit Number) on a visible to the hosts via FC (Fiber Channel). The replicate the LUNs with an asynchronous replica to the DR (Disaster Recovery) datacenter.

The customer, during the year, make multiple simulation on the DR datacenter to validate if the DR work correctly, but in the last simulation the Orchestrator server in the DR datacenter stopped the execution of the runbooks, that automate the import of the VMs from the LUNs, with the following error message:

“Add-ClusterVirtualMachineRole : The File exists.”

We identified inside of the orchestrator runbooks the piece of code that generate the exception and we retried the execution of the cmdlet only on a single VM like in this case:


We start thinking that may be the was already present in the , but why we receive the message “The File Exists”! So we have retried the command with the “-verbose” parameter, and we received this additional info:


From this info, it seams that the cmdlet “Add-ClusterVirtualMachineRole” (that add the to the cluster) is trying to get a temporary file:

VERBOSE: Connecting to cluster on local computer HV1-2016.

VERBOSE:    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.Path.InternalGetTempFileName(Boolean checkHost)

So we downloaded the Process Monitor and we retried Add-ClusterVirtualMachineRole for a single VM and we noticed these events:


During the execution of the cmdlet we saw that the PowerShell try to create multiple TMP files like “tmp2C0E.tmp” but it fail with a “NAME COLLISION” message.

So we start to investigate on the method used by this cmdlet:



Reading the documentation of the method used by the cmdlet, we identified the root cause:

This method creates a temporary file with a .TMP file extension. The temporary file is created within the user's temporary folder, which is the path returned by the GetTempPath method.

The GetTempFileName method will raise an IOException if it is used to create more than 65535 files without deleting previous temporary files.

The GetTempFileName method will raise an IOException if no unique temporary file name is available. To resolve this error, delete all unneeded temporary files.

Official Reference available here.

Because the customer during the year have made multiple tests on the DR environment, the Orchestrator user that start the runbooks had more than 65535 temporary files in the temp user folder with this kind of syntax tmpXXXX.tmp”.

The final solution was to add few line of code at the beginning of the runbook on the orchestrator server to cleanup the temp folder of the orchestrator user.


I was able to reproduce the Customer issue on a single Hyper-V Cluster in my lab, by using the New-TemporaryFile cmdlet, because it use the same method for the creation of the TMP files.


This article was originally published by Microsoft's Azure Blog. You can find the original article here.