The post Handling Machine Reboots with VM Applications appeared first on Azure VM Runtime Team.

Well, in that case the extensions will run again. Is this the desired behavior? Well, we don’t know. There are many types of extensions. Some handle monitoring and security, so those you’ll probably want to keep. Some install applications, like VM Applications. You’ll probably want those re-installed. Others run a command, such as RunCommand or CustomScript. Those scripts will be re-run, which may be bad or good.

Sometimes, those scripts setup the environment on the machine. In that case, it’s good that they re-run for a new OS disk. Others drop database tables or perform other tasks that were useful at one time, but harmful now. You probably don’t want those to re-run.

Ultimately, you’ll need to decide before swapping the OS disk, which extensions you’re comfortable with running and which you’re not. If you’re unsure, it’s safest to just remove them. You can always add them back later.

Why do these extensions re-run?

Most extensions have some form of state. On Windows, this state is kept in the registry or on disk. On Linux, it’s kept on disk. When you swap out that disk, the state is gone. For many extensions, the most important part of that state is whether the extension previously ran. Thus, when the disk is gone, and the state is thus gone, the extension believes it’s being installed for the first time and re-runs. Again, sometimes this is beneficial, and sometimes it isn’t. Only you know that.

But, I don’t have any extensions, and my script still ran!

Perhaps you ran Get-AzVmExtension and didn’t see anything there. Unfortunately, not all extensions are reported there, because some are what we call implicit extensions. These are essentially “something else” that performs its operations via extensions, but are externally represented as something else. RunCommands are a key example. Therefore, you need to search not just for extensions, but for RunCommands. RunCommands can be returned using Get-AzVmRunCommand or Invoke-AzVmRunCommand, depending on the type of RunCommand you ran (action vs managed).

So, how do I prevent my scripts from re-executing?

If you previously used either managed or action RunCommand on the VM, and you don’t want it to re-execute after swapping out the OS disk, then you can remove the RunCommand or the extension.

Action RunCommand You can remove the extension via: Invoke-AzVMRunCommand -ResourceGroupName ‘rgname’ -VMName ‘vmname’ -CommandId ‘RemoveRunCommandWindowsExtension’

Alternatively, you can just change the script to do something benign like IpConfig (or ifconfig on Linux): Invoke-AzVMRunCommand -ResourceGroupName ‘rgname’ -VMName ‘vmname’ -CommandId ‘IpConfig’

Managed RunCommand Unlike action RunCommand, managed RunCommand supports multiple commands. You can list them via Get-AzVmRunCommand (or via CLI: az vm run-command show). You can remove a RunCommand using: az vm run-command delete, or Remove-AzVmRunCommand.

Custom Script Custom Script operates as any other extension, and can thus be removed via Remove-AzVmExtension.

The post Extension concerns when replacing the OS disk appeared first on Azure VM Runtime Team.

Here’s what you need to do.

Ensure your VM has Powershell7

Powershell7 is not installed by default. To ensure it’s available on your machine, you have the following options.

1. Create a VM image with Powershell7 pre-installed and use that to create your VMs. For more information on creating images, see here. For information on sharing images, see here.
2. Use RunCommand to install Powershell7 on your VM.

Here’s an example on how to do this:

az vm run-command create --name "blah" --vm-name $vmname--resource-group $rg --script "Invoke-WebRequest -Uri https://github.com/PowerShell/PowerShell/releases/download/v7.4.6/PowerShell-7.4.6-win-x64.msi -OutFile PowerShell-7.msi; Start-Process msiexec.exe -ArgumentList '/i PowerShell-7.msi /quiet /norestart' -Wait;Remove-Item -Path PowerShell-7.msi" --location $location

By the time you read this, you’ll likely need to update the download of Powershell7, which can be found here.

To verify that everything installed, check the instance view. Note that you may need to open an outbound networking rule for this to work.

az vm run-command show --name "blah" --vm-name $vmname --resource-group $rg --expand instanceView

3. Download the package yourself and create a VmApplication from it. More detail on VM Applications can be found here.

This option is more complicated, but is useful in situations where your VM is locked down and you don’t wish to open the outbound connection. Since VM Applications are downloaded via the host as a proxy, there’s no need to open any ports.

Run your script using Powershell7

As of this writing, this feature isn’t yet supported in the Portal (which only supports Action RunCommand and not Managed RunCommand), or via CLI and Powershell, where the necessary parameter isn’t supported. You’re therefore going to need to use an ARM template. Here is an example.

{    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",    "contentVersion": "1.0.0.0",    "resources": [    {       "type": "Microsoft.Compute/virtualMachines/runCommands",       "apiVersion": "2024-07-01",       "name": "[concat(parameters('vmName'), '/RunPowershellSeven')]",       "location": "[parameters('region')]",       "dependsOn": [],       "properties": {          "asyncExecution": false,          "source": {             "script": "Write-Output $PSVersionTable",             "scriptShell": "Powershell7"          },          "parameters": []        }    }    ],    "parameters": {       "vmName": {          "type": "string",       },       "region": {          "type": "string"       }    } }

Note the ScriptShell property that indicates the command should be run on Powershell7. Also note that the apiVersion must be at least 2024-07-01 for this to work.

To run this command on a VM, you then may run:

New-AzResourceGroupDeployment -ResourceGroupName $rg -TemplateFile $templateFilePath -vmName $vmname -region $region

And again to validate that the command ran:

az vm run-command show --name "RunPowershellSeven" --vm-name $vmname --resource-group $rg --expand instanceView

Of course your script doesn’t (and shouldn’t typically be) in line. All of the standard Managed RunCommand properties that download a script are valid.

The post Using Powershell7 with Managed Runcommand appeared first on Azure VM Runtime Team.

There are several basic security practices involving something like a domain password:

• Store them in keyvault
• Use managed identities to access the keyvault
• Change the password every six months are so

The standard procedure for cycling passwords is to actually have two keyvaults. Each contains a password, but only one is valid. When a new password is set, the oldest keyvault changes.

The JSonADDomain extension now has this capability. Two keyvaults are supported – primary and secondary. If the primary password fails, we’ll try again with the secondary. This now gives you total control over the password storage and allows you to follow modern security practices.

The settings to accomplish this are as follows:

{
  "apiVersion": "2015-06-15",
  "type": "Microsoft.Compute/virtualMachines/extensions",
  "name": "[concat(parameters('dnsLabelPrefix'),'/joindomain')]",
  "location": "[parameters('location')]",
  "dependsOn": [
    "[concat('Microsoft.Compute/virtualMachines/', parameters('dnsLabelPrefix'))]"
  ],
  "properties": {
    "publisher": "Microsoft.Compute",
    "type": "JsonADDomainExtension",
    "typeHandlerVersion": "1.3",
    "autoUpgradeMinorVersion": true,
    "settings": {
      "Name": "[parameters('domainToJoin')]",
      "OUPath": "[parameters('ouPath')]",
      "User": "[concat(parameters('domainToJoin'), '\\', parameters('domainUsername'))]",
      "Restart": "true",
      "Options": "[parameters('domainJoinOptions')]"
    },
    "protectedSettings": {
      "PrimaryPasswordKeyVault": {
        "KeyVaultUri": "[parameters('primaryKeyvaultUri')]",
        "ManagedIdentityClientId": "[parameters('managedIdentityClientId')]"
      },
      "SecondaryPasswordKeyVault": {
        "KeyVaultUri": "[parameters('secondaryKeyvaultUri')]",
        "ManagedIdentityObjectId": "[parameters('managedIdentityObjectId')]"
      }
    }
  }
}

The SecondaryPasswordKeyvault is of course optional, but recommended. For each, either a ManagedIdentityClientId or a ManagedIdentityObjectId (not both) must be specified in order to access it.

The post Properly cycling domain passwords with the JSonADDomain extension appeared first on Azure VM Runtime Team.

First, the basics. When you specify a replica count, we create one storage account behind the scenes for each replica. These are shared across versions of the same application. So, if you have version 1, 2, and 3, and each has 3 replicas, then they’ll all use the same storage account underneath. Different applications will have their own storage accounts. They won’t share.

If you are using block blobs, which is most common, then you’re limited to three replicas. If you use page blobs, the limit is much higher, but then your blobs need to be 512 byte aligned.

That being said, the major factor in deciding the number of replicas is egress. That’s the number of simultaneous connections to the storage account. The amount varies by region, but a good rule-of-thumb is 200 simultaneous connections per storage account. Keep in mind this is per region, because each region has its own accounts. We intentionally avoid cross-region copies.

Next, you need to factor in the size of your application. Larger files will take longer to download. How long depends on a number of conditions, but if you’re concerned about bandwidth you should run some tests to determine how long on average it takes to download, deducting perhaps thirty seconds for getting the information to download the package to the VM.

Then, you must determine how many simultaneous application downloads you will have at a time. The most common scenario where this may stress the system is a large VMSS being updated at the same time. However, many companies use upgrade domains for VMSS, in which case fewer instances will be updated at the same time. How many will actually be updated at a time will obviously depend on your business needs.

One scenario you may have to factor is the creation of a large VMSS. Since the goal is the simultaneous creation of VM instances, that means your application may be downloaded to every one simultaneously. There some delay exists for retries, this won’t help as much for large packages. The obvious solution here is to add your application as a subsequent request that takes advantage of upgrade domains. If that is not possible, then you will need to factor these in to the number of replicas.

From there, it should be a simple calculation. In practice, we’ve found that three replicas per region meet the vast majority of needs, but the page blob route exists just in case.

Note that there are retry mechanisms built into VM application package download. If we have an issue with one replica, we’ll automatically try another one. If the download fails in mid-run, we’ll also continue where we left off rather than trying to download the entire package from the beginning.

The post So how many replicas should my VM Application use? appeared first on Azure VM Runtime Team.

1. Through the Script parameter. You can inline your entire script in this manner, though we don’t recommend the practice if your script contains confidential information.
2. Through the CommandId parameter, which may be one of the values specified in the documentation.
3. Through the ScriptUri parameter, in which case RunCommand will download your script from this uri. To provide access, you may either provide a SAS uri or a managed identity.

However, what if your script uses various artifacts that also must be downloaded to the machine? Well, in the past it was necessary to call RunCommand are use some other technique to get those files on the machine. That process is now simplified.

Added to the properties for Managed RunCommand is “artifacts”. This is a collection containing artifacts with three properties. artifactUri – The uri from which to download the artifact. As with scriptUri, this may be a SAS uri or you may use managed identities fileName – This is the name to rename the file to after download. If you don’t supply a name we’ll use the entirely original names of “artifact0”, “artifact1” and so on. artifactManagedIdentity – This is the managed identity to use with the artifact. It follows the same syntax as elsewhere, where you may specify the clientId or objectId of the managed identity.

You may specify up to five artifacts besides the script. We’ll download these and place them in the same directory as the script. If one fails to download, the operation will fail.

Note that this functionality currently exists only through ARM templates or direct Rest API calls. Extra artifacts aren’t yet supported through Powershell and CLI and, as of this writing, Managed RunCommand is not supported via Portal – only the older Action RunCommand.

Below is an example of artifacts in an ARM template.

{
  "type": "Microsoft.Compute/virtualMachines/runCommands",
  "name": "[concat(parameters('linuxVmName'),'/linuxVmRunCommand')]",
  "apiVersion": "2019-12-01",
  "location": "[parameters('region')]",
  "properties": {
    "source": {
      "scriptUri": "[parameters('linuxScriptUri')]"
    },
    "parameters": [
      {
        "name": "[parameters('firstParameterName')]",
        "value": "[parameters('firstParameterValue')]"
      },
      {
        "name": "[parameters('secondParameterName')]",
        "value": "[parameters('secondParameterValue')]"
      }
    ],
    "artifacts": [
      {
        "artifactUri": "[parameters('firstArtifactUri')]",
        "fileName": "my_first_artifact"
      },
      {
        "artifactUri": "[parameters('secondArtifactUri')]",
        "fileName": "my_second_artifact"
      }
    ],
    "outputBlobUri": "[parameters('linuxVMOutputBlobUri')]",
    "errorBlobUri": "[parameters('linuxVMErrorBlobUri')]",
    "timeoutInSeconds": 20
  }
}

The post Introducing Managed RunCommand Artifacts appeared first on Azure VM Runtime Team.

Recently, I needed to issue a RunCommand in an ARM template, so I looked around for examples how to do this. Yes, even though we wrote RunCommand, we’re just as lazy as anyone else. However, I didn’t find anything, so I thought I’d share how this works so others may be lazy where I failed.

The following is an example resource for a VM.

{
  "type": "Microsoft.Compute/virtualMachines/runCommands",
  "name": "[concat(parameters('linuxVmName'),'/linuxVmRunCommand')]",
  "apiVersion": "2019-12-01",
  "location": "[parameters('region')]",
  "properties": {
    "source": {
      "scriptUri": "[parameters('linuxScriptUri')]"
    },
    "parameters": [
      {
        "name": "[parameters('firstParameterName')]",
        "value": "[parameters('firstParameterValue')]"
      },
      {
        "name": "[parameters('secondParameterName')]",
        "value": "[parameters('secondParameterValue')]"
      }
    ],
    "outputBlobUri": "[parameters('linuxVMOutputBlobUri')]",
    "errorBlobUri": "[parameters('linuxVMErrorBlobUri')]",
    "timeoutInSeconds": 20
  }
}

This is fairly straightforward. In this case, we’re downloading the script using a Sas uri, providing blobs to capture the output and error logs (also as Sas uris), requiring the script to finish in 20 seconds. I found the VMSS syntax to be slightly more confusing.

{
  "type": "Microsoft.Compute/virtualMachineScaleSets/virtualMachines/runCommands",
  "name": "[concat(parameters('windowsVmssName'), '/0/windowsVmRunCommand')]",
  "apiVersion": "2019-12-01",
  "location": "[parameters('region')]",
  "properties": {
    "source": {
      "scriptUri": "[parameters('windowsScriptUri')]"
    },
    "parameters": [
      {
        "name": "[parameters('firstParameterName')]",
        "value": "[parameters('firstParameterValue')]"
      },
      {
        "name": "[parameters('secondParameterName')]",
        "value": "[parameters('secondParameterValue')]"
      }
    ],
    "outputBlobUri": "[parameters('windowsVMSSOutputBlobUri')]",
    "errorBlobUri": "[parameters('windowsVMSSErrorBlobUri')]",
    "timeoutInSeconds": 20
  }
}

While everything else is the same, the name is a bit more confusing due to rather maddening rules governing path segment counts in ARM templates. The thing to keep in mind here is we only support RunCommands on specific instances of a VMSS. We don’t currently support running a command across an entire VMSS, though it isn’t difficult to automate this.

After some trial and error, the name of the RunCommand needs to have three segments, which should be the VMSS name + the instance + the name of your runCommand.

My hope is this helps you avoid the same trial and error.

Important note: On some Linux distros, adding a managed RunCommand may fail if the distro uses a very old version of the Guest Agent that does not support managed RunCommand. Changes were necessary to the agent to support the ability for multiple commands.

The post Using Managed RunCommand in an ARM Template appeared first on Azure VM Runtime Team.

• Write scripts that are idempotent, so that running them more than once accidentally doesn’t cause system changes.

However, more than once I’ve been asked: when does CSE actually re-run the script?

The answer is, it may run on a reboot. This can happen if your script never finished running. This is actually by design, since many scripts run by CSE may reboot the machine. So, in that case the scripts runs, installs some stuff, reboots the machine, then we call it again so it may complete the setup. Note that we say may because the OS may not accept the request to reboot immediately, so it depends on how your script is written. If you really want to continue after a reboot, use DesiredStateConfiguration (DSC) instead of CSE.

RunCommand is very different here. Once your script starts, regardless what happens, we won’t rerun it – even if it reboots the machine.

So, for a comparison:

So, that’s when CustomScript will re-execute. That seems simple, but the problem is you often can’t guarantee that the machine won’t be rebooted when running it. Numerous times I’ve seen a script start executing, then something else reboot the machine. This could be patching, some other process on the VM, or an operator. When the script comes back up, it executes from the beginning and causes havoc – or just fails.

For that reason, we advise all CSE scripts to be idempotent. You may think you have total control over whether the VM will reboot while executing your script, but you don’t.

The post When will CustomScript extension re-execute my script? appeared first on Azure VM Runtime Team.

This flag originated in VmApplications, where the question arose “what if my application should fail to install?” Should this result in a failed deployment? In some cases yes, but in others no. The truth is we don’t know what you might want, so treatFailureAsDeploymentFailure allows you to choose during deployment. By default it’s false, meaning if you application fails to install or update, the deployment itself won’t fail. To know whether the application succeeded, you’ll need to query the instance view.

Similarly, for RunCommand a command failure won’t fail the deployment unless this flag is set. Therefore, you’ll again need to query the instance view of the VM otherwise.

The post The treatFailureAsDeploymentFailure flag appeared first on Azure VM Runtime Team.

Therefore, you create a blob with the name MyBlob and set your install script to the following:

myApp.exe /S

You then name your VM Application “MyVmApp” and deploy it to a VM. The install will not work.

Why? Because when we download your package to your VM, we’ll rename it as “MyVmApp” (no extension). Why do we do that? Well, the VM doesn’t know what name your package should be called. Therefore, it gives it the best name it can – which is the name of the application itself – MyVmApp. It sets no extension because it has no idea what the original extension was.

There are several ways around this. One way is to change your script to this.

move .\\MyVmApp .\\myApp.exe & myApp.exe /S

Here, you just rename the file and then execute it. In practice, I see most customers just do this. However, there are two other possibilities.

The primary one is to use the “packageFileName” (there’s also a corresponding “configFileName”) property. This tells us what to rename your file. So, if you set it to “MyApp.exe”, then your install script only needs to be.

myApp.exe /S

Now, here’s the kicker. If your blob was originally named “myApp.exe” instead of MyBlob, then the above would have worked without setting packageFileName!

This is us trying to help you, though at times I admit we’ve confused you more. Obviously, we look at telemetry to see what problems our customers are having. We want you to use our features and so we look at all failures. When VM Applications first shipped, we noticed that the renaming was causing considerable frustration. Therefore, we try to help out with the following logic.

If packageFileName isn’t specified and the install script does not contain the application name and the install script does contain the blob name, then we automatically set packageFileName to the blob name for you. In other words, it just works.

Unfortunately, we’ve seen a few cases where customers are confused how this rename automatically occurs. We’re just trying to help you. If you don’t want any help, then just set packageFileName yourself and we’ll get out of your way.

Note that the packageFileName and configFileName properties are available today in Portal, Powershell, and CLI.

The post Working with VM Application Names appeared first on Azure VM Runtime Team.