As a follow up to my previous post, let’s write an equivalent “loop” in Azure DevOps using the same “matrix” strategy.
In this example, I want to run the same set of Terraform Infrastructure as Code on a bunch of different Azure subscriptions. I can pass in each subscription ID as a Terraform input variable.
value = "${var.subscription_id}"Azure DevOps provides the “jobs.job.strategy.matrix” syntax that we can use for this work. Using matrix, a new job will get created for each subscription ID and run the same Terraform code.
First, I need to get a list of all of the items I want to loop over. In this case, I can store the values in a CSV file in the repo with the rest of the code.
SubscriptionId,SubscriptionName
1234-4567-8901-23456789,Subscription1
5678-9012-3456-78901234,Subscription2I can retrieve these values and set up the JSON that will get stored in a variable.
The JSON needs to be generated to look similar to the following:
{
'Subscription1': {
'subscriptionId': '1234-4567-8901-23456789'
},
'Subscription2': {
'subscriptionId': '5678-9012-3456-78901234'
}
}Here is some PowerShell that can generate the correct JSON and store the result in an Azure DevOps variable for the next jobs to pick up.
- job: generateInput
steps:
- task: PowerShell@2
name: GenerateMatrix
inputs:
targetType: 'inline'
script: |
$subscriptions = Import-Csv "$(System.DefaultWorkingDirectory)/.azdo/subscriptionIds.csv"
$matrixItems = $($subscriptions | ForEach-Object { "'$($_.SubscriptionName)': { 'subscriptionId': '$($_.SubscriptionId)' }," })
$matrix = "{ " + "$matrixItems" + " }"
Write-Host "##vso[task.setvariable variable=matrix;isOutput=true]$matrix"The most important part of the above code is generating the correct JSON syntax for each line of the CSV file.
$matrixItems = $($subscriptions | ForEach-Object { "'$($_.SubscriptionName)': { 'subscriptionId': '$($_.SubscriptionId)' }," })We are going to generate a JSON object with the key subscriptionId and the value the GUID from the CSV file. This value will show up as a variable named subscriptionId in the next step.
We can now define the steps of the next part of the pipeline.
- job: runTerraform
dependsOn: generateInput
strategy:
matrix: $[ dependencies.generateInput.outputs['GenerateMatrix.matrix'] ]
steps:
- task: TerraformInstaller@1
displayName: install terraform
inputs:
terraformVersion: latest
- task: TerraformTaskV4@4
displayName: Initialize Terraform
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: 'rg-terraform'
backendAzureRmResourceGroupName: 'rg-terraform'
backendAzureRmStorageAccountName: 'saterraformrjb'
backendAzureRmContainerName: 'azdo-yaml-loop-terraform'
backendAzureRmKey: 'state.tfstate'
- task: TerraformTaskV4@4
displayName: 'terraform plan'
inputs:
provider: 'azurerm'
command: plan
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
commandOptions: '-var="subscription_id=$(subscriptionId)" -out=main.tfplan'
environmentServiceNameAzureRM: 'rg-terraform'
- task: TerraformTaskV4@4
displayName: 'terraform apply'
inputs:
provider: 'azurerm'
command: apply
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
commandOptions: 'main.tfplan'
environmentServiceNameAzureRM: 'rg-terraform'These steps will run the Terraform plan and apply steps, using the subscriptionId variable that comes from the previous step.
- job: runTerraform
dependsOn: generateInput
strategy:
matrix: $[ dependencies.generateInput.outputs['GenerateMatrix.matrix'] ]This code will generate the “loop”, multiple jobs for each item in the matrix variable. Each job will contain the subscriptionId variable.
- task: TerraformTaskV4@4
displayName: 'terraform plan'
inputs:
provider: 'azurerm'
command: plan
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
commandOptions: '-var="subscription_id=$(subscriptionId)" -out=main.tfplan'
environmentServiceNameAzureRM: 'rg-terraform'We are pulling the subscriptionId variable and passing in as a variable in the Terraform code.
The overall pipeline YAML looks like the following.
trigger:
- main
variables:
System.Debug: true
jobs:
- job: generateInput
steps:
- task: PowerShell@2
name: GenerateMatrix
inputs:
targetType: 'inline'
script: |
$subscriptions = Import-Csv "$(System.DefaultWorkingDirectory)/.azdo/subscriptionIds.csv"
$matrixItems = $($subscriptions | ForEach-Object { "'$($_.SubscriptionName)': { 'subscriptionId': '$($_.SubscriptionId)' }," })
$matrix = "{ " + "$matrixItems" + " }"
Write-Host "##vso[task.setvariable variable=matrix;isOutput=true]$matrix"
- job: runTerraform
dependsOn: generateInput
strategy:
matrix: $[ dependencies.generateInput.outputs['GenerateMatrix.matrix'] ]
steps:
- task: TerraformInstaller@1
displayName: install terraform
inputs:
terraformVersion: latest
- task: TerraformTaskV4@4
displayName: Initialize Terraform
inputs:
provider: 'azurerm'
command: 'init'
backendServiceArm: 'rg-terraform'
backendAzureRmResourceGroupName: 'rg-terraform'
backendAzureRmStorageAccountName: 'saterraformrjb'
backendAzureRmContainerName: 'azdo-yaml-loop-terraform'
backendAzureRmKey: 'state.tfstate'
- task: TerraformTaskV4@4
displayName: 'terraform plan'
inputs:
provider: 'azurerm'
command: plan
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
commandOptions: '-var="subscription_id=$(subscriptionId)" -out=main.tfplan'
environmentServiceNameAzureRM: 'rg-terraform'
- task: TerraformTaskV4@4
displayName: 'terraform apply'
inputs:
provider: 'azurerm'
command: apply
workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
commandOptions: 'main.tfplan'
environmentServiceNameAzureRM: 'rg-terraform'