Deploy Terraform to AWS w/ Microsoft Azure DevOps
Introduction
That is correct you read accurately. This post will guide you through how to deploy AWS resources via Azure DevOps. This feels like a bit of a mashup; however, in this modern age organizations constantly crave the flexibility and capability of being able to deploy their resources to multiple clouds.
Background
We are going to build a simple Azure DevOps pipeline to deploy an AWS S3 account. For those unfamiliar with AWS an S3 (Simple Storage Solution) is the comparable service to Azure Storage Accounts. We are also going to leverage the Azure Pipelines Terraform Task found on the market place and one I have written about before, Azure DevOps Terraform Task.
This process will involve a remote state file which will also be contained in an additional S3 bucket.
If you follow me then you will also know that a static one use pipeline has a limited use. We will write this in such a way to introduce new task, job, and stage YAML Pipelines into my TheYAMLPipeline Repository. If this is new to you free to check out my additional posts on my blog and/or the Microsoft Community series of blog posts on the topic.
I am gong to exclude how to write an S3 Terraform block as this is something that can easily be deciphered from Hashicorp’s documentation. My one piece of advice is the provider will be “aws” and the backend will be “s3”
Getting Started
Fortunately, for us, the Marketplace task supports BOTH AWS and Azure (as well as GCP….maybe will do that one day). So for this to work we will require the following to be in place:
- Install Azure Pipelines Terraform Task
- Configure AWS Toolkit for Azure DevOps
- An IAM account in AWS with ability to create an S3 account
- A secret for said account
AWS Toolkit for Azure DevOps
Once we have the AWS Toolkit for Azure DevOps marketplace extension in stalled, we will have ability to create a Service Connection to AWS:
![](/wp-content/uploads/2024/07/image.png)
Now I am only going to use the Access Key ID and Secret Access Key as those are also the most familiar for those coming from an Azure background. These would be the Azure Service Principal ID and Secret.
Something to understand if you aren’t verse in AWS. This Access Key ID is attached to an Amazon Resource Name, ARN, which is AWS’s equivalent to an Azure Resource ID. This access Key ID then will have an Access Key associated with it for signing the request. The ARN will have a role in IAM which will have the required access to create and manage the S3 account.
Azure Pipelines Terraform Task
Now that we have this service connection we can write the Terraform Task. The documentation for this task is honestly well written. Effectively we can reuse the pattern from Terraform, CI/CD, Azure DevOps, and YAML Templates. The only difference is we need to use the AWS specific parameters.
Here is an example of the init task:
parameters:
- name: serviceName
type: string
- name: TerraformDirectory
type: string
- name: backendServiceAws
type: string
- name: backendAwsRegion
type: string
- name: backendAwsBucket
type: string
steps:
- task: TerraformCLI@1
displayName: 'Terraform : init'
inputs:
command: init
backendType: aws
workingDirectory: ${{ parameters.TerraformDirectory }}
backendServiceAws: ${{ parameters.backendServiceAws }}
backendAwsRegion: ${{ parameters.backendAwsRegion }}
backendAwsBucket: ${{ parameters.backendAwsBucket }}
backendAwsKey: ${{ parameters.serviceName }}.tfstate
For brevity, since Terraform, CI/CD, Azure DevOps, and YAML Templates already covers a lot of the structure and templating I am going to refer to the AWS specific tasks here which ultimately transpile into the finished pipeline.
- Tasks
- Jobs
- Stages
The fully expanded YAML can be found here
The source code for this s3 bucket can be found on my public repo cicd_aws_s3
Results
So now we will hook up the cicd_aws_s3 to Azure DevOps and Run a pipeline:
![](/wp-content/uploads/2024/07/image-1.png)
And confirmation of our apply working:
![](/wp-content/uploads/2024/07/image-4.png)
Even better the features we’ve come to love from the Azure Pipelines Terraform Task are available with AWS. Here are the plan change view:
![](/wp-content/uploads/2024/07/image-2.png)
And our built in condition to deploy if only an apply was detected is still working:
![](/wp-content/uploads/2024/07/image-3.png)
Learnings
First, this was easy! I didn’t realize the learning curve was so small. Yes, Terraform is multi cloud so that helped. We aren’t looking at the actual service deployments like an App Service vs a Lambda. Additionally the Azure Pipelines Terraform Task abstracted out a lot of the nitty gritty for us.
For me I didn’t realize that when writing the TheYAMLPipelineOne I assumed I’d only ever deploy to Azure…..well now I am using it to deploy to AWS. As such I will need to be more specific when naming my job/stages/task to include which cloud the deployment is going to.