One YAML Pipeline for CI/CD

Introduction

One can leverage YAML If Expressions to dynamically load the required stages/jobs/tasks templates, this will lead to one YAML pipeline for CI/CD. This is important to build clean and DRY (Don’t Repeat Yourself) pipelines. This post will walk through how I leverage this when using one Pipeline for both CI and CD in Azure DevOps (ADO)

Background

Prior to YAML Pipelines, Classic Build and Classic Releases enforced the notion that the build and the deployment were two separate activities. This process isn’t ideal for YAML Pipelines as we’d like to use the artifact from the build for each of the deployment stages. Thus, a YAML Pipeline should always build and store an artifact. If we broke this into a separate build and deployment pipeline then we’d be managing two pipelines vs one. This becomes a risk as the build associated with CI may not have the same steps as our CD.

So….how do we reuse the same pipeline for CI and CD activities?

If/Else Expression

The If Expression is the answer to this question. This is different then the condition property at a given stage, job, task. A condition will still be evaluated every time and will skip over the Stage/Job/Task. This could lead to a messy user experience as pipeline steps are still being loaded though never used.

I typically use condition when evaluating the status of multiple predecessor stages/jobs/tasks or if we need to evaluate a variable that is only read at runtime, such as a secret or something we won’t know until the stage executes.

When leveraging if statements these are evaluated at pipeline execution and if the expression isn’t met then the subsequent code will never be loaded.

Now to tie this back to our CI/CD process.

CI/CD Stages

Leveraging YAML Templates is the preferred way to achieve and scale this; however, not required. If interested in seeing how to leverage YAML templates can check out any of the following blog posts: YAML Deployment Pipelines or Leveraging YAML Pipelines: The New Deployment Architecture.

stages:
- template: stages/terraform_build_stage.yml@templates
  parameters:
    environmentObjects: ${{ parameters.environmentObjects }}
    templateFileName: ${{ parameters.templateFileName }}
    serviceName: ${{ parameters.serviceName }}

- ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/main')}}:
  - template: stages/terraform_apply_stage.yml@templates
    parameters:
      environmentObjects: ${{ parameters.environmentObjects }}
      templateFileName: ${{ parameters.templateFileName }}
      serviceName: ${{ parameters.serviceName }}

In the example above this pipeline will execute the build stage every time while only creating deployment stages when the source branch is main. This condition though could be set to any variable so it doesn’t necessarily have to be the main branch. I have set this to accommodate a trunk based branching strategy.

To see more about this specific use case can read my blog on Terraform, CI/CD, Azure DevOps, and YAML Templates or check out my GitHub TheYAMLPipelineOne.

So what does this look like from the UI perspective?

Results

Can see the run 2021209.1 above was associated with a PR. This pipeline only executed the build. 20221209.2 can see ran a deployment after the merge was completed. The key difference here between the if and the condition statements is the deployment jobs never appear as stages as part of the pipeline. If setting the condition as part of the stage the pipeline will show skipped on the stage.

Conclusion

The if tool is one tool in the toolbelt which can be used for one YAML Pipeline for CI/CD. It can also be used for subsequent things like jobs and tasks which only enhance the DRY capability of your pipelines. Feel free to check out more examples on TheYAMLPipelineOne repository.