Site icon John Folberth

Multi-Stage YAML Pipelines in a Microservice Architecture. Making Life Easier

John Folberth

MultiStage Pipelines can make deployments much much easier. Be sure to check out my previous post an overview of YAML Pipelines in Azure DevOps

A common question that tends to arise when dealing with these pipelines is how to embrace the concept when implementing microservices. For those that are not aware microservice architecture is an architecture structure where there may be multiple projects or endpoints that are catered to a small (micro) function. This is opposed to how applications used to be very monolithic. There are quite a few positives to architecting solutions this way. The least of which is the ability to deploy/update small sections of code, be able to better scope changes, etc.. Here is some more information on microservice design patterns.

How this design patterns translates into a deployment strategy is that each endpoint/project should follow a very close set of steps for it’s deployment and maintenance. The basic YAML Pipeline will say that each service will have it’s own pipeline…..That’s is great; however, what if we want to maintain the ability to easily redeploy all services. This would happen say when doing major software version upgrades, standing up a new environment, an environment refresh, etc… Well can’t we have both? The ability to deploy a single service will also deploying an entire platform? Enter what I am referring to as a “Viewer-Controler” YAML deployment strategy.

In an ordinary approach to a YAML Multi-Stage deployment pipeline the pipeline would look like:

resources:
  repositories:
  - repository: repositoryTemplate
    type: git
    name: JF Demo/YAMLTemplates
trigger:
  branches:
    include:
    - master
pool:
  vmImage: windows-latest
variables:
- name: regionNameEUS
  value: East US
- name: regionabrvEUS
  value: eus
- name: regionNameWUS
  value: West US
- name: regionabrvWUS
  value: wus
- name: environmentNameDEV
  value: dev
- name: environmentNameTST
  value: tst
- group: DemoApp
- name: solutionPath
  value: JohnOSTApp
- name: testAppProjectName
  value: TestApp
- name: testAppName
  value: johnostdemoapp
- name: secondAppName
  value: johnostdemoapp2
- name: secondAppProjectName
  value: SecondApp
stages:
- stage: Build_TestApp
  jobs:
  - job: Build_TestApp
    dependsOn: []
    steps:
    - task: DotNetCoreCLI@2
      displayName: dotnet restore
      inputs:
        command: restore
        projects: $(Build.SourcesDirectory)/$(solutionPath)/TestApp**/*.csproj
    - task: DotNetCoreCLI@2
      displayName: dotnet build
      inputs:
        projects: $(Build.SourcesDirectory)/$(solutionPath)/TestApp**/*.csproj
        arguments: --configuration Release
    - task: DotNetCoreCLI@2
      displayName: dotnet test
      inputs:
        command: test
        projects: $(Build.SourcesDirectory)/$(solutionPath)/TestApp.Tests/*.csproj
        arguments: '--configuration Release --collect "Code coverage" '
  - job: Publish_TestApp
    dependsOn:
    - Build_TestApp
    condition: and(succeeded(),eq( true, 'true'))
    steps:
    - task: DotNetCoreCLI@2
      displayName: dotnet publish
      inputs:
        command: publish
        publishWebProjects: false
        projects: $(Build.SourcesDirectory)/$(solutionPath)/TestApp**/*.csproj
        arguments: --configuration Release --output $(build.artifactstagingdirectory)
        zipAfterPublish: True
    - task: PublishBuildArtifacts@1
      displayName: 'Publish Artifact: drop'
........

This same block would be unique for each pipeline. This would work in successfully deploying at the pipeline level. Sure we can also do a better job of leveraging templates in our process; however, still how would we go about deploying the entire platform? This can still be a puzzle.

Introduce the Viewer-Controller deployment design. These terms should sound familiar if you have background in an Model-Viewer-Controller (MVC) design pattern in application development. Essentially same concept with Viewer-Controller. Our Viewer here will contain the necessary components to define our pipeline and we leverage the Controller to do the heavy lifting and orchestration of calling subsequent templates. Thus when we need to deploy an entire platform we have a Viewer of calling all the Controllers. Everything else is built for us and we can guarantee with 100% certainty that each component is being deployed the exact same way in an individual service deployment vs an entire environment deployment.

So what does this look like for your YAML files?

The Viewer YAML for an individual pipeline may look like:

trigger:
    branches:
     include:
       - YAML_Pipeline
   
pool:
    vmImage: 'ubuntu-latest'

resources:
  repositories:
  - repository: YAMLTemplates
    type: git
    name: Test Repo/YAML_Templates
   
name: $(TeamProject)_$(Build.DefinitionName)_$(SourceBranchName)_$(Date:yyyyMMdd)$(Rev:.r)
stages:
- template: webapp1-template.yml

Our webapp1-template would look like:

parameters:
- name: serviceName
  type: string
  default: WebApp1

stages:
- template: stages/build.yml@YAMLTemplates
  parameters:
    environmentName: Build
    serviceName: ${{ parameters.serviceName}}
- template: stages/env_deploy.yml@YAMLTemplates
  parameters:
    environmentName: Dev
    serviceName: ${{ parameters.serviceName}}

Can see our template is going to do all the heavy lifting in this scenario and call a set of shared YAML tasks in a shared repository. This is key as our microservices may lie in different repos. As such our reusable templates should be in their own repo as well. The only difference will be what we pass into the build/deployment process. i.e. artifacts, publish settings, etc..

So this is great what if we want to deploy all the micro services. We’d set up a Viewer similar to the one outlined about; however our Controller will look like:

trigger:
    branches:
     include:
       - YAML_Pipeline
   
pool:
    vmImage: 'ubuntu-latest'

resources:
  repositories:
  - repository: YAMLTemplates
    type: git
    name: Test Repo/YAML_Templates
   
name: $(TeamProject)_$(Build.DefinitionName)_$(SourceBranchName)_$(Date:yyyyMMdd)$(Rev:.r)
stages:
- template: webapp1-template.yml
- template: webapp2-template.yml

See just one line of code! That’s it! Now to get to this level if you noticed we made some default values in our Controller. Which is key for saving us some space; however, we can also override certain things. Let’s say we have a parameter to denote the region we are going to deploy to. Set that up as a defaulted parameter in our Controller and if we want to do something multi region just call the Controller and override the parameter.

Next in the series I’ll show some ways to organize and outline YAMLTemplates that we may look at using across multiple repositories.

Exit mobile version