The Anatomy of a Resilient Azure DevOps YAML Pipeline (Series)

Introduction

Back on 2021 when I started to work with Azure DevOps (AzDO) services, YAML pipelines were something relatively new [April 2020 - Announcing General Availability of YAML CD features], I already had developed a few pipelines using the Classic approach (UI-based) but It only took one YAML pipeline to convince me that the days of Classic pipelines were numbered; the learning curve to get a good grasp to the YAML variety could be steep (talking from personal experience).

I'll guide you through configuring Azure DevOps services and constructing a YAML pipeline that is not just robust, but also easy to maintain, flexible and versatile... but the pipeline is just the "last mile" of a long road, that's why I divided this series into 3 parts... my target audience for Part 1 are beginners and provide guidance to get the Azure and DevOps accounts, Part 2 is for intermediates and how to work with code repositories and my suggested branching strategy, but I'm including the topic of Infrastructure as Code (IaC) using BICEP, and Part 3 I would explain the code behind the YAML pipeline in detail and how to use it to achieve a multi-environment deployment.

Setting the stage

An Azure DevOps pipeline is a powerful and flexible automation tool that allows you to automate the building, testing, and deployment of your code.

In order to create and execute our AzDO YAML pipeline, we are going to need the following accounts, services & components in place:

1. Azure Account & subscription. Since we are going to automate the creation of a few resources in Azure using this YAML pipeline, we need and account & subscription
2. AzDO Account. Access to Azure DevOps is available through the Azure DevOps Services web portal
3. AzDO project. The code and pipelines are organized within projects in Azure DevOps.
4. Repository Hosting. AzDO supports various version control systems, we are going to use a Git-base repository
5. Codebase. Our application's source code that we want to automate! It could be a Database (Provisioning Database Security thru Visual Studio Database Projects and Deploying via DevOps YAML Pipelines), a web app, Infrastructure, Power BI report, Python code, you name it!! In this article, I'm going to deploy Azure Shared Infrastructure (Handling Infrastructure in Azure: Shared vs Data-Product) using BICEP
6. Service Connections. The pipeline is going to connect to the Azure Account where it will provision a set of of resources defined in BICEP (IaC), hence, we need a service connection to securely store credentials and connection details.
7. Environment. It is another service that would help us model and manage the various stages of your application lifecycle
8. The YAML file (pipeline). YAML file that defines your pipeline, it outlines the steps, jobs, and configurations for your build and/or release process.

Where to start?

I split this series in tree articles thinking on the following audiences:

• People with a good understanding on cloud computing in general, they have worked previously with other platforms like AWS or Google cloud but they are new to Azure, if your among this group, I'll suggest start from Part1. Azure Account vs DevOps Account of these series and move afterwards to Parts 2 & 3.
• People with experience and Azure and with all most likelihood with Azure DevOps services from a developer's perspective; you know and most likely work with Git or Bitbucket (or other), I'll suggest you to start from Part 2. Repository hosting & branching strategy and move afterwards to Part 3.
• Experienced people working in Azure, Azure DevOps and/or GitHub interested in automating application deployment using AzDO Pipelines or they already have some built using the Classic method but they want to migrate to the YAML kind, go directly to Part 3. Resilient Azure DevOps YAML Pipeline
• If you are somehow in-between of any of these audiences, explorer any/all of the articles in this series, I'm confident that you will find some useful.

Part 1. Azure Account vs DevOps Account

On this article we are going to cover the first 3 items of the list: #1 (Azure Account & subscription), #2 (AzDO Account) and #3 (AzDO Project); it is NOT a step-by-step guide (lot's of them already on the internet) but rather some advises, tips & tricks for you to get the most of your accounts 😉

Part 2. Repository hosting & branching strategy

I'll be covering item #4 (Repository Hosting) and #5 (Code base), on this I'll give priority on branching strategies and my recommended approach.

Part 3. Resilient Azure DevOps YAML Pipeline

I hope items #6 (Service Connections), #7 (Environment ) & #8 (The YAML file) are going to the main course of these series; I'll explain each part of the pipeline and the variables it uses and the rational behind its design, it will be a ride, but I hope it would be worth it 😁