Automate Fabric Deployment Pipelines with DevOps
On this article, I'll explain how to use Microsoft's DevOps extension Power BI automation tools to automate the execution of your Fabric Deployment Pipelines in order to deploy all/selected items in a Power BI pipeline to the next stage.
Welcome 😁! If you already familiar with Fabric's new features Git-integration & Deployment pipelines you probably would like to go straight to the solution, if not, I STRONGLY suggest you to start my prev. article... in a way, this would be an extension of it.
Hector Sven
Setting the stage
If you read my prev. article Version-control & CI/CD your Power BI reports: Using Fabric's Git Integration & Deployment pipelines you may have noticed that it required lot's of mouse clicks 😬 ... that is equivalent to say that there was a lot of manual labor, although this is not bad by itself, we can definitely automate lot's of steps to reduce errors and make this process less prone to errors and less time-consuming!
Considerations and limitations
As shown on my prev. article, on workspaces that are Git-Integrated, you can for instance, commit changes to the collaboration branch (1) directly from your local branch, commit and push back to the source.
When this happens, you'll receive a message from your workspace indicating "You have pending updates from Git. We recommend you update the incoming changes and then continue working.", hence, click the Update all button (1)
This is why I wanted to make sure that our automation pipeline included a tasks that invoke whatever Rest API call the Update all button does... and I found it! 😁
In MSFT article Automate Git integration by using APIs and Azure DevOps lists a group of Git integration API functions, among them: Update the workspace
There's a catch tough... currently Service principals aren't supported 😬.
There're good and bad news on this one...
- The good news. We can use Azure's EntraID User Account to authenticate instead of a service principal, we are going to use Example 2: Connect to Azure using organizational ID credentials from cmdlet Connect-AzAccount
- The bad news. This scenario works only when the user does not have multi-factor auth turned on; creating a user without multi-factor auth requires some additional configurations at EntraID service.
I'm confident there's plenty info out there on this topic and I'll create a whole new article to describe how to do this... for the time being, you should require an Automation Account (user without multi-factor auth) before proceeding to the Solution 😉
This account should also be properly licensed, it needs Power BI Pro as minimum; I'm using my 59 days trial premium license for demonstration purposes.
Solution
Let's make it happen 😎👍
Grant Automation Account access to DevOps organization
From your Azure DevOps (AzDO) Organization, go to Users (1) and add your Automation Account (2) ... it is important that you grant Basic Access level (3), this will allow the account to "[...] access to all features to manage code using Git repositories" (Ref. About access levels), grant this account also to the Project Contributors group (4)
Install MSFT Extension Power BI automation tools
Authored by Microsoft, this extension is meant to "Automate release management tasks related to Power BI deployment pipelines", here are a few examples of what can be done using the extension:
- Manage Power BI pipelines from start to finish, including creating a pipeline, assigning a workspace to any stage, deploying and deleting the pipeline.
- Assign users to a pipeline.
- Assign users to a workspace.
- Deploy all/selected items in a Power BI pipeline to the next stage.
So, go ahead an and install it, go to your Organization settings/Extensions (1= and search for Power BI automation tools in the marketplace (2), is free! 😉
Create a service connection
From your Projects settings/Service connection, search for Power BI and click Next
Configure the service connection using Username and Password Auth. method (1), automation account user name & password (2); I'm using a very simple naming convention to name my connections (3).
Allowing Service Principals to use Fabric API
Go to your Power BI Cloud service, click on Settings (1) > Admin Portal (2) and from Tenant settings (3) scroll down until you locate Developer settings section (4), enable the toggle switch (5) for Service principals can use Fabric API's
Grant Automation Account access to Fabric Workspaces
Go to Development's Workspace and gran the Admin role (2) to the Automation Account (2), do the same towards Production's Workspace.
Grant Service Connection Security principal access to Fabric Deployment
We would also need to grant Admin role to the Automation Account towards the Deployment Pipeline.
Last but not least, the AzDO Yaml pipeline!
On the next part of the article, I'll explain just the most relevant parts of the code
Remember that "Update all" button we talked about?
I'm fetching from the Key Vault the account password for my Automation account (1), then, connecting to Azure using organizational ID credentials (2) with this I'm able to generate an access token (3) to call REST API endpoints; we need to additional pieces of information from the workspace's current Git status: Commit Hash and Branch Head (4)... hence, we call for another REST API endpoint (5).
Finally, we can now invoke endpoint behind Update all button: updateFromGit (6)
Next is to trigger our Fabric Deployment pipeline (1) using the connection we created and we finally by triggering the refresh of our Dataset (2)
Would you like to see this process in action?
Check out this video y prepared for you
Conclusion
Microsoft is heavily investing on its Fabric, the platform is growing and evolving at speeds I've did not expected; this also creates interesting opportunities to get ahead of the wave and to strategically position yourself for the future.
Call to action
Give it a try and let me know how it goes... wish you the best of lucks and hope you find it useful
