Building CI/CD maturity for report development

Key Takeaways

• CI/CD is for everyone: Even if you aren’t a professional programmer, incremental improvements in process can reduce defects and frustration.
• Start by making changes smaller: Use PBIP, TMDL, and very basic Git use to make your changes smaller and more manageable.
• Separate dev, test, and prod: Describe the final takeaway, you can include more than 3 bullet points, if necessary.
• Add automation: Describe the final takeaway, you can include more than 3 bullet points, if necessary.

This summary is produced by the author, and not by AI.

CI/CD practices benefit everyone

It’s tempting to assume that Continuous Integration / Continuous Delivery (CI/CD) is only for large enterprises or professional software companies. In this article, we argue that many CI/CD and related practices form a maturity ladder you can climb. Even basic steps can be useful and this is true even if you are the only developer in an organization. You should still wear a seatbelt even if you aren’t a professional racecar driver.

The general idea with CI/CD is if you can ship smaller pieces of work with automated testing at each stage, it becomes easier to prevent defects and diagnose the cause of those defects. CI/CD also means tracking those changes and having dedicated testing environments. Automated tests means that is possible to keep releases in a reliable and known good state. For reports this automated testing means scanning for best practices, confirming that measures report expected values, confirming that visuals still render, and more.

You can imagine how a strong CI/CD process can help with Power BI reports. If all you have for your report deliverables are PBIX files, changes tend to be much larger and grouped together. If a user finds an issue with a report, you may have to hope you have an old copy of a PBIX lying around and trying to fix the change can be quite challenging. Even then, you may not remember what you changed, making it harder to pinpoint the root cause.

In contrast, if you make use of the PBIP format and have all of it stored in source control, it’s much easier to track each and every small change, down to the line of code. It then becomes easier to track “what changed?” when a user says the numbers are wrong. You can revert back to an earlier version or to surgically undo a single error, all without having to lose work. Source control also makes it much easier to have separate development, test, and production environments so that you can catch bugs or issues before they get to production.

In the rest of this article, we are going to cover several layers of CI/CD maturity for semantic models ranging from pewter to gold, the idea being that even small incremental improvements can provide significant benefits. The highest levels require buy-in from your team and organization, which can take time and work. But you can start taking small steps today with a minimum of effort.

Pewter-level maturity

This level is whatever most folks do when they start out, typically storing PBIX files somewhere locally and publishing from Power BI Desktop. This presents a number of risks. Changes are generally larger, unless you meticulously keep copies of every change. If you do, each copy can take up significant storage space. One could argue that storing dated copies of PBIX is a very crude form of version history.

A slightly more robust situation is using OneDrive or SharePoint for version history and management. Both naturally keep a basic history of updated files and SharePoint has support for checking in and checking out files to control who is allowed to make edits. Early on in the life of Power BI (think 2017), this was the encouraged way of tracking changes with Power BI because we didn’t have the PBIP format yet.

In fact, Microsoft supports automatically publishing reports from OneDrive or SharePoint as well as refreshing semantic models. This is a wonderful feature for anyone who is uncomfortable or inexperienced with coding. However, if you are willing to branch out a bit we strongly encourage working up to the next level of maturity and learning the basics of Git source control. If you are using AI agents in any capacity, this is an absolute must so they are less likely to wreck your work.

[[Explain how they can wreck your work]]

Additionally, users at this level may have a test or development workspace where they deploy to first. This is a key first step on the way to CI/CD, since it’s much better to have multiple people testing reports before they are deployed to production. Some people may even use Fabric deployment pipelines, which are a GUI-first way of moving objects in Microsoft Fabric from workspace to workspace. This is a convenient tool for moving objects from workspace to workspace. It can identify differences between workspaces and allows for basic environmental settings for different workspaces, such as data sources.

Bronze-level maturity

Moving up from ad-hoc and messy version history to proper source control means using formats such as PBIP and TMDL (or Tabular Editor’s database.json + supporting files format for just the model) that are better supported with source control. The reason these formats are better is that they store the components of the semantic model and report as individual text files. This makes it easy to compare the files to see exactly which lines have changed. This is sometimes called taking a “diff” of the files, short for difference.

This bronze level also means using a source control tool like Git as slightly more advanced version control and not worrying about the full feature set, such as branching and merging. While Git allows you to make multiple parallel copies of your report, there is no reason you have to learn how to do that starting out. The goal here is to make the change tracking smaller and more targeted, not to emulate a full software development lifecycle.

One thing to understand when you start using Git is that code is stored in a code repository, often a local folder that has been initialized by Git with some tracking files and folders. File changes are saved in checkpoints called commits. A commit is an atomic unit of work and can be as small as changing a formatting string or measure definition. Commits can optionally be “pushed” to a remote host, such as GitHub for backup and collaboration purposes. This is really all you need to know to start using Git as local version control.

[[git screenshot]]

At this stage, you also should definitely have separate test and deployment workspaces. You may also optionally want a third development workspace. What is important is that you are moving testing and error detection earlier and earlier in the process. It’s difficult to do this if you only have a production workspace.

Using deployment pipelines at this point is a good choice for moving reports and models from development to test to production. Azure Pipelines may make more sense if your organization is already set up with Azure DevOps. Azure Pipelines are more powerful and flexible, but require more setup and research.

[[note about workspace mode]

Part of moving testing earlier means doing validation testing in your development environment, which may mean on your local machine. One great step is to start using the Best Practices Analyzer before you publish a report. This can be done with Tabular Editor 3 or the TE CLI, whichever you prefer. If you are using AI agents, the TE CLI is ideal because the Best Practices Analyzer is on by default for deployments, preventing your agent from deploying a model that doesn’t follow your rules.

Silver-level maturity

The next level up generally requires getting buy-in from the rest of your team, whereas no one can really stop you from using Git and Tabular Editor locally, aside from the IT department perhaps.

The first step is choosing a source control provider. If you are using Microsoft Fabric’s built-in Git integration, the supported options are Azure DevOps and GitHub. By configuring a workspace to use a code repository, changes you make in the service will automatically be tracked and you can commit those changes from the service. This means it’s possible for developers on your team start to getting the benefits of source control without having to learn the commands.

If you are using a source control provider not integrated with Fabric or not supported by Fabric, then you’ll want to use the TE CLI and XMLA endpoints to deploy semantic models as part of your CI/CD process. To deploy reports you will want to use the Power BI Rest APIs. This approach makes the most sense when your organization is already well-established with a given provider. If you are starting fresh, it makes sense to go with a provider supported by Fabric instead.

Part of setting up source control for your team also requires choosing a branching strategy. A branching strategy is a plan for how you will take the source code and create isolated copies, called branches, and then recombine them, called merging. This allows for developing and testing small changes in isolation, reducing the risk of deploying partial work along with completed work. If you are used to working in Power BI Desktop, this will be a fairly novel concept since copying a PBIX file to make changes and then copying the changes back into the original file is tedious and risky.

With Fabric Git integration, there are at least three workspace and branching strategies you could take. First, you can maintain individual workspaces for development, testing, and production. While this is the simplest to set up, it doesn’t scale particularly well to large teams since you are all working in the same workspace with a risk of other people accidentally committing your work.

Second, you could create a development branch for each developer with a branched workspace tied to it. This reduces contention among developers but still requires each developer to be cautious about what work they commit and when. Otherwise, you get back to the issue of submitting big chunks of work at the same time.

A third option is to use feature branches and branched workspaces. This is where you create a new workspace for each meaningful chunk of work. Feature branches are extremely common in software development, but it is a bit more clunky in Fabric because you are provisioning an entirely new workspace. This approach is the more robust than single developer branches but can be overwhelming. It also may require support from IT, since developers may not have permissions to make new workspaces.

Lastly, this is a good stage to start adding very basic source control gates. A simple one is using the TE CLI to enforce that models pass best practice analyzer rules. You can customize those rules or choose to only gate on the most severe violations. Doing this requires setting up an Azure DevOps Pipeline, Github Action, or similar.

Gold-level maturity

Reaching a gold level maturity is mainly about having a high level of consistency in applying your policies and exploring more advanced techniques. It also means treating semantic model development closer to professional software development. It is less prescriptive than the previous levels because you have more options and choices.

For a branching strategy, we recommend considering two combined techniques called GitHub Flow and Octopus Merge. Github Flow has short lived feature branches and when you want to make a merge, you make a pull request where any reviewers can look at the changes and leave comments before choosing to merge it in.

Octopus Merge adds a testing branch that is constantly running automated tests. When a developer wants to merge their feature work into the main branch, a continuous integration pipeline runs tests against the feature work.

This combination has a few benefits:

• Main is always in a good state, because tests have to pass before being merged in
• The test branch represents the combination of all the active work. This catches errors that only occur when features are combined.
• Merge conflicts, where two changes can’t be automatically combined, are rarer because feature branches are short lived. This means it’s less likely for two people to work on the exact some code at the same time.

Another piece is a matter of expanding out the types of automated tests you run. We already mentioned running the Best Practice Analyzer with the TE CLI. You can also do the following:

• Enforce formatting with te format
• Run DAX assertions against a live model to make sure certain formulas return the right value. You can do this with te test run or you can make something simple with Fabric notebooks and Semantic Link.
• Run schema checks if you are using SQL server as your data source.

Further reading

• Power BI implementation planning: Deploy content (Microsoft).
• Example link to an article. This article is by someone and about something, and reading it will teach you things that are important and useful.
• Another example. Explanation follows.

In conclusion

The conclusion is mandatory. You cannot publish without a conclusion. This is 1 paragraph of concluding statements. Do not introduce any new ideas here. Summarize the key points and provide actionable takeaways for the reader.