There are many times you may need to share source code, tools, common scripts, template YAML files or other items across projects when developing software. One way to accomplish this is to store the common files in a separate source repository and check out multiple repositories in your Azure DevOps pipeline.
I am a huge fan of Rencore's SPCAF tooling and we use it in all of our projects related to Microsoft 365 customizations. Since we have many projects that require similar custom rule sets, we needed to address sharing some centralized SPCAF rules files across multiple projects. By using Azure DevOps multi-repository checkout capabilities, we can share our license file and custom project rule sets across projects from a central location.
Create a simple YAML build process
First, create a simple Azure Pipeline build process using a YAML file. For simplicity's sake I created a basic .NET Core Console app. Nothing special, just the following:
# Simple .NET Core Console App resources: repositories: - repository: self type: git trigger: - main trigger: - master pool: vmImage: 'windows-latest' variables: solution: '**/*.sln' buildPlatform: 'Any CPU' buildConfiguration: 'Debug' steps: - task: NuGetToolInstaller@1 - task: NuGetCommand@2 inputs: restoreSolution: '$(solution)' - task: VSBuild@1 inputs: solution: '$(solution)' platform: '$(buildPlatform)' configuration: '$(buildConfiguration)'
Get Shared Files from a Separate Azure DevOps Project Repository
You can now add additional repositories to the yaml file. The first one we will add is a reference to another Azure DevOps project. In my example I have a sample project named CommonFilesProject and a default repository names CommonFilesProject. I know, very creative right! Simply add another repository resource after the current projects self repository.
resources: repositories: - repository: self type: git trigger: - main # Azure DevOps access requires correct settings for access - repository: CommonFilesProject type: git name: CommonFilesProject/CommonFilesProject```
From the multiple repository documentation, if you are using another Azure DevOps project, prefix the name with that project's name. My sample case above has the project name and repository name. I did run into an error about repo not existing, but this was due to the pipeline settings not allowing access to the common project. The resolution was to toggle the setting in the consuming project, and then the pipeline completed. Simply click on Project Settings -> Pipeline -> Settings and toggle the following setting.
Note: you can also add additional repositories from the same project, but our use case needed cross project sharing capabilities.
Get Shared Files from a Separate GitHub Project Repository
You can also add a Github project as a connection. This requires a Service Connection to be configured in Azure DevOps Pipelines. For full details see the Manage Service Connections documentation. Simply click on Project Settings -> Pipeline -> Service Connections and create a Service Connection to Github.
Now, edit the yaml file to include checking out the Github repository as well. For github connections, the name is the Github account, personal or organization, and the repository name. In our case, this a private repository in my Github account, pkskelly/sharedbits.
resources: repositories: - repository: self type: git trigger: - main # Azure DevOps access requires correct settings for access - repository: CommonFilesProject type: git name: CommonFilesProject/CommonFilesProject # Github Access required configured Service Connection - repository: SharedBits type: github name: pkskelly/sharedbits endpoint: github.com_pkskelly
We now have three repositories configured to get files from. The current project (CommonFilesConsumerProject), our shared files from another DevOps project repository (CommonFilesProject), and a private Github repository (SharedBits)
Checkout the Repositories
Since we now have more than a single repository, we need to check out each of them, including the current project by using the self reference. Add the following after the variables section.
steps: - checkout: self - checkout: CommonFilesProject - checkout: SharedBits
Once these repositories are checked out, the source is checked out into directories named after the repositories as a subfolder of
(Agent.BuildDirectory) variable location. Let's add an task to output the folders after checking out the files just to show where this location is.
- script: dir "$(Agent.BuildDirectory)\s" displayName: 'Display Build Folders for repositories'
Get the Shared Files in DevOps Tasks
And last, let's use the files from our shared DevOps and Github project code bases checked out in our build. The first task will call a PowerShell script that takes in a file path argument and will echo the content from our second DevOps project. The second task will call the sample file from the Github project.
- task: PowerShell@2 displayName: 'Execute GitHub Project Test' inputs: filePath: "$(Agent.BuildDirectory)\\s\\SharedBits\\Get-SharedFile.ps1" arguments: "$(Agent.BuildDirectory)\\s\\SharedBits\\rules.xml" - task: PowerShell@2 displayName: 'Execute Azure DevOps Project Test' inputs: filePath: "$(Agent.BuildDirectory)\\s\\CommonFilesProject\\Get-SharedFile.ps1" arguments: "$(Agent.BuildDirectory)\\s\\CommonFilesProject\\rules.xml"
Our goal of sharing SPCAF license and rules files is now a simple matter of using the
$(Agent.BuildDirectory) and the location of the files to add the files as arguments or get the content from the files for use in other ways. The image below shows the files all checked out the the
$(Agent.BuildDirectory)\s folder and into named folders for each repository.
And our PowerShell scripts from each remote repository have executed, consumed the rules.xml file path from both, and echoed the xml content for demo purposes. Success! We can now use a centralized license.lic file and reuse common SPCAF rules files.
Sharing files, templates, tools and more across repositories is a simple, yet extremely powerful technique for your DevOps Pipelines. Andrew Connell also has a fantastic example of this process with his Yaml Pipelines for SPFx Projects which enable sharing build template YAML files for SPFx project builds. If you haven't seen this post, it is worth a read to see another great use case of this technique.
As always, HTH and feel free to let me know in the comments below or Tweet at me!