When using Azure data factory or Azure synapse with Git, sometimes some “garbage” is left behind on ADF live mode.
For example, you rename a pipeline, you do that on a new branch, and then do a pull request to merge the change to the collaboration branch (“master”).
Then you publish the change from master to live mode.
But then, when you look at the live mode, you see 2 pipelines!
One with the old name and one with the new name.
What just happened? Well, ADF merges changes from master to live mode, it adds new objects and updates existing objects, but it does not delete objects that were removed.
You cannot delete these “leftovers” yourself, since publishing is disabled in live mode if you are working with GIT.
To tidy up your environment here is what you can do:
Go to: manage --> Git configuration, and then click on “overwrite live mode”
ADF will take the version from your collaboration branch (‘master’) and force it into live mode.
Something similar can happen if you set up a release pipeline to move objects to a Test\Production environment. On the other environment, the release will create a new pipeline, instead of renaming the old one.
There is a sample PowerShell script from Microsoft to add to your release pipeline. This script supports deleting objects.
Unfortunately, it only support ADF and not Synapse pipelines for now.
Comments