Azure DevOps repos support

[jsburckhardt]

Is your feature request related to a problem? Please describe.
Our project hosts the target repos in Azure DevOps. I can see the tool is currently using go-git which doesn't support he Azure DevOps git protocol (libgit2 is required)

Describe the solution you'd like
Should be able to target a https://dev.azure.com/<sample org>/<sample project>/_git/<sample repo> link

Describe alternatives you've considered

• Mirroring repository || not secure and a pain to maintain
• Inspected how flux is doing it -> for azdo they use libgit2

Additional context

• Flux used to have the same issue -> https://fluxcd.io/docs/components/source/gitrepositories/#git-implementation

[jeff-mccoy]

Interesting problem, thanks for opening an issue. We mainly use AWS/GCP internally and are light on Azure experience. Could you clarify what version of git you would normally use on your machine and what OS/version?

The reason we went with go-git initially was because it was a pure go implementation. The reason that Flux can do what they are doing is they are controlling a very fine-grained build process to produce an OCI image that runs for a controller to handle the git operations using https://github.com/fluxcd/golang-with-libgit2 and https://github.com/libgit2/git2go from what I can see. We don't have that luxury because we need to support multiple architectures and OSs for the CLI and so running in a very controlled environment like a container isn't possible.

Thinking out loud it might be worth just falling back to https://pkg.go.dev/os/exec#example-Cmd.Run and use the host-os git instead for zarf package create. This is also the recommendation mentioned for Pulumi facing the same issue.

A good summary from someone who tried to fix this in go-git is here: buildpacks-community/kpack#213 (comment). The related PR in that tool that actually swapped go-git -> libgit2 is here: buildpacks-community/kpack#636, but note the ability to control the environment with a container.

[jeff-mccoy]

One other note, if you could run zarf version and zarf package create -l=trace for any additional info that would be great.

[jeff-mccoy]

I'm getting a 400 on a public repo: https://dev.azure.com/me0515/_git/zarf-public-test.

[jsburckhardt]

Yeah, the error is vague, but I saw that before when working with Azure DevOps and go-git issue. The main challenge comes with the multi_ack required by AzDO.

[jeff-mccoy]

Agreed re: multi_ack. What are your thoughts on falling back to a host-provided git binary?

[jsburckhardt]

Agreed re: multi_ack. What are your thoughts on falling back to a host-provided git binary?

Sorry mate didn't follow. What do you mean by host-provided git binary? Something like download zip file?

[jeff-mccoy]

Yeah sorry, so basically my proposal (only on the zarf package create side) would be to attempt to us the internal go library to do the pull, and then if that errored out for any reason, to then attempt to simply call git clone or similar using the machine's git installation/configuration if possible. Does that make sense?

[jeff-mccoy]

@jsburckhardt we're hoping pulumi/pulumi#471 will resolve this (it currently passes for our test azure repo).

[jsburckhardt]

@jsburckhardt we're hoping pulumi/pulumi#471 will resolve this (it currently passes for our test azure repo).

Hi mate, it does solve the issue as far as Git is in the host. I'm working in a small proposal to include git2go in zarf. The same way flux mates use it. The annoying thing is compiling libgit2 but I'll put the details in the doco.

Cheers