Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(BOLT-1589) Support installing any git-based module #3115

Merged
merged 4 commits into from
Jul 27, 2022

Conversation

beechtom
Copy link
Contributor

@beechtom beechtom commented Jun 23, 2022

This updates Bolt to support installing git-based modules from any
source. Now, Bolt will attempt to download metadata directly from GitHub
or GitLab for public repositories and fall back to cloning the git
repo's metadata if that fails.

This also updates the spec resolver to no longer calculate a SHA based
on the ref provided in project configuration. Previously, Bolt would
calculate a SHA and write it to the Puppetfile to pin the module to a
specific commit when modules are installed. However, doing this is
difficult and time-consuming with the move to supporting any git-based
module.

!feature

  • Support installing any git-based module
    (#3109)

    Bolt now supports installing any git-based module. Previously, Bolt
    only supported installing and resolving dependencies for modules
    hosted in a public GitHub repository. Now, Bolt will check both GitHub
    and GitLab before falling back to cloning a module's metadata using
    the git executable. This allows users to install modules from
    private repositories, or from locations other than GitHub.


This fixes a bug where the proxy configured in module-install.proxy
was not being set when retrieving metadata for git-based modules.
Previously, the proxy was only configured when resolving and
installing modules.

!bug

  • Configure proxy when retrieving metadata for git-based modules

    Bolt now configures a proxy when module-install.proxy is set and it
    retrieves metadata for git-based modules. Previously, Bolt was only
    configuring a proxy when it resolved and installed modules.

@beechtom beechtom linked an issue Jun 23, 2022 that may be closed by this pull request
@beechtom beechtom force-pushed the 1589/git-modules branch 2 times, most recently from e690e74 to 269235d Compare June 23, 2022 17:18
@beechtom beechtom marked this pull request as ready for review June 23, 2022 19:07
@beechtom beechtom requested a review from a team as a code owner June 23, 2022 19:07
@nicklewis
Copy link
Contributor

Not calculating a SHA means the Puppetfile no longer refers to a specific version of the module, correct? In the case where the ref is a branch, that is.

@beechtom
Copy link
Contributor Author

@nicklewis Correct. I can add that back in, just a bit more work to get it working for GitLab & cloned sources.

@beechtom beechtom marked this pull request as draft June 27, 2022 22:34
@beechtom beechtom force-pushed the 1589/git-modules branch 2 times, most recently from f8fd164 to 8b5c9b6 Compare June 27, 2022 23:19
@beechtom beechtom added the Do Not Merge Work that should not be merged yet. label Jun 27, 2022
@beechtom beechtom force-pushed the 1589/git-modules branch 2 times, most recently from e3e95cb to efdd65b Compare June 27, 2022 23:35
@beechtom beechtom marked this pull request as ready for review June 27, 2022 23:49
@beechtom
Copy link
Contributor Author

Blocked by glennsarti/puppetfile-resolver#38

Copy link
Member

@donoghuc donoghuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the "in person" walkthrough on this. Looks good, and we have some thoughts for improving this moving forward.

This updates Bolt to support installing git-based modules from any
source. Now, Bolt will attempt to download metadata directly from GitHub
or GitLab for public repositories and fall back to cloning the git
repo's metadata if that fails.

This also updates the spec resolver to no longer calculate a SHA based
on the ref provided in project configuration. Previously, Bolt would
calculate a SHA and write it to the Puppetfile to pin the module to a
specific commit when modules are installed. However, doing this is
difficult and time-consuming with the move to supporting any git-based
module.

!feature

* **Support installing any git-based module**
  ([puppetlabs#3109](puppetlabs#3109))

  Bolt now supports installing any git-based module. Previously, Bolt
  only supported installing and resolving dependencies for modules
  hosted in a public GitHub repository. Now, Bolt will check both GitHub
  and GitLab before falling back to cloning a module's metadata using
  the `git` executable. This allows users to install modules from
  private repositories, or from locations other than GitHub.
This updates documentation for installing git modules to no longer say
only public GitHub repositories are supported.
This fixes a bug where the proxy configured in `module-install.proxy`
was not being set when retrieving metadata for git-based modules.
Previously, the proxy was only configured when resolving and
installing modules.

!bug

* **Configure proxy when retrieving metadata for git-based modules**

  Bolt now configures a proxy when `module-install.proxy` is set and it
  retrieves metadata for git-based modules. Previously, Bolt was only
  configuring a proxy when it resolved and installed modules.
Copy link
Member

@donoghuc donoghuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor suggestions, but overall this looks good!

This updates the `GitSpec` class to resolve the SHA for a git module's
ref.
@beechtom
Copy link
Contributor Author

@donoghuc Updated with suggestions.

@beechtom beechtom removed the Do Not Merge Work that should not be merged yet. label Jul 26, 2022
Copy link
Member

@donoghuc donoghuc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! How do we want to handle bumping the puppetfile resolver dep in the runtime? https://github.com/puppetlabs/puppet-runtime/blob/master/configs/components/rubygem-puppetfile-resolver.rb

I think it may be simplest to bump it in the runtime and then once that promotes merge this.

@beechtom
Copy link
Contributor Author

@donoghuc Here's the PR to bump puppetfile-resolver (and other gem components): puppetlabs/puppet-runtime#561

@beechtom beechtom merged commit ada6800 into puppetlabs:main Jul 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bump puppetfile-resolver to 0.6.0
3 participants