Possibility of allowing Chef's omnitruck to install community distibutions

[bobchaos]

As a community member, I would like to be able to use knife bootstrap, test-kitchen and other tools that install Chef Infra and Chef products transparently whether I'm using the official Chef Infra or a community distribution.

Describe the problem

Currently, many tools in the ecosystem makes hard coded calls to omnitruck when it's time to install Chef Infra. Since the URLs are often hardcoded that means there's a lot of surface to cover to allow those tools to individually select a distro. This would also lead to consistency issues where cooks would need to edit the distro they want in many different configuration files.

Software Version

All currently supported OSes!

Replication Case

N/A

Stacktrace

N/A

Possible Solution

I see two broad options (there could be more!): Go through all of these tools and add support for alternate distribution channels, or add support to Omnitruck itself (which would imply that Chef also managed the infrastructure used to deploy alternate distributions).

This support could come in the form of an additonal parameter exposed to the users called "distribution" or perhaps "binary_source", and adding this parameter explicitly would output a warning that while you are using Chef's Omnitruck, you're getting your binaries from a 3rd party source along with any other appropriate disclaimer.

Assuming Chef is comfortable with that notion, I'd be willing to draft a PR.

[bobchaos]

Ramereth pointed out mixlib-install's PRODUCT_MATRIX ( https://github.com/chef/mixlib-install/blob/e930c83b0ee9fe594194dd40ae83c7af9f026fe2/lib/mixlib/install/product.rb#L197 ) as something else that would need to be addressed. It looks like it already supports overriding the source, so we could make it a Dist constant too. Perhaps we could go so far as to externalize the product matrix and feed it in as JSON or something. I'm mentioning it here to minimize the "surface" of this issue but let me know if you prefer I open a separate issue against that repo

[cscobie]

You are struggling with years of software architecture that was never designed or implemented to meet the use case of multiple distributions. I personally have a lot of empathy for that challenge.
However, Chef will not distribute or manage the distribution infrastructure for community or alternate distributions of the software. Therefore it feels like the only path here is to refactor all the tools to be multi-distribution aware. Maybe there are other clever alternatives but at the end of the day it seems like the best alternative is to externalize and parameterize the distribution channel as a singleton so it can be easily switched.

[schisamo]

Just wanted to jump in and add some more context to what @cscobie said. I believe the best way forward here would be:

• Update chef/mixlib-install to be multi-distribution aware.
• The community would then need to run their own instance of Omnitruck whose poller code (which uses mixlib-install under the covers) would be configured in to load the community distribution matrix as it's source of truth.

At this current time Chef Software doesn't have the capacity to take on this work.

[bobchaos]

If Chef's got time to review our PRs we should be able to cook up solutions! We've started looking into options to host our own omnitruck too FWIW

[Tensibai]

Partial workaround by monkey-patching mixlib-install:

module Mixlib
  class Install
    class Options 
      SUPPORTED_PRODUCT_NAMES += %w(cinc auditor cinc-server)

      def initialize(options)
        @options = options
        @errors = []
        
        PRODUCT_MATRIX.product "auditor" do
          product_name "Cinc Auditor"
          package_name "auditor"
          downloads_product_page_url :not_available
        end

        PRODUCT_MATRIX.product "cinc" do
          product_name "Cinc client"
          package_name "cinc"
          downloads_product_page_url :not_available
        end

        PRODUCT_MATRIX.product "cinc" do
          product_name "Cinc client"
          package_name "cinc"
          downloads_product_page_url :not_available
        end
        ENV['PACKAGE_ROUTER_ENDPOINT'] =  ENV.fetch('PACKAGE_ROUTER_ENDPOINT', 'http://packages.cinc.sh') if %w(cinc auditor cinc-server).include? options[:product_name] 
        # Store original options in cases where we must remap
        @original_platform_version = options[:platform_version]

        resolve_platform_version_compatibility_mode!

        map_windows_versions!

        validate!
      
      end
    end
  end
end

I've an idea to make mixlib-install a bit more versatile without breaking changes, I'll try to bake a PR this week on it.