Unable to use NooBaa as Storage Backend

[kurtismullins]

During testing, it was discovered that NooBaa does not work using the 0.0.8 operator. It appears that it's possible some configuration data is missing when running the setup application.

Example CR

spec:
  quay:
    registryBackends:
      - name: rhocs
        rhocs:
          hostname: CENSORED_IP
          port: 80
          secure: false
          accessKey: CENSORED
          secretKey: CENSORED
          bucketName: quay

Operator Pod Log Entry

{"level":"error","ts":1576051727.7906559,"logger":"controller_quayecosystem","msg":"Failed to Validate Component","error":"registry-storage Validation Failed: Missing required parameter(s) for storage rhocs","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/tmp/quay-operator/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.51.al/redhat-cop/quay-operator/pkg/controller/quayecosystem/setup.(*QuaySetupManager).SetupQuay\n\t/tmp/quay-operator/pkg/controller/quayecosystem/setup/setup.go:217\ngithub.51.al/redhat-cop/quay-operator/pkg/controller/quayecosystem.(*ReconcileQuayEcosystem).Reconcile\n\t/tmp/quay-operator/pkg/controller/quayecosystem/quayecosystem_controller.go:180\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/tmp/quay-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/tmp/quay-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:192\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/tmp/quay-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:171\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/tmp/quay-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/tmp/quay-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/tmp/quay-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

Config Tool Log Output

2019-12-11 08:08:33,952 [51] [ERROR] [util.config.validator] Validation exception
Traceback (most recent call last):
  File "/quay-registry/util/config/validator.py", line 78, in validate_service_for_config
    VALIDATORS[service](validator_context)
  File "/quay-registry/util/config/validators/validate_storage.py", line 18, in validate
    providers = _get_storage_providers(config, ip_resolver, config_provider).items()
  File "/quay-registry/util/config/validators/validate_storage.py", line 52, in _get_storage_providers
    raise ConfigValidationException('Missing required parameter(s) for storage %s' % name)
ConfigValidationException: Missing required parameter(s) for storage rhocs
gunicorn-config stdout | 2019-12-11 08:08:33,952 [51] [ERROR] [util.config.validator] Validation exception
Traceback (most recent call last):
  File "/quay-registry/util/config/validator.py", line 78, in validate_service_for_config
    VALIDATORS[service](validator_context)
  File "/quay-registry/util/config/validators/validate_storage.py", line 18, in validate
    providers = _get_storage_providers(config, ip_resolver, config_provider).items()
  File "/quay-registry/util/config/validators/validate_storage.py", line 52, in _get_storage_providers
    raise ConfigValidationException('Missing required parameter(s) for storage %s' % name)
ConfigValidationException: Missing required parameter(s) for storage rhocs
10.131.0.29 () - quayconfig [11/Dec/2019:08:08:33 +0000] "POST /api/v1/superuser/config/validate/registry-storage HTTP/1.1" 200 79 "-" "Go-http-client/1.1" (0.003 2517 0.003)

[sabre1041]

@kurtismullins during testing, defined the following CR and was able to successfully make use of OCS (using Quay v3.1.3)

spec:
  quay:
    registryBackends:
      - name: rhocs
        rhocs:
          accessKey: OMITTED
          bucketName: quay-0dd68a8a-457f-4da9-9807-ca5c4c941e99
          secretKey: OMITTED
          hostname: OMITTED
          secure: true
          port: 443

[kurtismullins]

@sabre1041

Thank you for updating the documentation.

Additional testing has been done with the updated documentation and operator. From what I understand, the Quay pod is not deployed when configuring S3 or Google Cloud with secrets in a secrets file.

However, it does appear to work correctly if the secrets are included in the CR directly.

Error Output:

{"level":"info","ts":1576830630.295516,"logger":"controller_quayecosystem","msg":"Reconciling QuayEcosystem","Request.Namespace":"quay-app","Request.Name":"quay-test-320"}
{"level":"error","ts":1576830630.295652,"logger":"controller_quayecosystem","msg":"Secret Validation","Namespace":"quay-app","Name":"s3-account","error":"Failed to validate provided secret with required parameters","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/travis/gopath/pkg/mod/github.com/go-logr/zapr@v0.1.1/zapr.go:128\ngithub.51.al/redhat-cop/quay-operator/pkg/controller/quayecosystem/validation.validateSecret\n\t/home/travis/gopath/src/github.com/redhat-cop/quay-operator/pkg/controller/quayecosystem/validation/validate.go:525\ngithub.51.al/redhat-cop/quay-operator/pkg/controller/quayecosystem/validation.Validate\n\t/home/travis/gopath/src/github.com/redhat-cop/quay-operator/pkg/controller/quayecosystem/validation/validate.go:236\ngithub.51.al/redhat-cop/quay-operator/pkg/controller/quayecosystem.(*ReconcileQuayEcosystem).Reconcile\n\t/home/travis/gopath/src/github.com/redhat-cop/quay-operator/pkg/controller/quayecosystem/quayecosystem_controller.go:126\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/travis/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.3.0/pkg/internal/controller/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/travis/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.3.0/pkg/internal/controller/controller.go:192\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/home/travis/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.3.0/pkg/internal/controller/controller.go:171\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/home/travis/gopath/pkg/mod/k8s.io/apimachinery@v0.0.0-20190817020851-f2f3a405f61d/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/home/travis/gopath/pkg/mod/k8s.io/apimachinery@v0.0.0-20190817020851-f2f3a405f61d/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/home/travis/gopath/pkg/mod/k8s.io/apimachinery@v0.0.0-20190817020851-f2f3a405f61d/pkg/util/wait/wait.go:88"}

[sabre1041]

Resolved per #120

[gorantornqvist]

Still receive this error, config

spec:
    registryBackends:
      - name: rhocs
        rhocs:
          credentialsSecretName: s3-credentials
          bucketName: quay-bucket-2649ee59-66cd-4ebd-92cf-223533aa3b78
          hostname: s3.openshift-storage.svc
          secure: true
          port: 443

Specifying accessKey/secretKey gets passed thru the validation:

#          credentialsSecretName: s3-credentials
          accessKey: *******
          secretKey: *******