feat: add --custom-resources flag to validator rules check as an alternate to -f

[ahmad-ibra]

Issue

Resolves #178
Resolves #136

Description

This PR adds support for passing in custom resource documents to the validator rules check command via the --custom-resources flag. The --custom-resources flag can receive either a path to a directory or to a file.

File selection

If a directory path is provided, we check each file within the directory for CR documents. Note that we walk the entire file tree from a particular directory. This means that users are able to organize their CRs in a manner like this:

❯ tree tests/integration/_validator/testcases/data/validator-crs
tests/integration/_validator/testcases/data/validator-crs
├── combined.yaml
├── network
│   └── network.yaml
└── oci
    └── oci.yaml

3 directories, 3 files

In the above example, we'll look for CRs within the slice of all 3 files found (combined.yaml, network.yaml, and oci.yaml). Users have the flexibility to organize their CRs as they desire.

If a single file is instead passed in via the --cr flag, we only work with a slice containing the single file that was provided.

CR Unmarshalling

Given a slice of files (which may only contain a single file), we range over each file and attempt the following:

We try to read all the CR documents that are contained in that file and run validations against them.

For instance, if the path to the following file is passed in:

apiVersion: validation.spectrocloud.labs/v1alpha1
kind: OciValidator
metadata:
  name: oci-validator-combined-oci-rules
spec:
  ociRegistryRules:
    - name: "public oci registry with tag"
      host: "docker.io"
      validationType: "none"
      artifacts:
        - ref: "library/redis:7.2.4"
---
apiVersion: validation.spectrocloud.labs/v1alpha1
kind: NetworkValidator
metadata:
  name: network-validator-combined-network-rules
spec:
  dnsRules:
  - name: Resolve Google
    host: google.com

We'll run validations against both he network and the oci plugin rules that are defined in the file. Users can choose to store all CRs in a single file, or spread them across multiple files. Whatever configuration they choose will work.

Error handling

Succesfull validation can result in a lot of text being printed to a users terminal. Given this, I wanted to avoid "hiding" an error in the flood of validation results. If any CR document is invalid, an error message will be returned and no validations will run. It'll be obvious what failed and how to fix it.

[codecov]

Codecov Report

Attention: Patch coverage is 58.76289% with 40 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/cmd/validator/validator.go	53.42%	21 Missing and 13 partials ⚠️
pkg/utils/file/file.go	63.63%	2 Missing and 2 partials ⚠️
...integration/_validator/testcases/test_validator.go	75.00%	1 Missing and 1 partial ⚠️

@@            Coverage Diff             @@
##             main     #218      +/-   ##
==========================================
+ Coverage   53.33%   53.43%   +0.09%     
==========================================
  Files          45       46       +1     
  Lines        6272     6367      +95     
==========================================
+ Hits         3345     3402      +57     
- Misses       2082     2105      +23     
- Partials      845      860      +15     

Files with missing lines	Coverage Δ
cmd/validator.go	82.60% <100.00%> (+1.27%)	⬆️
pkg/config/config.go	66.03% <ø> (ø)
pkg/config/constants.go	100.00% <ø> (ø)
...integration/_validator/testcases/test_validator.go	93.40% <75.00%> (-0.28%)	⬇️
pkg/utils/file/file.go	63.63% <63.63%> (ø)
pkg/cmd/validator/validator.go	60.07% <53.42%> (-0.77%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97875f3...af7f421. Read the comment docs.

[TylerGillson]

Nice work! This is awesome. One tiny nit... sorry 😁

Can you please raise a follow up PR in the docs repo to cover this new usage?

[TylerGillson]

I'm having naming hesitancy about --cr. Typically the long name for flags is not abbreviated. What do you think about --cr-path? Or --custom-resources?