Sync node labels and taints #4

annismckenzie · 2021-01-04T13:12:13Z

This closes #5.

It can now only handle these in its own namespace. The fix was to add `namespace=k3os-config-operator-system` to the RBAC markers for the CR.

This is necessary in order to get secrets and update the nodes.

This is still very much a work in progress.

I need this over in the picl-k3os-image-generator repository.

README.md

annismckenzie

A couple nits that need to be addressed but overall this looks good.

README.md

controllers/config/k3osconfig_controller.go

controllers/config/k3osconfig_setup.go

pkg/util/taints/taints.go

pkg/nodes/labeler.go

…ed from For now it’s from the environment via the downward API but this might change and the reconciler should not care when this changes.

This is another approach to handle terminal errors in a reconciler. For the case where an object was deleted (like the node, the secret or the K3OSConfig CR object) we have to return a nil error to the caller so that our reconciler isn’t called anymore. I might flesh this out more in the future with some functional options to configure the response error further. For now, this will do.

…conciler There’s now a couple more smaller methods to make this easier to read, understand and test.

This removes the need for the nasty panic I added earlier.

This cuts down on the amount of parameters these need to lug around.

This way we can change the implementation of that and the rest of the code doesn’t need to care.

This creates metadata-only watches on our secret (filtered by the unique label it has) as well as the node the current instance is running on. It’s pretty light on the network and the memory but boy was this a pain to set up. 🤞 that it works correctly.

We’re using the context everywhere so enabling this should be safe.

…eases`

This feature has been on by default since v1.14 and this operator was never deployed anywhere.

This includes about the same number of tests as the node label reconciler but the implementation is more complex because of the fact that taints have a lot more intricacies than labels.

This mostly deals with updates to comments and removing dead code.

This handles IsNotFound, IsGone and IsForbidden API server error responses.

annismckenzie · 2021-01-08T10:42:35Z

README.md

-1. k3OS cluster that's running nominally
-2. A (local) directory of YAML files as describes in https://github.com/sgielen/picl-k3os-image-generator#getting-started:
+1. A k3OS cluster that's running [nominally](https://joshdance.medium.com/what-does-nominal-mean-when-spacex-mission-control-says-it-39c2d249da27#:~:text=performing%20or%20achieved%20within%20expected,within%20expected%20and%20acceptable%20limits.).
+2. A local clone of https://github.com/sgielen/picl-k3os-image-generator.


This will probably have to point to my fork to work.

annismckenzie · 2021-01-08T10:43:00Z

This is done – merging.

annismckenzie added 12 commits January 3, 2021 20:42

Extract K3OSConfigReconciler setup code into a separate file

eb49d0e

Add and initialize a node lister

e958aab

Initialize node lister when configuring K3OSConfigReconciler

a9b011c

Rework K3OSConfig handler return values

7adff41

Fixup image build

5ad0b62

Fix error message when the manager cannot be created

7efecc1

Allow the operator to get and watch secrets in its namespace

6355380

Fix RBAC roles for the K3OSConfig CR

d225ec2

It can now only handle these in its own namespace. The fix was to add `namespace=k3os-config-operator-system` to the RBAC markers for the CR.

Add a Kubernetes clientset to the K3OSConfig reconciler

f0fc1b2

This is necessary in order to get secrets and update the nodes.

Add dev-delete Makefile target

ad26193

Add NAMESPACE environment variable via the downward API

444c80b

Restrict manager to the namespace the operator is running in

54a85cd

annismckenzie self-assigned this Jan 4, 2021

Add missing role binding for the newly created manager role

9cb6165

annismckenzie force-pushed the sync-node-labels branch 2 times, most recently from 39929d1 to b803fd9 Compare January 4, 2021 14:10

annismckenzie changed the title ~~Sync node labels~~ Sync node labels and taints Jan 5, 2021

annismckenzie added this to the v0.1 Initial release milestone Jan 5, 2021

annismckenzie force-pushed the sync-node-labels branch 2 times, most recently from 2e990b4 to 587230f Compare January 6, 2021 22:04

annismckenzie and others added 5 commits January 6, 2021 23:10

Add initial implementation of node label reconciliation

b56dbac

This is still very much a work in progress.

Add gopkg.in/yaml.v3 library

7b596b0

Parse node config YAML into struct

d71bef1

Rename node config secret constant

add5507

Deserialize hostname from k3OS config YAML

4a1e433

I need this over in the picl-k3os-image-generator repository.

annismckenzie force-pushed the sync-node-labels branch from 587230f to 6eb31df Compare January 6, 2021 22:11

annismckenzie marked this pull request as ready for review January 8, 2021 00:51

annismckenzie commented Jan 8, 2021

View reviewed changes

README.md Outdated Show resolved Hide resolved

annismckenzie commented Jan 8, 2021

View reviewed changes

README.md Outdated Show resolved Hide resolved

annismckenzie commented Jan 8, 2021

View reviewed changes

annismckenzie and others added 23 commits January 8, 2021 11:37

Add GetNodeName to consts that hides where the node name is retriev…

c31210b

…ed from For now it’s from the environment via the downward API but this might change and the reconciler should not care when this changes.

Simplify and streamline logic to fetch node and node config in the re…

1fc081d

…conciler There’s now a couple more smaller methods to make this easier to read, understand and test.

Add no-op node tainter

1fde51c

Update node labeler and tainter instantiation and log changes

150b513

Fix added labels annotations not updated correctly

11063bb

Upgrade controller-runtime to v0.7.0 and upgrade dependencies

84fbcd7

Validate node name is set when starting the operator

0063b96

This removes the need for the nasty panic I added earlier.

Pass the shutdown context to SetupWithManager and set the namespace

f3fe16f

Reattach the extracted getters to the reconciler struct

0038db6

This cuts down on the amount of parameters these need to lug around.

Extract updating the node resource into its own method

eb3b704

This way we can change the implementation of that and the rest of the code doesn’t need to care.

Enable the new LeaderElectionReleaseOnCancel option on the manager

6581f2f

We’re using the context everywhere so enabling this should be safe.

Allow leader election role to interact with `coordination.k8s.io/v1/l…

d6f0256

…eases`

Allow the operator to also list secrets in its namespace

bf875c2

Switch leader election resource lock to leases only

52a8276

This feature has been on by default since v1.14 and this operator was never deployed anywhere.

Add comment on enqueueObjectsOnChanges method

39991dc

Clean up leader election role RBAC YAML

70b9870

Add taints util package copied over from the K8s source tree

c105f8a

Fix spelling of separator in const name

fe36bf3

Finish implementation of node taint reconciler

79496c0

This includes about the same number of tests as the node label reconciler but the implementation is more complex because of the fact that taints have a lot more intricacies than labels.

Clean up code that was mentioned in the review

b281432

This mostly deals with updates to comments and removing dead code.

Add log statements in result error handler

4cf2e0a

This handles IsNotFound, IsGone and IsForbidden API server error responses.

annismckenzie force-pushed the sync-node-labels branch from 204541c to 4cf2e0a Compare January 8, 2021 10:39

annismckenzie commented Jan 8, 2021

View reviewed changes

annismckenzie merged commit 01502d6 into main Jan 8, 2021

annismckenzie deleted the sync-node-labels branch January 8, 2021 10:43

annismckenzie added enhancement New feature or request go Pull requests that update Go code labels Jan 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sync node labels and taints #4

Sync node labels and taints #4

annismckenzie commented Jan 4, 2021 •

edited

Loading

annismckenzie left a comment •

edited

Loading

annismckenzie Jan 8, 2021

annismckenzie commented Jan 8, 2021

Sync node labels and taints #4

Sync node labels and taints #4

Conversation

annismckenzie commented Jan 4, 2021 • edited Loading

annismckenzie left a comment • edited Loading

Choose a reason for hiding this comment

annismckenzie Jan 8, 2021

Choose a reason for hiding this comment

annismckenzie commented Jan 8, 2021

annismckenzie commented Jan 4, 2021 •

edited

Loading

annismckenzie left a comment •

edited

Loading