Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Add labels to ephemeral (listener) volumes #534

Merged
merged 12 commits into from
Jun 4, 2024
Merged

Conversation

razvan
Copy link
Member

@razvan razvan commented May 28, 2024

Description

🟢 CI https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/hdfs-operator-it-custom/143/

Part of stackabletech/issues#555

Depends on stackabletech/operator-rs#799

Ensure the ephemeral (listener) volumes attached directly to the datanode containers are labeled with the "recommended labels".

This is now the expected structure:

metadata:
  name: hdfs-datanode-default
spec:
  template:
    volumes:
    - ephemeral:
        volumeClaimTemplate:
          metadata:
            annotations:
              listeners.stackable.tech/listener-class: external-unstable
            creationTimestamp: null
            labels:
              app.kubernetes.io/component: datanode
              app.kubernetes.io/instance: hdfs
              app.kubernetes.io/name: hdfs
              app.kubernetes.io/role-group: default
          spec:
            accessModes:
            - ReadWriteMany
            resources:
              requests:
                storage: "1"
            storageClassName: listeners.stackable.tech
            volumeMode: Filesystem
      name: listener

Stacklet listing now includes datanode endpoints as they inherit the labels of the (listener) ephemeral volumes:

❯ stackablectl stacklet list -n kuttl-test-direct-wolf

┌───────────┬─────────┬────────────────────────┬──────────────────────────────────────────────────────────────┬─────────────────────────────────┐
│ PRODUCT   ┆ NAME    ┆ NAMESPACE              ┆ ENDPOINTS                                                    ┆ CONDITIONS                      │
╞═══════════╪═════════╪════════════════════════╪══════════════════════════════════════════════════════════════╪═════════════════════════════════╡
│ hdfs      ┆ hdfs    ┆ kuttl-test-direct-wolf ┆ datanode-default-0-listener-data     172.18.0.2:30580        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-0-listener-http     http://172.18.0.2:32297 ┆                                 │
│           ┆         ┆                        ┆ datanode-default-0-listener-ipc      172.18.0.2:31998        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-0-listener-metrics  172.18.0.2:30805        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-data     172.18.0.2:30990        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-http     http://172.18.0.2:32605 ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-ipc      172.18.0.2:30825        ┆                                 │
│           ┆         ┆                        ┆ datanode-default-1-listener-metrics  172.18.0.2:30264        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-0-http              http://172.18.0.2:31438 ┆                                 │
│           ┆         ┆                        ┆ namenode-default-0-metrics           172.18.0.2:31633        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-0-rpc               172.18.0.2:31447        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-1-http              http://172.18.0.2:30285 ┆                                 │
│           ┆         ┆                        ┆ namenode-default-1-metrics           172.18.0.2:32287        ┆                                 │
│           ┆         ┆                        ┆ namenode-default-1-rpc               172.18.0.2:30711        ┆                                 │
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
│ zookeeper ┆ hdfs-zk ┆ kuttl-test-direct-wolf ┆ zk                                   172.18.0.2:32241        ┆ Available, Reconciling, Running │
└───────────┴─────────┴────────────────────────┴──────────────────────────────────────────────────────────────┴─────────────────────────────────┘

Use "stackablectl stacklet credentials [OPTIONS] <PRODUCT_NAME> <STACKLET_NAME>" to display credentials for deployed stacklets.

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Author

Reviewer

Acceptance

@razvan razvan marked this pull request as ready for review May 29, 2024 13:15
@razvan razvan requested a review from a team May 29, 2024 13:15
@razvan razvan self-assigned this May 30, 2024
@razvan razvan marked this pull request as draft May 30, 2024 12:27
@razvan razvan marked this pull request as ready for review May 31, 2024 09:45
@sbernauer sbernauer changed the title bugfix/pvc labels fix: Add labels to ephemeral (listener) volumes Jun 3, 2024
@sbernauer sbernauer self-requested a review June 3, 2024 11:15
Copy link
Member

@sbernauer sbernauer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM otherwise

Cargo.toml Outdated Show resolved Hide resolved
rust/operator-binary/src/hdfs_controller.rs Outdated Show resolved Hide resolved
@sbernauer
Copy link
Member

sbernauer commented Jun 3, 2024

I noticed the journalnodes are missing Listeners. As you can not port-forward, there is no way to access the journalnode UI AFAIK. I think we should expose that UI as well, what do you think?

Was already discussed in Slack, no need to expose the JN UI

Cargo.toml Outdated Show resolved Hide resolved
@razvan razvan requested a review from sbernauer June 3, 2024 15:12
@razvan razvan enabled auto-merge June 3, 2024 15:12
@razvan razvan added this pull request to the merge queue Jun 4, 2024
Merged via the queue into main with commit cf66ff4 Jun 4, 2024
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants