Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitnami/mongodb] MongoDB replicaset fails when "primary" is in the release name #29821

Open
Lygoria opened this issue Oct 8, 2024 · 0 comments · May be fixed by #29824
Open

[bitnami/mongodb] MongoDB replicaset fails when "primary" is in the release name #29821

Lygoria opened this issue Oct 8, 2024 · 0 comments · May be fixed by #29824
Assignees
Labels
tech-issues The user has a technical issue about an application triage Triage is needed

Comments

@Lygoria
Copy link

Lygoria commented Oct 8, 2024

Name and Version

bitnami/mongodb 15.0.0 to 16.0.0 (at least)

What architecture are you using?

None

What steps will reproduce the bug?

Deploy MongoDB charts with replicaset and a release-name that contains primary:

Command to reproduce:

helm install \
  --namespace=mongo \
  --create-namespace \
  mongo-primary \
  oci://registry-1.docker.io/bitnamicharts/mongodb \
  --set="arbiter.enabled=false" \
  --set="architecture=replicaset" \
  --set="replicaCount=3" \
  --set="persistence.enabled=true" \
  --set="global.storageClass=local-path" \
  --set="auth.enabled=false"

Are you using any custom parameters or values?

architecture: replicaset
replicaCount: 3
auth:
  enabled: false
global:
  storageClass: local-path
persistence:
  enabled: true
arbiter:
  enabled: false

What is the expected behavior?

All members of the replicaset starts correctly

What do you see instead?

The second (or sometimes third, it seems random) pod fails.
The pod starts with the following logs:

$ kubectl -n mongo logs --previous mongo-primary-mongodb-1
Defaulted container "mongodb" out of: mongodb, log-dir (init)
mongodb 15:02:27.57 INFO  ==> Advertised Hostname: mongo-primary-mongodb-1.mongo-primary-mongodb-headless.mongo.svc.cluster.local
mongodb 15:02:27.57 INFO  ==> Advertised Port: 27017
realpath: /bitnami/mongodb/data/db: No such file or directory
mongodb 15:02:27.58 INFO  ==> Data dir empty, checking if the replica set already exists
mongodb 15:02:28.43 INFO  ==> Detected existing primary: mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongodb 15:02:28.43 INFO  ==> Current primary is different from this node. Configuring the node as replica of mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongodb 15:02:28.44 INFO  ==> 
mongodb 15:02:28.44 INFO  ==> Welcome to the Bitnami mongodb container
mongodb 15:02:28.44 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
mongodb 15:02:28.44 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
mongodb 15:02:28.45 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
mongodb 15:02:28.45 INFO  ==> 
mongodb 15:02:28.45 INFO  ==> ** Starting MongoDB setup **
mongodb 15:02:28.50 INFO  ==> Validating settings in MONGODB_* env vars...
mongodb 15:02:28.55 INFO  ==> Initializing MongoDB...
mongodb 15:02:28.64 INFO  ==> Writing keyfile for replica set authentication...
mongodb 15:02:28.65 INFO  ==> Deploying MongoDB from scratch...
MongoNetworkError: connect ECONNREFUSED 192.168.78.34:27017
mongodb 15:02:30.11 INFO  ==> Creating users...
mongodb 15:02:30.11 INFO  ==> Users created
mongodb 15:02:30.12 INFO  ==> Enabling authentication...
mongodb 15:02:30.19 INFO  ==> Configuring MongoDB replica set...
mongodb 15:02:30.19 INFO  ==> Stopping MongoDB...
mongodb 15:02:33.81 INFO  ==> Trying to connect to MongoDB server mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local...
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
the required argument `port` was not provided
mongodb 15:03:08.84 ERROR ==> Unable to connect to host mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local:27017
mongo-primary-mongodb-0.mongo-primary-mongodb-headless.mongo.svc.cluster.local
mongodb 15:03:08.85 INFO  ==> Stopping MongoDB...

The log the required argument 'port' was not provided is strange as there is a port configured.
Then the pod restarts and stays in error (Readiness probe failed: Error: Not ready) apparently due to an incomplete configuration:

$ kubectl -n mongo logs  mongo-primary-mongodb-1 | head -n 100
Defaulted container "mongodb" out of: mongodb, log-dir (init)
mongodb 15:03:10.73 INFO  ==> Advertised Hostname: mongo-primary-mongodb-1.mongo-primary-mongodb-headless.mongo.svc.cluster.local
mongodb 15:03:10.73 INFO  ==> Advertised Port: 27017
mongodb 15:03:10.73 INFO  ==> Pod name doesn't match initial primary pod name, configuring node as a secondary
mongodb 15:03:10.74 INFO  ==> 
mongodb 15:03:10.75 INFO  ==> Welcome to the Bitnami mongodb container
mongodb 15:03:10.75 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
mongodb 15:03:10.75 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
mongodb 15:03:10.75 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
mongodb 15:03:10.75 INFO  ==> 
mongodb 15:03:10.75 INFO  ==> ** Starting MongoDB setup **
mongodb 15:03:10.77 INFO  ==> Validating settings in MONGODB_* env vars...
mongodb 15:03:10.84 INFO  ==> Initializing MongoDB...
mongodb 15:03:10.94 INFO  ==> Writing keyfile for replica set authentication...
mongodb 15:03:11.00 INFO  ==> Deploying MongoDB with persisted data...
mongodb 15:03:11.02 INFO  ==> ** MongoDB setup finished! **
mongodb 15:03:11.03 INFO  ==> ** Starting MongoDB **
{"t":{"$date":"2024-10-08T15:03:11.109Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"enableLocalhostAuthBypass":{"default":true,"value":false}}}}
{"t":{"$date":"2024-10-08T15:03:11.148+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"main","msg":"***** SERVER RESTARTED *****"}

[ ...]

{"t":{"$date":"2024-10-08T15:03:13.000+00:00"},"s":"W",  "c":"QUERY",    "id":23799,   "ctx":"ftdc","msg":"Aggregate command executor error","attr":{"error":{"code":26,"codeName":"NamespaceNotFound","errmsg":"Unable to retrieve storageStats in $collStats stage :: caused by :: Collection [local.oplog.rs] not found."},"stats":{},"cmd":{"aggregate":"oplog.rs","cursor":{},"pipeline":[{"$collStats":{"storageStats":{"waitForLock":false,"numericOnly":true}}}],"$db":"local"}}}

Additional information

Replicaset is KO:

> rs.conf()
MongoServerError[NotYetInitialized]: no replset config has been received
@Lygoria Lygoria added the tech-issues The user has a technical issue about an application label Oct 8, 2024
@github-actions github-actions bot added the triage Triage is needed label Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech-issues The user has a technical issue about an application triage Triage is needed
Projects
None yet
2 participants