mongodb / mongodb-kubernetes-operator

MongoDB Community Kubernetes Operator

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Steps to upgrade from 4.4 to 6.0 via 5.0?

prachiwaghulkar opened this issue · comments

What did you do to encounter the bug?
Steps to reproduce the behavior:
Upgraded from 4.4 to 5.0.23. The MongoDBCommunity operator went into running state. When I tried to go to 6.0.12 I encountered the following error in the logs

{"t":{"$date":"2024-01-25T07:57:26.086+00:00"},"s":"F",  "c":"CONTROL",  "id":20573,   "ctx":"initandlisten","msg":"Wrong mongod version","attr":{"error":"UPGRADE PROBLEM: Found an invalid featureCompatibilityVersion document (ERROR: Location4926900: Invalid featureCompatibilityVersion document in admin.system.version: { _id: \"featureCompatibilityVersion\", version: \"4.4\" }. See https://docs.mongodb.com/master/release-notes/5.0-compatibility/#feature-compatibility. :: caused by :: Invalid feature compatibility version value, expected '5.0' or '5.3' or '6.0. See https://docs.mongodb.com/master/release-notes/5.0-compatibility/#feature-compatibility.). If the current featureCompatibilityVersion is below 5.0, see the documentation on upgrading at https://docs.mongodb.com/master/release-notes/5.0/#upgrade-procedures."}}
{"t":{"$date":"2024-01-25T07:57:26.086+00:00"},"s":"I",  "c":"REPL",     "id":4784900, "ctx":"initandlisten","msg":"Stepping down the ReplicationCoordinator for shutdown","attr":{"waitTimeMillis":15000}}
{"t":{"$date":"2024-01-25T07:57:26.086+00:00"},"s":"I",  "c":"REPL",     "id":4794602, "ctx":"initandlisten","msg":"Attempting to enter quiesce mode"}

Even I tried to update the feature compatibility after upgrade to 5.0.23 using command
db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )
However the feature compatibility version remained the same. Refer log file below and the commands executed.

log.txt

image

What did you expect?
After 4.4 to 5.0 upgrade, The set FeatureCompatibilityVersion command should have set the value to 5.0.

What happened instead?
he set FeatureCompatibilityVersion command didn't set the value to 5.0 and remained 4.4

Operator Information

  • Operator Version: mongodb/mongodb-kubernetes-operator : 0.8.3
  • MongoDB Image used: 5.0.23

Hey

Even i am facing the same issue while trying to upgrade the mongo from 4.4 to 6.0 via 5.0

{"t":{"$date":"2024-01-29T09:21:46.835+00:00"},"s":"F", "c":"CONTROL", "id":20573, "ctx":"initandlisten","msg":"Wrong mongod version","attr":{"error":"UPGRADE PROBLEM: Found an invalid featureCompatibilityVersion document (ERROR: Location4926900: Invalid featureCompatibilityVersion document in admin.system.version: { _id: \"featureCompatibilityVersion\", version: \"4.4\" }. See https://docs.mongodb.com/master/release-notes/5.0-compatibility/#feature-compatibility. :: caused by :: Invalid feature compatibility version value, expected '5.0' or '5.3' or '6.0. See https://docs.mongodb.com/master/release-notes/5.0-compatibility/#feature-compatibility.). If the current featureCompatibilityVersion is below 5.0, see the documentation on upgrading at https://docs.mongodb.com/master/release-notes/5.0/#upgrade-procedures."}}

Operator Information:

  • Community operator version : 0.9.0
  • MongoDB image used : 5.0.24

Hi,
same issue here as well when trying to upgrade from 4.4.10 to 5.0.24 (and eventually 6.0.13).
The RS was created with 4.4.10 and an older version of the community operator.

Some seconds after the setFeatureCompatibilityVersion: "5.0" command, getParameters returns 5.0, but then it is 4.4 again.

From the attached log, I see the following parameter changes:

{"t":{"$date":"2024-02-01T09:50:17.768+00:00"},"s":"I", "c":"REPL", "id":20459, "ctx":"conn22163","msg":"Setting featureCompatibilityVersion","attr":{"currentVersion":"4.4","newVersion":"upgrading from 4.4 to 5.0"}} {"t":{"$date":"2024-02-01T09:50:19.785+00:00"},"s":"I", "c":"REPL", "id":20459, "ctx":"conn22163","msg":"Setting featureCompatibilityVersion","attr":{"currentVersion":"upgrading from 4.4 to 5.0","newVersion":"5.0"}} {"t":{"$date":"2024-02-01T09:50:23.276+00:00"},"s":"I", "c":"REPL", "id":20459, "ctx":"conn10","msg":"Setting featureCompatibilityVersion","attr":{"currentVersion":"5.0","newVersion":"downgrading from 5.0 to 4.4"}} {"t":{"$date":"2024-02-01T09:50:25.291+00:00"},"s":"I", "c":"REPL", "id":20459, "ctx":"conn10","msg":"Setting featureCompatibilityVersion","attr":{"currentVersion":"downgrading from 5.0 to 4.4","newVersion":"4.4"}}

But no real indication why it is going back to 4.4 again.
Edit
After staring at it for some time, I thing the following line might be leading in the right direction:
"The default write concern has been changed by downgrading the featureCompatibilityVersion from 5.0. To keep the default write concern the same as before, use setDefaultRWConcern to set a cluster-wide write concern."}

Edit 2
After setting the defaultRWConcern, the warning is gone but the setFeatureCompatibilityVersion command is still getting rolled back.

MongoUpgrade.txt

Operator Information:

Community operator version : 0.8.3
MongoDB image used : 5.0.24

Hi,
The upgrade of mongo is done successfully after explicitly setting the variable featureCompatibilityVersion in mongodb spec to 5.0 while upgrading db from 5.0 series to 6.0 series.

apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
spec:
  featureCompatibilityVersion: "5.0"
  members: 3
  ......

Documentation for featureCompatiblityVersion is here

Hi,
I am still getting the log "Wrong mongod version","attr":{"error":"UPGRADE PROBLEM: Found an invalid featureCompatibilityVersion document (ERROR: Location4926900: Invalid featureCompatibilityVersion document in admin.system.version: { _id: \"featureCompatibilityVersion\", version: \"4.4\" }. See https://docs.mongodb.com/master/release-notes/5.0-compatibility/#feature-compatibility. :: caused by :: Invalid feature compatibility version value, expected '5.0' or '5.3' or '6.0. See https://docs.mongodb.com/master/release-notes/5.0-compatibility/#feature-compatibility.). If the current featureCompatibilityVersion is below 5.0, see the documentation on upgrading at https://docs.mongodb.com/master/release-notes/5.0/#upgrade-procedures."}}
even after setting featureCompatibilityVersion: "5.0" when upgrading from 5.0.26 to 6.0.14. Anybody else run into this?

@trevor-carlson1 It looks like the featureCompatibility is not updated properly. You can check the featureCompatibiltyVersion in the database by using this command db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } ) in admin database and set it to the desired value using db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )

This issue is being marked stale because it has been open for 60 days with no activity. Please comment if this issue is still affecting you. If there is no change, this issue will be closed in 30 days.