Helm upgrade Operator yields errors

I do an upgrade from 1.10 to 1.11 with the helm upgrade command and see the following in the logs:

{"level":"error","ts":1645668000.7298462,"logger":"controller_psmdb","msg":"failed to ensure version","error":"failed to update CR: Operation cannot be fulfilled on perconaservermongodbs.psmdb.percona.com \"percona-db-psmdb-db\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/percona/percona-server-mongodb-operator/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).sheduleEnsureVersion.func1\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/version.go:81\ngithub.com/robfig/cron/v3.FuncJob.Run\n\t/go/src/github.com/percona/percona-server-mongodb-operator/vendor/github.com/robfig/cron/v3/cron.go:136\ngithub.com/robfig/cron/v3.(*Cron).startJob.func1\n\t/go/src/github.com/percona/percona-server-mongodb-operator/vendor/github.com/robfig/cron/v3/cron.go:312"}
{"level":"info","ts":1646496334.400243,"logger":"controller_psmdb","msg":"statefullSet was changed, start smart update","name":"percona-db-psmdb-db-rs0"}
{"level":"info","ts":1646496334.400283,"logger":"controller_psmdb","msg":"can't start/continue 'SmartUpdate': waiting for all replicas are ready"}
{"level":"info","ts":1646496334.5727537,"logger":"controller_psmdb","msg":"statefullSet was changed, start smart update","name":"percona-db-psmdb-db-rs0"}
{"level":"info","ts":1646496334.5727818,"logger":"controller_psmdb","msg":"can't start/continue 'SmartUpdate': waiting for all replicas are ready"}
{"level":"info","ts":1646496339.5731938,"logger":"controller_psmdb","msg":"statefullSet was changed, start smart update","name":"percona-db-psmdb-db-rs0"}
{"level":"info","ts":1646496339.5732274,"logger":"controller_psmdb","msg":"can't start/continue 'SmartUpdate': waiting for all replicas are ready"}

How can I upgrade if not with helm upgrade?

1 Like

@devops-1 could you please share the exact set of commands that you execute?

2 Likes

@spronin

I used

$ helm upgrade percona-operator percona/psmdb-operator --version 1.11.0

However, that was 4 days ago and seems to have recovered. Now I retried to reproduce and am seeing since 1 hour:

{"level":"info","ts":1647366655.8734343,"logger":"leader","msg":"Leader pod has been deleted, waiting for garbage collection do remove the lock."}

1 Like