Ok seems like instruction works just fine when there is a fresh Kubernetes cluster. This is deployed on GKE. The fact that pgbouncer is on “Pending” state is because, vCPU’s ran out…
So I think deployment fails on another Kubernetes cluster because of older version of Operator (0.2.0). Is this possible to upgrade? Or am I looking at the right direction?
Hi guys. Our Kubernetes cluster was re-installed. I was able to deploy PosgreSQL cluster on another namespace, but then I’m not able to deploy clusters to other namespaces. So it is a fresh k8s cluster. We are utilizing local storage with OpenEBS.
Operator logs on a broken envs show that there might be some issues with permissions. One row from logs: “1. Logs (or some of the latest rows) from operator “kubectl logs postgres-operator-6d8c847594-skhwd operator -n possu2”: 2. time=“2021-12-01T07:24:56Z” level=error msg=“Controller Manager: Controller Group for namespace possu2 does not have the required list privileges for resource pods in the Core API” func=“github.com/percona/percona-postgresql-operator/internal/controller/manager.(*ControllerManager).hasListerPrivs()” file=”/go/src/github.com/percona/percona-postgresql-operator/internal/controller/manager/controllermanager.go:370" version=0.2.0".
Not working environment:
[root@dbaasjump002 postgresql_production_configuration]# kubectl get pods -n dbaas-postgresql-possu02
NAME READY STATUS RESTARTS AGE
pgo-deploy-wktd7 0/1 Completed 0 25h
postgres-operator-d9bfdd95c-qfblv 4/4 Running 0 25h
Not working environment:
[root@dbaasjump002 deploy]# kubectl get pods -n possu2
NAME READY STATUS RESTARTS AGE
pgo-deploy-lp5td 0/1 Completed 0 60m
postgres-operator-6d8c847594-skhwd 4/4 Running 0 60m
I reproduced the test on Google Cloud and got the same situation:
conaPGClusterh-Al:~/percona-postgresql-operator/deploy (fi-katajistok-test-project)$ kubectl get Per
NAMESPACE NAME AGE
pgo cluster1 34m
pgo2 cluster1 4m21s
kimkat@cloudshell:~/percona-postgresql-operator/deploy (fi-katajistok-test-project)$ kubectl get pods -n pgo2
NAME READY STATUS RESTARTS AGE
pgo-deploy-k29vk 0/1 Completed 0 11m
postgres-operator-66b6ccd7c5-jtvl8 4/4 Running 0 11m
Logs from Operator in pgo2 namespace:
time=“2021-12-01T10:55:38Z” level=error msg=“Controller Manager: Controller Group for namespace pgo2 does not have the required list privileges for resource pgpolicies inthe pg.percona.com API” func=“github.com/percona/percona-postgresql-operator/internal/controller/manager.(*ControllerManager).hasListerPrivs()” file="/go/src/github.com/percona/percona-postgresql-operator/internal/controller/manager/controllermanager.go:357" version=0.2.0
time=“2021-12-01T10:55:38Z” level=error msg=“Controller Manager: Controller Group for namespace pgo2 does not have the required list privileges for resource pods in the Core API” func=“github.com/percona/percona-postgresql-operator/internal/controller/manager.(*ControllerManager).hasListerPrivs()” file="/go/src/github.com/percona/percona-postgresql-operator/internal/controller/manager/controllermanager.go:370" version=0.2.0
time=“2021-12-01T10:55:38Z” level=error msg=“Controller Manager: Controller Group for namespace pgo2 does not have the required list privileges for resource jobs in the Batch API” func=“github.com/percona/percona-postgresql-operator/internal/controller/manager.(*ControllerManager).hasListerPrivs()” file="/go/src/github.com/percona/percona-postgresql-operator/internal/controller/manager/controllermanager.go:382" version=0.2.0
time=“2021-12-01T10:55:38Z” level=error msg=“Namespace Controller: error syncing Namespace ‘pgo2’: Controller Manager: cannot start controller group for namespace pgo2 because it does not have the required privs, will attempt to start on the next ns refresh interval” func=“github.com/percona/percona-postgresql-operator/internal/controller/namespace.(*Controller).processNextWorkItem()” file="/go/src/github.com/percona/percona-postgresql-operator/internal/controller/namespace/namespacecontroller.go:151" version=0.2.0
I would like to go this thru with you guys in a meeting. Would this be possible?
kubectl get secret cluster1-pguser-secret -o yaml & tested the Postgresql connectivity:
pgdb=> \dn
List of schemas
Name | Owner
-----------±---------
pgbouncer | postgres
pguser | pguser
public | postgres
(3 rows)
Now created another namespace pgo1:
kubectl create namespace pgo1
kubectl config set-context $(kubectl config current-context) --namespace=pgo1
Edited to operator.yaml with the new namespace value on the below lines:
line no5: namespace: pgo1
line no123: namespace: pgo1
line no 192: pgo_operator_namespace: “pgo1”
line no 263: namespace: pgo1
line no 269: namespace: pgo1
kubectl apply -f deploy/operator.yaml
kubectl get pods -n pgo2 (When verified only the Deployment is running & the Percona operator itself not created)
NAME READY STATUS RESTARTS AGE
pgo-deploy-npzs9 0/1 Completed 0 122m
Please verify why Postgresql Operator pod is not getting started in another namespace other than pgo. Kindly share your inputs.
If you want to install several operators under one k8s cluster you need to change percona-postgresql-operator/operator.yaml at main · percona/percona-postgresql-operator · GitHub option to ‘disabled/readonly’ but I have found the issue connected with it and it can’t be used now. We are working to fix this issue and the fix will be available in the next release. If you want I can inform you as soon as we merge our changes into main and you can test it from your end too.
