@steve.hoffman I created a Jira bug for the above issue. Could you please follow up with the dev team and fix it in the 2.39 version as we are unable to use the PMM because the UI is not responding? Otherwise, we have to remove the nodes from the PMM server.
NOTE: I had removed a few nodes, and now Load has been reduced and the UI is responding fine. Now, I am running with 60 Nodes for testing purposes, and PMM is doing well. Again, I added around 40 servers, then again, same issue.
I can only promise the team will review and prioritize the issue accordingly. If we discover a workaround or config change as a fix we will get that out right away but I can’t promise a fix in any particular release version. If you are a Percona customer please escalate through your CSM and we possibly can issue a custom build until it’s fixed in a public release.
Unlike the other journaling modes, PRAGMA journal_mode=WAL is persistent. If a process sets WAL mode, then closes and reopens the database, the database will come back in WAL mode. In contrast, if a process sets (for example) PRAGMA journal_mode=TRUNCATE and then closes and reopens the database will come back up in the default rollback mode of DELETE rather than the previous TRUNCATE setting.
The persistence of WAL mode means that applications can be converted to using SQLite in WAL mode without making any changes to the application itself. One has merely to run "PRAGMA journal_mode=WAL;" on the database file(s) using the command-line shell or other utility, then restart the application.
The WAL journal mode will be set on all connections to the same database file if it is set on any one connection.
I am thinking and was going down the path of looking to see that maybe it wasnt grafana but something in containerd (While I am also VERY new to) I looked at the containerd config file and said yep. someone other than me needs to figure out what each of ones do… . I have not eyet looked to see if sqllite is configured in WAL mode or not…
Hi @Naresh9999 can I ask 2 questions?
Have you migrated to service accounts? Is it possible that one of your old PMM Clients are still configured to old server or have incorrect API key and keeps trying to connect to PMM Server?
While I do NOT know if it is suggested I have not seen the issue since I did the following as well as the above (Again on each grafana.db file as I had two but was not sure WHY I had two)
Do this at your own risk…**** I only did the first three .
When you issue the commands you MUST be in the directory that the grafana.db file is in… Else it does nothing
sqlite> pragma journal_mode=wal;
sqlite> PRAGMA cache_size = 10000; -- Set cache size to 10000 pages
sqlite> PRAGMA synchronous = OFF; -- Disable synchronous write (less durable, more performance)
sqlite> PRAGMA locking_mode = EXCLUSIVE; -- Use exclusive locking mode
sqlite> PRAGMA busy_timeout = 1000; -- Set a timeout of 1 second for waiting on locks
sqlite> PRAGMA temp_store = MEMORY; -- Use memory for temporary storage (if feasible)
Also if you are like me and new to Docker Make sure you commit the changes as well (and then I normally restart the docker)