After some users who were evaluating PMM have left the company, I removed some users and organisations, now I can’t see any dashboards. They all show “Dashboard not found”.
The only relevant I can see in the server diagnostic bundle is in grafana.log:
t=2022-03-07T15:01:56+0000 lvl=info msg=“Request Completed” logger=context userId=5 orgId=5 uname=psumner method=GET path=/api/dashboards/uid/pmm-home status=404 remote_addr=xx.xx.xx.xx time_ms=1 size=33 referer=http://xx.xx.xx.xx/graph/d/pmm-home/home-dashboard
I’m guessing I’ve deleted some kind of master organization but I’ve no idea how to get it back - what can I do?
I tried restarting everything from the CLI using supervisorctl and now I’m getting a “502 Bad Gateway” error in the web browser.
After lots of poking round I found this in supervisorctl tail -f grafana:
service init failed: Datasource provisioning error: failed to provision “Metrics” data source: organization not found
What do I need to do to recover from this?
I’ve got half-way there through lots of searching and guesswork. I found the grafana sqlite DB and added a new Main Org. with ID1:
[root@xxxxx grafana]# sqlite3 /srv/grafana/grafana.db
SQLite version 3.7.17 2013-05-20 00:56:22
Enter “.help” for instructions
Enter SQL statements terminated with a “;”
sqlite> INSERT INTO “org” VALUES(1,0,‘Main Org.’,NULL,NULL,NULL,NULL,NULL,NULL,NULL,‘2022-03-09 15:00:00’,‘2022-03-09 15:00:00’);
I added my user to the Main Org.
I can now load the main home dashboard, the inventory still shows all the servers I have added in the past but I can’t load any other dashboards - “Dashboard not found” error.