PMM shutdown flow

[zard14]

Hi everyone

Yesterday I shut down my pmm server, today I open that VM and shows the error message.

Finally, I found its Prometheus API down, then I checked the log.

service prometheus status -l

I found that it seems I don’t shut down the server in right way, and the database running recovery.

For a while, the dashboard is ok again (maybe 20 mins later).

I would like to ask how to shut down the whole system in right way?
Should I stop all the client servers to push data to pmm server? what kind of commands should I use?

Thank you.

[alexey.palazhchenko]

Hi,

I think the issue there that Prometheus shutdown process took more time that system waited for it. We will look into tweaking it. In the meantime, “service prometheus stop” before VM shutdown should help.

[zard14]

Thank you for your advice, I’ll run this command in my shell

[Mykola]

the situation should be improved in 1.3.2 (coming soon)
now systemd (and supervisord) should wait at least 5 minutes after SIGTERM, before SIGKILL.

thank you for your feedback.