# Could we stop Prometheus in docker gracefully?

**URL:** <https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117>\
**Category:** PMM 1.x\
**Created:** [January 10, 2018, 1:24am UTC](https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117 "2018-01-10T01:24:23Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![bckim](https://avatars.discourse-cdn.com/v4/letter/b/67e7ee/32.png) [@bckim](https://forums.percona.com/u/bckim)\
**Post date:** [January 10, 2018, 1:24am UTC](https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117/1 "2018-01-10T01:24:23Z")

</div>

Hi,

I have one quesion about stopping Prometheus in Docker.

when I restart PMM docker , Prometheus could be crashed like this :

.  
.  
.  
time=“2018-01-10T06:53:00Z” level=info msg=“5170000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:01Z” level=info msg=“5180000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:01Z” level=info msg=“5190000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:01Z” level=info msg=“5200000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:01Z” level=info msg=“5210000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:01Z” level=info msg=“5220000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:01Z” level=info msg=“5230000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:01Z” level=info msg=“5240000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5250000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5260000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5270000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5280000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5290000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5300000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5310000 archived metrics checked.” source=“crashrecovery.go:418”  
time=“2018-01-10T06:53:02Z” level=info msg=“5320000 archived metrics checked.” source=“crashrecovery.go:418”  
.  
.  
.

I think this log is related with recover process in prometheus.

but it takes too long time.

so I want to know how can we stop the prometheus gracefully without crash.

thanks.

---

<div class="post-metadata">

**Author:** ![Peter](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/peter/32/2_2.png) [@Peter](https://forums.percona.com/u/Peter)\
**Post date:** [January 10, 2018, 9:00am UTC](https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117/2 "2018-01-10T09:00:38Z")

</div>

Hi,

What PMM Version are you running ? Also how much memory do you have allocated for Metrics ?

---

<div class="post-metadata">

**Author:** ![bckim](https://avatars.discourse-cdn.com/v4/letter/b/67e7ee/32.png) [@bckim](https://forums.percona.com/u/bckim)\
**Post date:** [January 10, 2018, 6:37pm UTC](https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117/3 "2018-01-10T18:37:16Z")

</div>

hi,

My PMM version is 1.5.3

and I allocated 40G memory for prometheus.

and prometheus data file size is 174Gb.

thanks.

$ docker run -d   
-p 80:80   
-e METRICS\_MEMORY=48318382080 -e METRICS\_RETENTION=2160h -e QUERIES\_RETENTION=2160h -e METRICS\_RESOLUTION=5s   
–volumes-from pmm-data   
–name pmm-server   
–restart always   
percona/pmm-server:latest

---

<div class="post-metadata">

**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\
**Post date:** [January 11, 2018, 1:31am UTC](https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117/4 "2018-01-11T01:31:11Z")

</div>

Hi [bckim](https://percona.vanillacommunities.com/profile/x/x/41469),

the root of the issue: “docker stop” command sends TERM signal, docker wait 10 seconds and send KILL signal.  
but very large prometheus installations cannot dump gigabytes of memory to disk in 10 seconds.  
workaround - use “docker stop -t 300 pmm-server” command

---

<div class="post-metadata">

**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\
**Post date:** [January 11, 2018, 10:07am UTC](https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117/5 "2018-01-11T10:07:04Z")

</div>

also, you can add “–stop-timeout 600” option to “docker run” command, in this case, “docker stop” command should work better

---

<div class="post-metadata">

**Author:** ![bckim](https://avatars.discourse-cdn.com/v4/letter/b/67e7ee/32.png) [@bckim](https://forums.percona.com/u/bckim)\
**Post date:** [January 11, 2018, 6:47pm UTC](https://forums.percona.com/t/could-we-stop-prometheus-in-docker-gracefully/6117/6 "2018-01-11T18:47:07Z")

</div>

thank you.

I could solve this problem with your help.