Hi all,
Per page, Upgrade
I upgrade pmm-server from 2.19 to 2.23.
In summary section, it says upgrade includes four steps: stop container; backup container and data; pull the latest image; and run.
However, in the detail section, after " Run it", it shows “Performa a restore”. In which case, I have to restore after upgrade? I know a little about docker, but I see the mertrics looks okay after upgrade to 2.23. I’m not sure which are missing. Could you please advise?
===============>UPD: The previous metrics were lost … It looks like my upgrade had a problem. So, how to recover metrics after docker image is upgrade?
And one more question, Step 3 “Run it”. Does the operation missing a “-p 80:80”? I tried this and couldn’t access PMM website. Then, I removed the docker container and rerun with “-p 80:80”, it’s okay.
BTW: I created PMM dockers by the following commands:
docker create -v /srv --name pmm-data percona/pmm-server:2.19.0 /bin/true
docker run -d -p 80:80 -p 443:443 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2.19.0
[root@]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
percona/pmm-server 2 b031ff61b727 2 weeks ago 1.91GB
percona/pmm-server 2.23.0 b031ff61b727 2 weeks ago 1.91GB
percona/pmm-server 2.19.0 82d29be43d66 4 months ago 1.88GB
Thanks,
Dillon
Hi Dillon,
Could you provide an output of the next command?
docker ps -a
1 Like
Hi adivinho,
Thank you for advise again.
1 Like
It looks ok.
I tried to reproduce the issue but 2.19.0 data is shown correct on new pmm-server 2.23.0
e.g.
Could you check VM logs?
docker exec -it pmm-server grep "opened storage" /srv/logs/victoriametrics.log
1 Like
I’ll post the log message later.
But, I don’t believe this is a bug for upgrade. I’m not sure whether this cause the problem:
In document, it create pmm-data by
docker create -v /srv --name pmm-data percona/pmm-server:2 /bin/true
However, I use this:
docker create -v /srv --name pmm-data percona/pmm-server:2.19.0 /bin/true
Will this “tag” cause the problem?
Thanks,
Dillon
1 Like
Tag 2 is equal to the latest version. So it was 2.19.0 at the time of creating the container.
1 Like
It looks like logs have been already rotated.
Please try to check rotated logs.
e.g.
docker exec -it pmm-server grep "opened storage" /srv/logs/victoriametrics.log.1
docker exec -it pmm-server grep "opened storage" /srv/logs/victoriametrics.log.2
docker exec -it pmm-server grep "opened storage" /srv/logs/victoriametrics.log.3
1 Like
Yes. It have been rotated.
Please see screenshot.
1 Like
So VM pulled all old metrics after the migration. It had more than 5 millions of rows.
But after pmm-server restart all such data were cleaned up.
Please take a look at my installation screenshot. VM data are constantly increasing.
Maybe you could find out the reason of such behaviour in VM logs.
docker exec -it pmm-server less /srv/logs/victoriametrics.log.3
1 Like
Unfortunately, victoriametrics.log.3 was rotated and overwrote.
I tried to reproduct this problem but failed:
- stop pmm-server docker container for 2.23.
- remove pmm-server docker by “docker container rm container-id”
- run a new docker by “docker run --detach --restart always -p 80:80 -p 443:443 --volumes-from pmm-data --name pmm-server percona/pmm-server:2.23.0”
- the data still shows in the new PMM webUI.
May this was a mistake when I upgrade 2.19 to 2.23. I’ll check again and try to upgrade 2.23 against prod.
Thanks for advise.
Dillon
1 Like
Hi adivinho,
Thank for advise above. I tried to upgrade on prod environment. The data is okay.
1 Like
