AWS RDS connection error

Percona Monitoring and Management (PMM) PMM 2.x
1

Description:

[Detailed description of the issue or question]

I am monitoring by linking aws aurora mysql with pmm 2.32, but the AWS IAM access key is old, so when I reconnect the rds after issuing a new one, the error below occurs:
Also, the old access key does not connect.
However, there is no problem with monitoring the previously connected instance. I even restarted docker, but it still doesn’t work.

Steps to Reproduce:

[Step-by-step instructions on how to reproduce the issue, including any specific settings or configurations]

Version:

[Insert the version number of the software]

Logs:

[If applicable, include any relevant log files or error messages]
e[33mWARNe[0m[2023-09-08T08:48:04.591+00:00] RPC /management.RDS/DiscoverRDS done in 1.247351892s with gRPC error: rpc error: code = Unknown desc = InvalidParameterValue: Unrecognized engine name: aurora
status code: 400, request id: af7066a1-676b-496a-ae49-53f656682efc e[33mrequeste[0m=6c09d38c-4e24-11ee-a09c-0242ac110002

Expected Result:

[What the user expected to see or happen before the issue occurred]

Actual Result:

[What actually happened when the user encountered the issue]

Additional Information:

[Include any additional information that could be helpful to diagnose the issue, such as browser or device information]

2

Hello @anonymous_lee, can you please upgrade to the latest PMM Server 2.39? AWS changed the name of the aurora engine reported by their API and a fix was needed in PMM.

3

thank you for your answer
As you mentioned, I upgraded to pmm version 2.39, but an error occurred during the upgrade and the docker status is unhealthy.
I’m currently using pmm-server-backup_0927 and 2.39 isn’t working well, so I’ve reverted to 2.32.

image
image1082×190 8.83 KB

Detailed logs are attached below.

2023-09-27 08:35:34,311 INFO Included extra file “/etc/supervisord.d/alertmanager.ini” during parsing
2023-09-27 08:35:34,311 INFO Included extra file “/etc/supervisord.d/dbaas-controller.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/grafana.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/pmm.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/prometheus.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/qan-api2.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/supervisord.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/victoriametrics.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/vmalert.ini” during parsing
2023-09-27 08:35:34,312 INFO Included extra file “/etc/supervisord.d/vmproxy.ini” during parsing
2023-09-27 08:35:34,312 INFO Set uid to user 0 succeeded
2023-09-27 08:35:34,316 INFO RPC interface ‘supervisor’ initialized
2023-09-27 08:35:34,316 INFO supervisord started with pid 1
2023-09-27 08:35:35,320 INFO spawned: ‘pmm-update-perform-init’ with pid 12
2023-09-27 08:35:35,322 INFO spawned: ‘postgresql’ with pid 13
2023-09-27 08:35:35,325 INFO spawned: ‘clickhouse’ with pid 14
2023-09-27 08:35:35,327 INFO spawned: ‘grafana’ with pid 23
2023-09-27 08:35:35,330 INFO spawned: ‘nginx’ with pid 24
2023-09-27 08:35:35,333 INFO spawned: ‘victoriametrics’ with pid 25
2023-09-27 08:35:35,337 INFO spawned: ‘vmalert’ with pid 26
2023-09-27 08:35:35,347 INFO spawned: ‘alertmanager’ with pid 27
2023-09-27 08:35:35,350 INFO spawned: ‘vmproxy’ with pid 28
2023-09-27 08:35:35,354 INFO spawned: ‘qan-api2’ with pid 32
2023-09-27 08:35:35,357 INFO spawned: ‘pmm-managed’ with pid 33
2023-09-27 08:35:35,359 INFO spawned: ‘pmm-agent’ with pid 34
2023-09-27 08:35:35,375 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:35:35,399 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:35:35,500 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:35:36,160 INFO exited: pmm-update-perform-init (exit status 1; not expected)
2023-09-27 08:35:36,347 INFO success: postgresql entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,347 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,347 INFO success: victoriametrics entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,347 INFO success: vmalert entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,347 INFO success: alertmanager entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,348 INFO success: vmproxy entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,562 INFO spawned: ‘clickhouse’ with pid 108
2023-09-27 08:35:36,566 INFO spawned: ‘grafana’ with pid 109
2023-09-27 08:35:36,569 INFO spawned: ‘qan-api2’ with pid 110
2023-09-27 08:35:36,570 INFO success: pmm-managed entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,570 INFO success: pmm-agent entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-09-27 08:35:36,584 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:35:36,596 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:35:36,653 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:35:37,164 INFO spawned: ‘pmm-update-perform-init’ with pid 123
2023-09-27 08:35:37,891 INFO exited: pmm-update-perform-init (exit status 1; not expected)
2023-09-27 08:35:38,891 INFO spawned: ‘clickhouse’ with pid 352
2023-09-27 08:35:38,894 INFO spawned: ‘grafana’ with pid 353
2023-09-27 08:35:38,897 INFO spawned: ‘qan-api2’ with pid 354
2023-09-27 08:35:38,917 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:35:38,925 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:35:39,011 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:35:40,096 INFO spawned: ‘pmm-update-perform-init’ with pid 384
2023-09-27 08:35:40,959 INFO exited: pmm-update-perform-init (exit status 1; not expected)
2023-09-27 08:35:41,963 INFO spawned: ‘grafana’ with pid 434
2023-09-27 08:35:41,967 INFO spawned: ‘qan-api2’ with pid 435
2023-09-27 08:35:41,978 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:35:41,982 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:35:42,987 INFO spawned: ‘clickhouse’ with pid 444
2023-09-27 08:35:43,079 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:35:44,115 INFO spawned: ‘pmm-update-perform-init’ with pid 451
2023-09-27 08:35:44,833 INFO exited: pmm-update-perform-init (exit status 1; not expected)
2023-09-27 08:35:45,109 INFO gave up: pmm-update-perform-init entered FATAL state, too many start retries too quickly
2023-09-27 08:35:46,119 INFO spawned: ‘grafana’ with pid 476
2023-09-27 08:35:46,124 INFO spawned: ‘qan-api2’ with pid 477
2023-09-27 08:35:46,132 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:35:46,137 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:35:47,113 INFO spawned: ‘clickhouse’ with pid 503
2023-09-27 08:35:47,202 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:35:51,263 INFO spawned: ‘grafana’ with pid 552
2023-09-27 08:35:51,265 INFO spawned: ‘qan-api2’ with pid 553
2023-09-27 08:35:51,277 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:35:51,281 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:35:52,392 INFO spawned: ‘clickhouse’ with pid 573
2023-09-27 08:35:52,476 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:35:57,411 INFO spawned: ‘grafana’ with pid 645
2023-09-27 08:35:57,413 INFO spawned: ‘qan-api2’ with pid 646
2023-09-27 08:35:57,423 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:35:57,428 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:35:58,498 INFO spawned: ‘clickhouse’ with pid 659
2023-09-27 08:35:58,584 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:36:04,494 INFO spawned: ‘grafana’ with pid 681
2023-09-27 08:36:04,497 INFO spawned: ‘qan-api2’ with pid 682
2023-09-27 08:36:04,508 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:36:04,512 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:36:05,847 INFO spawned: ‘clickhouse’ with pid 695
2023-09-27 08:36:05,933 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:36:12,849 INFO spawned: ‘grafana’ with pid 708
2023-09-27 08:36:12,852 INFO spawned: ‘qan-api2’ with pid 709
2023-09-27 08:36:12,861 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:36:12,865 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:36:14,135 INFO spawned: ‘clickhouse’ with pid 716
2023-09-27 08:36:14,235 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:36:21,866 INFO spawned: ‘grafana’ with pid 735
2023-09-27 08:36:21,868 INFO spawned: ‘qan-api2’ with pid 736
2023-09-27 08:36:21,877 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:36:21,881 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:36:23,457 INFO spawned: ‘clickhouse’ with pid 742
2023-09-27 08:36:23,544 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:36:32,503 INFO spawned: ‘grafana’ with pid 761
2023-09-27 08:36:32,506 INFO spawned: ‘qan-api2’ with pid 762
2023-09-27 08:36:32,516 INFO exited: grafana (exit status 2; not expected)
2023-09-27 08:36:32,527 INFO gave up: grafana entered FATAL state, too many start retries too quickly
2023-09-27 08:36:32,528 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:36:33,783 INFO spawned: ‘clickhouse’ with pid 774
2023-09-27 08:36:33,873 INFO exited: clickhouse (exit status 232; not expected)
2023-09-27 08:36:34,874 INFO gave up: clickhouse entered FATAL state, too many start retries too quickly
2023-09-27 08:36:43,859 INFO spawned: ‘qan-api2’ with pid 794
2023-09-27 08:36:43,873 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:36:55,912 INFO spawned: ‘qan-api2’ with pid 827
2023-09-27 08:36:55,926 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:37:09,338 INFO spawned: ‘qan-api2’ with pid 860
2023-09-27 08:37:09,352 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:37:23,900 INFO spawned: ‘qan-api2’ with pid 890
2023-09-27 08:37:23,913 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:37:39,422 INFO spawned: ‘qan-api2’ with pid 928
2023-09-27 08:37:39,436 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:37:55,915 INFO spawned: ‘qan-api2’ with pid 970
2023-09-27 08:37:55,929 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:38:13,091 INFO spawned: ‘qan-api2’ with pid 1009
2023-09-27 08:38:13,109 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:38:31,914 INFO spawned: ‘qan-api2’ with pid 1051
2023-09-27 08:38:31,928 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:38:51,135 INFO spawned: ‘qan-api2’ with pid 1099
2023-09-27 08:38:51,149 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:39:11,660 INFO spawned: ‘qan-api2’ with pid 1148
2023-09-27 08:39:11,674 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:39:33,593 INFO spawned: ‘qan-api2’ with pid 1197
2023-09-27 08:39:33,613 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:39:55,921 INFO spawned: ‘qan-api2’ with pid 1245
2023-09-27 08:39:55,940 INFO exited: qan-api2 (exit status 1; not expected)
2023-09-27 08:40:17,867 WARN received SIGTERM indicating exit request
2023-09-27 08:40:17,867 INFO waiting for postgresql, nginx, victoriametrics, vmalert, alertmanager, vmproxy, qan-api2, pmm-managed, pmm-agent to die
2023-09-27 08:40:17,902 INFO stopped: pmm-agent (exit status 0)
2023-09-27 08:40:20,906 INFO waiting for postgresql, nginx, victoriametrics, vmalert, alertmanager, vmproxy, qan-api2, pmm-managed to die
2023-09-27 08:40:20,918 INFO stopped: pmm-managed (exit status 0)
2023-09-27 08:40:20,919 INFO stopped: vmproxy (terminated by SIGINT)
2023-09-27 08:40:20,949 INFO stopped: alertmanager (exit status 0)
2023-09-27 08:40:20,951 INFO stopped: vmalert (exit status 0)
2023-09-27 08:40:21,089 INFO stopped: victoriametrics (exit status 0)
2023-09-27 08:40:21,449 INFO stopped: nginx (exit status 0)
2023-09-27 08:40:23,471 INFO stopped: postgresql (exit status 0)