Timeout on https://repo.percona.com

Hi,

I’m trying to run an update on my vServer located at Hetzner but it failed:

root@server :~# yum clean all
Loaded plugins: elrepo, post-transaction-actions, replace
Cleaning repos: base elrepo epel extras remi remi-php80 remi-php82 rspamd tools-release-noarch tools-release-x86_64 updates
Other repos take up 27 M of disk space (use --verbose for details)

root@server :~# yum update
Loaded plugins: elrepo, post-transaction-actions, replace
base                                                                                                                                                                   | 3.6 kB  00:00:00     
elrepo                                                                                                                                                                 | 3.0 kB  00:00:00     
epel/x86_64/metalink                                                                                                                                                   |  17 kB  00:00:00     
epel                                                                                                                                                                   | 4.7 kB  00:00:00     
extras                                                                                                                                                                 | 2.9 kB  00:00:00     
remi                                                                                                                                                                   | 3.0 kB  00:00:00     
remi-php80                                                                                                                                                             | 3.0 kB  00:00:00     
remi-php82                                                                                                                                                             | 3.0 kB  00:00:00     
rspamd/signature                                                                                                                                                       |  833 B  00:00:00     
rspamd/signature                                                                                                                                                       | 3.0 kB  00:00:00 !!! 
http://repo.percona.com/tools/yum/release/7/RPMS/noarch/repodata/repomd.xml: [Errno 12] Timeout on http://repo.percona.com/tools/yum/release/7/RPMS/noarch/repodata/repomd.xml: (28, 'Connection timed out after 30001 milliseconds')
Trying other mirror.

root@server :~# dig repo.percona.com
; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.14 <<>> repo.percona.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19692
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;repo.percona.com.		IN	A

;; ANSWER SECTION:
repo.percona.com.	81	IN	CNAME	repo.r53.percona.com.
repo.r53.percona.com.	106	IN	CNAME	eu.r53.percona.com.
eu.r53.percona.com.	222	IN	A	49.12.125.205

;; AUTHORITY SECTION:
r53.percona.com.	552	IN	NS	ns-1367.awsdns-42.org.
r53.percona.com.	552	IN	NS	ns-2000.awsdns-58.co.uk.
r53.percona.com.	552	IN	NS	ns-288.awsdns-36.com.
r53.percona.com.	552	IN	NS	ns-898.awsdns-48.net.

;; Query time: 1 msec
;; SERVER: 185.12.64.1#53(185.12.64.1)
;; WHEN: Do Aug 10 14:43:44 CEST 2023
;; MSG SIZE  rcvd: 238

root@server :~# tracepath -n repo.percona.com
 1?: [LOCALHOST]                                         pmtu 1500
 1:  172.31.1.1                                            5.602ms 
 1:  172.31.1.1                                            4.628ms 
 2:  116.203.162.34                                        1.259ms asymm  1 
 3:  no reply
 4:  88.198.253.21                                        23.320ms asymm  3 
 5:  no reply
 6:  213.239.203.105                                      19.745ms asymm  5 
 7:  213.239.252.21                                       10.181ms asymm  6 
 8:  5.101.109.194                                         4.698ms asymm  7 
 9:  no reply
10:  no reply
11:  no reply
12:  no reply
13:  no reply
14:  no reply
15:  no reply
16:  no reply
17:  no reply
18:  no reply
19:  no reply
20:  no reply
21:  no reply
22:  no reply
23:  no reply
24:  no reply
25:  no reply
26:  no reply
27:  no reply
28:  no reply
29:  no reply
30:  no reply
     Too many hops: pmtu 1500
     Resume: pmtu 1500 

The trace was made from 116.203.x.x

On July 25th I updated percona-xtrabackup, so the error occurs after that date

Any ideas?

with kind regards,
Bernd.

Hello @b.brodda,
The issue appears to be on your side. We have 10s of thousands of requests per day to our repo and there have been no other reports of issues. Have you tried from another machine not in the same network?

$ time curl --verbose http://repo.percona.com/tools/yum/release/7/RPMS/noarch/repodata/repomd.xml
*   Trying 147.135.54.159...
* TCP_NODELAY set
* Connected to repo.percona.com (147.135.54.159) port 80 (#0)
> GET /tools/yum/release/7/RPMS/noarch/repodata/repomd.xml HTTP/1.1
> Host: repo.percona.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 10 Aug 2023 16:02:08 GMT
< Content-Type: text/xml
< Content-Length: 2962
< Last-Modified: Tue, 19 Mar 2019 11:04:13 GMT
< Connection: keep-alive
< ETag: "5c90ccad-b92"
< Expires: Fri, 11 Aug 2023 16:02:08 GMT
< Cache-Control: max-age=86400
< Cache-Control: public
< Accept-Ranges: bytes
<
<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
 <revision>1552993452</revision>
<data type="filelists">
  <checksum type="sha256">401dc19bda88c82c403423fb835844d64345f7e95f5b9835888189c03834cc93</checksum>
  <open-checksum type="sha256">bf9808b81cb2dbc54b4b8e35adc584ddcaa73bd81f7088d73bf7dbbada961310</open-checksum>
  <location href="repodata/401dc19bda88c82c403423fb835844d64345f7e95f5b9835888189c03834cc93-filelists.xml.gz"/>
  <timestamp>1552993452</timestamp>
  <size>123</size>
  <open-size>125</open-size>
</data>
<data type="primary">
  <checksum type="sha256">dabe2ce5481d23de1f4f52bdcfee0f9af98316c9e0de2ce8123adeefa0dd08b9</checksum>
  <open-checksum type="sha256">e1e2ffd2fb1ee76f87b70750d00ca5677a252b397ab6c2389137a0c33e7b359f</open-checksum>
  <location href="repodata/dabe2ce5481d23de1f4f52bdcfee0f9af98316c9e0de2ce8123adeefa0dd08b9-primary.xml.gz"/>
  <timestamp>1552993452</timestamp>
  <size>134</size>
  <open-size>167</open-size>
</data>
<data type="primary_db">
  <checksum type="sha256">fd013707e27f8251b52dc4b6aea4c07c81c0b06cff0c0c26a5a257c3ae30fc06</checksum>
  <open-checksum type="sha256">a4b22266b4df02e314470facff02ecedc5c2ef0ceef5d6a9e1a3f0eb87666f60</open-checksum>
  <location href="repodata/fd013707e27f8251b52dc4b6aea4c07c81c0b06cff0c0c26a5a257c3ae30fc06-primary.sqlite.bz2"/>
  <timestamp>1552993453</timestamp>
  <database_version>10</database_version>
  <size>1156</size>
  <open-size>21504</open-size>
</data>
<data type="other_db">
  <checksum type="sha256">2f241a8387cf35372fd709be4ef6ec83b8a00cc744bb90f31d82bb27bdd80531</checksum>
  <open-checksum type="sha256">7bf958ff496e8adc4f671e720501f53bc6a40dda8bfc4e41006dfa33752b8169</open-checksum>
  <location href="repodata/2f241a8387cf35372fd709be4ef6ec83b8a00cc744bb90f31d82bb27bdd80531-other.sqlite.bz2"/>
  <timestamp>1552993452</timestamp>
  <database_version>10</database_version>
  <size>573</size>
  <open-size>6144</open-size>
</data>
<data type="other">
  <checksum type="sha256">6bf9672d0862e8ef8b8ff05a2fd0208a922b1f5978e6589d87944c88259cb670</checksum>
  <open-checksum type="sha256">e0ed5e0054194df036cf09c1a911e15bf2a4e7f26f2a788b6f47d53e80717ccc</open-checksum>
  <location href="repodata/6bf9672d0862e8ef8b8ff05a2fd0208a922b1f5978e6589d87944c88259cb670-other.xml.gz"/>
  <timestamp>1552993452</timestamp>
  <size>123</size>
  <open-size>121</open-size>
</data>
<data type="filelists_db">
  <checksum type="sha256">99a710d0cadf6e62be6c49c455ac355f1c65c1740da1bd829c1e3ee8d23fab02</checksum>
  <open-checksum type="sha256">fea62ed0d9588387c9310d9132363e1584029a7526a9874beacf2e986120668e</open-checksum>
  <location href="repodata/99a710d0cadf6e62be6c49c455ac355f1c65c1740da1bd829c1e3ee8d23fab02-filelists.sqlite.bz2"/>
  <timestamp>1552993452</timestamp>
  <database_version>10</database_version>
  <size>593</size>
  <open-size>7168</open-size>
</data>
</repomd>
* Connection #0 to host repo.percona.com left intact

real	0m0.591s


$ dig repo.percona.com

; <<>> DiG 9.11.3-1ubuntu1.17-Ubuntu <<>> repo.percona.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18684
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;repo.percona.com.		IN	A

;; ANSWER SECTION:
repo.percona.com.	798	IN	CNAME	repo.r53.percona.com.
repo.r53.percona.com.	198	IN	CNAME	us.r53.percona.com.
us.r53.percona.com.	36	IN	A	147.135.54.159


Maybe you can put the US IP into your /etc/hosts as a temp workaround?

$ dig repo.percona.com @1.1

; <<>> DiG 9.11.3-1ubuntu1.17-Ubuntu <<>> repo.percona.com @1.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36368
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;repo.percona.com.		IN	A

;; ANSWER SECTION:
repo.percona.com.	900	IN	CNAME	repo.r53.percona.com.
repo.r53.percona.com.	300	IN	CNAME	us.r53.percona.com.
us.r53.percona.com.	300	IN	A	147.135.54.159

;; Query time: 88 msec
;; SERVER: 1.0.0.1#53(1.0.0.1)
;; WHEN: Mon Aug 14 16:08:29 UTC 2023
;; MSG SIZE  rcvd: 101

Hi @matthewb

No, but someone put an entry in in /etc/hosts which was not working.

I removed the line and everything looks good now.

Anyhow, I’m wondering why the IP was sometimes ok, sometimes not.

Sorry for causing trouble.

with kind regards,
Bernd.