"Error": "driver failed programming external connectivity on endpoint pmm-server

[Bryan_Dees]

My server was hung, and I restarted it, but ran into this error. I noticed its attempting to create a chain on 172.17.0.2 instead of 172.17.0.1 which is assigned to Docker0 Interface. Is that expected?

I’ve tried removing it, running it again, creating the chain manually.

docker run --detach --restart always
-e METRICS_RETENTION=720h
–publish 5443:443
–volumes-from pmm-data
–name pmm-server
-v /etc/pmm-certs
percona/pmm-server:2

“Error”: “driver failed programming external connectivity on endpoint pmm-server (2b92ca2207dad029ba5da251b5403be521d7e8559fc002e47d68611f373c5cd7): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 5443 -j DNAT --to-destination 172.17.0.2:443 ! -i docker0: iptables: No chain/target/match by that name.\n (exit status 1))”,

docker version
Client:
Version: 20.10.7
API version: 1.41
Go version: go1.15.14
Git commit: f0df350
Built: Wed Nov 17 03:05:36 2021
OS/Arch: linux/amd64
Context: default
Experimental: true

Server:
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.15.14
Git commit: b0f5bc3
Built: Wed Nov 17 03:06:14 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.6
GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc:
Version: 1.0.0
GitCommit: 84113eef6fc27af1b01b3181f31bbaf708715301
docker-init:
Version: 0.19.0
GitCommit: de40ad0

iptables -L DOCKER
Chain DOCKER (0 references)
target prot opt source destination

ifconfig docker0
docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
inet6 fe80::42:5ff:fe15:f1d3 prefixlen 64 scopeid 0x20
ether 02:42:05:15:f1:d3 txqueuelen 0 (Ethernet)
RX packets 7199767 bytes 1906605496 (1.7 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8377388 bytes 51283528836 (47.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[Vadim_Yalovets]

Hi @Bryan_Dees,

Have you tried to specify an IP for the container?

e.g.
❯ docker run -d -p 80:80 --name pmm-server --restart always --net myInternalNetwork --ip 172.20.0.10 percona/pmm-server:2

[Bryan_Dees]

[root@aws-util01c ~]# docker network inspect bridge
[
{
“Name”: “bridge”,
“Id”: “731b8ea5191db3f8282fe03181365cec73471e87c230941e5b1602deb793e8aa”,
“Created”: “2022-03-20T08:37:26.199184404-07:00”,
“Scope”: “local”,
“Driver”: “bridge”,
“EnableIPv6”: false,
“IPAM”: {
“Driver”: “default”,
“Options”: null,
“Config”: [
{
“Subnet”: “172.17.0.0/16”,
“Gateway”: “172.17.0.1”
}
]
},
“Internal”: false,
“Attachable”: false,
“Ingress”: false,
“ConfigFrom”: {
“Network”: “”
},
“ConfigOnly”: false,
“Containers”: {},
“Options”: {
“com.docker.network.bridge.default_bridge”: “true”,
“com.docker.network.bridge.enable_icc”: “true”,
“com.docker.network.bridge.enable_ip_masquerade”: “true”,
“com.docker.network.bridge.host_binding_ipv4”: “0.0.0.0”,
“com.docker.network.bridge.name”: “docker0”,
“com.docker.network.driver.mtu”: “1500”
},
“Labels”: {}
}
]

docker run --detach --restart always
-e METRICS_RETENTION=720h
–publish 5443:443
–volumes-from pmm-data
–name pmm-server
–net bridge
-v /etc/pmm-certs
percona/pmm-server:2

returns:

446dc291f6babb356a8b110cb8e71663e175ff5853075a7b7d3fbc24f66b42f9
docker: Error response from daemon: driver failed programming external connectivity on endpoint pmm-server (c0375155d0629960a1462cd5af8a7b7af051a3b36cca8c14425f25cbd36fc876): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 5443 -j DNAT --to-destination 172.17.0.2:443 ! -i docker0: iptables: No chain/target/match by that name.
(exit status 1)).

If I add an IP say, 172.17.0.2 it returns:

docker run --detach --restart always \

-e METRICS_RETENTION=720h
–publish 5443:443
–volumes-from pmm-data
–name pmm-server
–net host --ip 10.100.110.21
-v /etc/pmm-certs
percona/pmm-server:2
WARNING: Published ports are discarded when using host network mode
cd05df58a4637e99b4fc43b397f8d5b0f6524211bb9f9d0f6c46fd81daeac9d8
docker: Error response from daemon: user specified IP address is supported on user defined networks only.

[Vadim_Yalovets]

It’s required to be created a separate network.

If “bridge” network is used then docker replies “User specified IP address is supported on user defined networks only”.

[Bryan_Dees]

@Vadim_Yalovets

Thank you for the assistance!

I figure it out. I was missing this chain:

iptables

Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp – anywhere ip-172-17-0-2.us-west-2.compute.internal tcp dpt:https