# Sharded instance for Mongodb failing to start **URL:** \ **Category:** Percona Distribution for MongoDB\ **Tags:** percona, closed-no-reply\ **Created:** [January 27, 2021, 10:30am UTC](https://forums.percona.com/t/sharded-instance-for-mongodb-failing-to-start/8960 "2021-01-27T10:30:58Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![alpikasingh](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@alpikasingh](https://forums.percona.com/u/alpikasingh)\ **Post date:** [January 27, 2021, 10:30am UTC](https://forums.percona.com/t/sharded-instance-for-mongodb-failing-to-start/8960/1 "2021-01-27T10:30:58Z")
Hi all, I reset my Mongo db clsuter. I deleted all the data from the data directory and reformatted the drive and remounted it. I ensure that mongod has the access to the log and data directory. However, when I start the mongod service it fails with the error below: I did delete the temp lock etc. ```auto [root@gtm-server31 ~]# /usr/bin/numactl --interleave=all /usr/bin/mongod --config /etc/mongodShard3-1.conf & [1] 16393 [root@server31 ~]# about to fork child process, waiting until server is ready for connections. forked process: 16395 ERROR: child process failed, exited with 48 ``` To see additional information in this output, start without the “–fork” option. ```auto [1]+ Exit 48 /usr/bin/numactl --interleave=all /usr/bin/mongod --config /etc/mongodShard3-1.conf [root@server31 ~]# cat /etc/mongodShard3-1.conf systemLog: destination: file logAppend: true path: /var/log/mongodb/mongod.log storage: dbPath: /var/lib/mongo journal: enabled: true engine: wiredTiger wiredTiger: engineConfig: journalCompressor: none cacheSizeGB: 70 collectionConfig: blockCompressor: none indexConfig: prefixCompression: false processManagement: fork: true # fork and run in background pidFilePath: /var/run/mongodb/mongod.pid # location of pidfile timeZoneInfo: /usr/share/zoneinfo sharding: clusterRole: shardsvr replication: replSetName: Shard3ReplSet net: port: 27018 bindIp: server31 ``` ```auto **log fine output:** {"t":{"$date":"2021-01-27T02:29:16.741-08:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"} {"t":{"$date":"2021-01-27T02:29:16.748-08:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"} {"t":{"$date":"2021-01-27T02:29:16.748-08:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."} {"t":{"$date":"2021-01-27T02:29:16.748-08:00"},"s":"I", "c":"STORAGE", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":19287,"port":27018,"dbPath":"/var/lib/mongo","architecture":"64-bit","host":"gtm-server31"}} {"t":{"$date":"2021-01-27T02:29:16.748-08:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.2-4","gitVersion":"5fca74b04eeebad5e7718e8ed65577b90de8bc5f","openSSLVersion":"OpenSSL 1.1.1g FIPS 21 Apr 2020","modules":[],"allocator":"tcmalloc","environment":{"distarch":"x86_64","target_arch":"x86_64"}}}} {"t":{"$date":"2021-01-27T02:29:16.748-08:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"CentOS Linux release 8.3.2011","version":"Kernel 4.18.0-240.10.1.el8_3.x86_64"}}} {"t":{"$date":"2021-01-27T02:29:16.748-08:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/etc/mongodShard3-1.conf","net":{"bindIp":"gtm-server31","port":27018},"processManagement":{"fork":true,"pidFilePath":"/var/run/mongodb/mongod.pid","timeZoneInfo":"/usr/share/zoneinfo"},"replication":{"replSetName":"Shard3ReplSet"},"sharding":{"clusterRole":"shardsvr"},"storage":{"dbPath":"/var/lib/mongo","engine":"wiredTiger","journal":{"enabled":true},"wiredTiger":{"collectionConfig":{"blockCompressor":"none"},"engineConfig":{"cacheSizeGB":70.0,"journalCompressor":"none"},"indexConfig":{"prefixCompression":false}}},"systemLog":{"destination":"file","logAppend":true,"path":"/var/log/mongodb/mongod.log"}}}} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"E", "c":"STORAGE", "id":20568, "ctx":"initandlisten","msg":"Error setting up listener","attr":{"error":{"code":9001,"codeName":"SocketException","errmsg":"Cannot assign requested address"}}} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"REPL", "id":4784900, "ctx":"initandlisten","msg":"Stepping down the ReplicationCoordinator for shutdown","attr":{"waitTimeMillis":10000}} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"COMMAND", "id":4784901, "ctx":"initandlisten","msg":"Shutting down the MirrorMaestro"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"SHARDING", "id":4784902, "ctx":"initandlisten","msg":"Shutting down the WaitForMajorityService"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"NETWORK", "id":4784905, "ctx":"initandlisten","msg":"Shutting down the global connection pool"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"NETWORK", "id":4784918, "ctx":"initandlisten","msg":"Shutting down the ReplicaSetMonitor"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"SHARDING", "id":4784921, "ctx":"initandlisten","msg":"Shutting down the MigrationUtilExecutor"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"CONTROL", "id":4784925, "ctx":"initandlisten","msg":"Shutting down free monitoring"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"FTDC", "id":4784926, "ctx":"initandlisten","msg":"Shutting down full-time data capture"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"STORAGE", "id":4784927, "ctx":"initandlisten","msg":"Shutting down the HealthLog"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"STORAGE", "id":4784929, "ctx":"initandlisten","msg":"Acquiring the global lock for shutdown"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"-", "id":4784931, "ctx":"initandlisten","msg":"Dropping the scope cache for shutdown"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"CONTROL", "id":20565, "ctx":"initandlisten","msg":"Now exiting"} {"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"I", "c":"CONTROL", "id":23138, "ctx":"initandlisten","msg":"Shutting down","attr":{"exitCode":48}} ``` Any suggestions to fix this? ---
**Author:** ![Ivan\_Groenewold](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_groenewold/32/6299_2.png) [@Ivan\_Groenewold](https://forums.percona.com/u/Ivan_Groenewold)\ **Post date:** [January 27, 2021, 4:16pm UTC](https://forums.percona.com/t/sharded-instance-for-mongodb-failing-to-start/8960/2 "2021-01-27T16:16:52Z")
Hi, please make sure that no mongod instances are left running. Error 48 means the port is unavailable ---
**Author:** ![alpikasingh](https://avatars.discourse-cdn.com/v4/letter/a/dc4da7/32.png) [@alpikasingh](https://forums.percona.com/u/alpikasingh)\ **Post date:** [January 27, 2021, 6:11pm UTC](https://forums.percona.com/t/sharded-instance-for-mongodb-failing-to-start/8960/3 "2021-01-27T18:11:56Z")
Thank you for responding. I ran netstat and I don’t see 27018 being used anywhere. Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:44321 0.0.0.0:\* LISTEN 4397/pmcd tcp 0 0 127.0.0.1:4330 0.0.0.0:\* LISTEN 4745/pmlogger tcp 0 0 0.0.0.0:111 0.0.0.0:\* LISTEN 1/systemd tcp 0 0 0.0.0.0:22 0.0.0.0:\* LISTEN 2457/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:\* LISTEN 2542/cupsd tcp6 0 0 ::1:44321 :::\* LISTEN 4397/pmcd tcp6 0 0 :::9090 :::\* LISTEN 1/systemd tcp6 0 0 ::1:4330 :::\* LISTEN 4745/pmlogger tcp6 0 0 :::111 :::\* LISTEN 1/systemd tcp6 0 0 :::22 :::\* LISTEN 2457/sshd tcp6 0 0 ::1:631 :::\* LISTEN 2542/cupsd netstat -tulpn Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:44321 0.0.0.0:\* LISTEN 4397/pmcd tcp 0 0 127.0.0.1:4330 0.0.0.0:\* LISTEN 4745/pmlogger tcp 0 0 0.0.0.0:111 0.0.0.0:\* LISTEN 1/systemd tcp 0 0 0.0.0.0:22 0.0.0.0:\* LISTEN 2457/sshd tcp 0 0 127.0.0.1:631 0.0.0.0:\* LISTEN 2542/cupsd tcp6 0 0 ::1:44321 :::\* LISTEN 4397/pmcd tcp6 0 0 :::9090 :::\* LISTEN 1/systemd tcp6 0 0 ::1:4330 :::\* LISTEN 4745/pmlogger tcp6 0 0 :::111 :::\* LISTEN 1/systemd tcp6 0 0 :::22 :::\* LISTEN 2457/sshd tcp6 0 0 ::1:631 :::\* LISTEN 2542/cupsd udp 0 0 0.0.0.0:111 0.0.0.0:\* 1/systemd udp 0 0 127.0.0.1:323 0.0.0.0:\* 2460/chronyd udp6 0 0 :::111 :::\* 1/systemd udp6 0 0 ::1:323 :::\* 2460/chronyd ---
**Author:** ![Akira\_Kurogane](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/akira_kurogane/32/1280_2.png) [@Akira\_Kurogane](https://forums.percona.com/u/Akira_Kurogane)\ **Post date:** [January 28, 2021, 2:55am UTC](https://forums.percona.com/t/sharded-instance-for-mongodb-failing-to-start/8960/4 "2021-01-28T02:55:45Z")
I agree with the diagnosis about a port being unavailable. This error message in the log file is the one that catches my eye: > [@alpikasingh](#): > > `{"t":{"$date":"2021-01-27T02:29:16.749-08:00"},"s":"E", "c":"STORAGE", "id":20568, "ctx":"initandlisten","msg":"Error setting up listener","attr":{"error":{"code":9001,"codeName":"SocketException","errmsg":"Cannot assign requested address"}}}` I wonder if it might be: - Some sort of double action. I.e. there are two mongod instances at the time, which are both dead again by the time you ran the netstat(?) command. - The unix socket is the one that cannot be bound. Look for another file, particularly one created by root or another user, at /tmp/mongodb-PORTNUMBER.sock. - If “server31”, used in the bindIP value, is not resolving to a local ethernet device’s IP address. ---
**Author:** ![radoslaw.szulgo](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/radoslaw.szulgo/32/17315_2.png) [@radoslaw.szulgo](https://forums.percona.com/u/radoslaw.szulgo)\ **Post date:** [July 16, 2024, 11:32am UTC](https://forums.percona.com/t/sharded-instance-for-mongodb-failing-to-start/8960/5 "2024-07-16T11:32:25Z")