# PBm agent killed while taking backup

**URL:** <https://forums.percona.com/t/pbm-agent-killed-while-taking-backup/17035>\
**Category:** MongoDB\
**Tags:** percona, pbm\
**Created:** [August 16, 2022, 3:18am UTC](https://forums.percona.com/t/pbm-agent-killed-while-taking-backup/17035 "2022-08-16T03:18:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rinki](https://avatars.discourse-cdn.com/v4/letter/r/cc9497/32.png) [@rinki](https://forums.percona.com/u/rinki)\
**Post date:** [August 16, 2022, 3:18am UTC](https://forums.percona.com/t/pbm-agent-killed-while-taking-backup/17035/1 "2022-08-16T03:18:09Z")

</div>

Hello Team,  
We are using percona for backup and restore of mongo and it has been working fine for most of cases.

PBM version - 1.6.1-1.el8.x86\_64  
mongo - 4.2.0  
DB - not sharded, running in cluster of 3 - one primary and two secondary  
Backup - triggered every mid night LDN

Recently, we have seen scenario where mongoPBMagent was killed after receiving command for backUP.  
Backup was started on one of secondary server, and then agent got killed

```auto
2022-08-14T16:36:53.000+0000 I pbm-agent:
Version: 1.6.1
Platform: linux/amd64
GitCommit: aebbc24a03f21f9ddee6ebb72f510ff1d1c58fdc
GitBranch: release-1.6.1
BuildTime: 2021-11-03_12:11_UTC
GoVersion: go1.16.9
2022-08-14T16:36:53.000+0000 I node: cat/uklvasapp1001.gdc.standardchartered.com:24242
2022-08-14T16:36:53.000+0000 I listening for the commands
2022-08-14T16:36:53.000+0000 I starting PITR routine
2022-08-15T23:30:00.000+0000 I got command backup [name: 2022-08-15T23:30:00Z, compression: s2] <ts: 1660606200>
2022-08-15T23:30:00.000+0000 I got epoch {1660495028 2}
2022-08-15T23:30:00.000+0000 I [backup/2022-08-15T23:30:00Z] backup started
2022-08-15T23:30:03.000+0000 D [backup/2022-08-15T23:30:00Z] wait for tmp users {1660606203 3501}
2022-08-15T23:30:05.039+0000	Setting num cpus to 32
2022-08-15T23:30:05.062+0000	enqueued collection 'admin.system.version'
2022-08-15T23:30:05.062+0000	enqueued collection 'admin.pbmLog'
2022-08-15T23:30:05.063+0000	enqueued collection 'admin.system.profile'
2022-08-15T23:30:05.063+0000	enqueued collection 'admin.pbmConfig'
2022-08-15T23:30:05.063+0000	enqueued collection 'admin.pbmBackups.old'
2022-08-15T23:30:05.063+0000	enqueued collection 'admin.pbmOpLog'
2022-08-15T23:30:05.064+0000	enqueued collection 'admin.pbmPITRChunks'
2022-08-15T23:30:05.064+0000	enqueued collection 'admin.pbmBackups'
2022-08-15T23:30:05.064+0000	enqueued collection 'admin.pbmRestores'
2022-08-15T23:30:05.064+0000	enqueued collection 'admin.pbmCmd'
2022-08-15T23:30:05.064+0000	enqueued collection 'admin.system.roles'
2022-08-15T23:30:05.065+0000	enqueued collection 'admin.pbmRRoles'
2022-08-15T23:30:05.065+0000	enqueued collection 'admin.pbmLockOp'
2022-08-15T23:30:05.065+0000	enqueued collection 'admin.pbmRUsers'
2022-08-15T23:30:05.065+0000	enqueued collection 'admin.pbmLock'
2022-08-15T23:30:05.065+0000	enqueued collection 'admin.system.users'
2022-08-15T23:30:05.065+0000	enqueued collection 'admin.pbmAgents'
2022-08-15T23:43:54.874+0000	writing cat.Tradecollect1 to archive on stdout
2022/08/15 23:43:54 [############............] cat.ccp55838/109278 (51.1%)
2022-08-15T23:43:54.883+0000	Mux close namespace cat.collection3
panic: send on closed channel

goroutine 784727 [running]:
github.com/mongodb/mongo-tools/common/archive.(*MuxIn).Open(0xc0000d4320, 0x0, 0xc0071e3c78)
	/mnt/jenkins/workspace/pbm-autobuild-RELEASE/test/rpmbuild/BUILD/src/github.com/percona/percona-backup-mongodb/vendor/github.com/mongodb/mongo-tools/common/archive/multiplexer.go:282 +0x31c
github.com/mongodb/mongo-tools/mongodump.(*MongoDump).dumpValidatedQueryToIntent(0xc0004ae370, 0xc0071e3ef8, 0xc0003ee280, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/mnt/jenkins/workspace/pbm-autobuild-RELEASE/test/rpmbuild/BUILD/src/github.com/percona/percona-backup-mongodb/vendor/github.com/mongodb/mongo-tools/mongodump/mongodump.go:666 +0x89
github.com/mongodb/mongo-tools/mongodump.(*MongoDump).dumpQueryToIntent(...)
	/mnt/jenkins/workspace/pbm-autobuild-RELEASE/test/rpmbuild/BUILD/src/github.com/percona/percona-backup-mongodb/vendor/github.com/mongodb/mongo-tools/mongodump/mongodump.go:633
github.com/mongodb/mongo-tools/mongodump.(*MongoDump).DumpIntent(0xc0004ae370, 0xc0003ee280, 0x0, 0x0, 0x0, 0x0)
	/mnt/jenkins/workspace/pbm-autobuild-RELEASE/test/rpmbuild/BUILD/src/github.com/percona/percona-backup-mongodb/vendor/github.com/mongodb/mongo-tools/mongodump/mongodump.go:616 +0x8fe
github.com/mongodb/mongo-tools/mongodump.(*MongoDump).DumpIntents.func1(0xc0004ae370, 0xc0004a84e0, 0x5)
	/mnt/jenkins/workspace/pbm-autobuild-RELEASE/test/rpmbuild/BUILD/src/github.com/percona/percona-backup-mongodb/vendor/github.com/mongodb/mongo-tools/mongodump/mongodump.go:501 +0x15a
created by github.com/mongodb/mongo-tools/mongodump.(*MongoDump).DumpIntents
	/mnt/jenkins/workspace/pbm-autobuild-RELEASE/test/rpmbuild/BUILD/src/github.com/percona/percona-backup-mongodb/vendor/github.com/mongodb/mongo-tools/mongodump/mongodump.go:490 +0x15b

```

Could you please let me know what might have triggered these errors.

Thanks

---

<div class="post-metadata">

**Author:** ![Ivan\_Groenewold](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_groenewold/32/6299_2.png) [@Ivan\_Groenewold](https://forums.percona.com/u/Ivan_Groenewold)\
**Post date:** [August 16, 2022, 12:08pm UTC](https://forums.percona.com/t/pbm-agent-killed-while-taking-backup/17035/2 "2022-08-16T12:08:53Z")

</div>

Hello, did you check the system messages? it might have been the operating system out-of-memory killer.