# Pbm restore failed: context deadline exceeded

**URL:** <https://forums.percona.com/t/pbm-restore-failed-context-deadline-exceeded/17494>\
**Category:** Percona Backup for MongoDB\
**Created:** [September 15, 2022, 3:22pm UTC](https://forums.percona.com/t/pbm-restore-failed-context-deadline-exceeded/17494 "2022-09-15T15:22:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dmitry\_Tulnov](https://avatars.discourse-cdn.com/v4/letter/d/a88e57/32.png) [@Dmitry\_Tulnov](https://forums.percona.com/u/Dmitry_Tulnov)\
**Post date:** [September 15, 2022, 3:22pm UTC](https://forums.percona.com/t/pbm-restore-failed-context-deadline-exceeded/17494/1 "2022-09-15T15:22:27Z")

</div>

Hi! I’m getting errors with PBM PITR restore.  
I really want to resolve this, please feel free to ask for any additional info, I’ll provide it ASAP  
i also tried to upgrade to the latest 1.8 version of PBM, but I got the same error  
All replicaSet nodes can access to s3 bucket and I can download pbm backup files with `aws s3` with no problems

Setup:

```auto
ReplicaSet with 3 nodes. 1 primary, 2 secondary
Nodes: AWS EC2 t3.large

```

How i start restore:

```auto
pbm config --set pitr.enabled=false
pbm restore --time="2022-09-15T14:15:56"

```

After a while these errors occur:

```auto

Sep 15 15:01:19 ip-10-0-14-38 pbm-agent[444]: 2022-09-15T15:01:19.000+0000 W [pitrestore/2022-09-15T14:15:56Z] got context deadline exceeded (Client.Timeout or context cancellation while reading body), try to reconnect in 10s
Sep 15 15:01:29 ip-10-0-14-38 pbm-agent[444]: 2022-09-15T15:01:29.000+0000 I [pitrestore/2022-09-15T14:15:56Z] session recreated, resuming download
Sep 15 15:01:29 ip-10-0-14-38 pbm-agent[444]: 2022-09-15T15:01:29.000+0000 E [pitrestore/2022-09-15T14:15:56Z] download 'private-bucket-for-backups/pbmPitr/meta-rs0/20220915/20220915140017-26.20220915141032-159.oplog.snappy' file from S3: context deadline exceeded (Client.Timeout or context cancellation while reading body)
Sep 15 15:01:29 ip-10-0-14-38 pbm-agent[444]: 2022-09-15T15:01:29.000+0000 E [pitrestore/2022-09-15T14:15:56Z] restore: replay chunk 1663250417.1663251032: apply oplog for chunk: download 'private-bucket-for-backups/pbmPitr/meta-rs0/20220915/20220915140017-26.20220915141032-159.oplog.snappy': context deadline exceeded (Client.Timeout or context cancellation while reading body)

```

**FULL log from the start of PBM restore** : [Sep 15 14:49:53 ip-10-0-14-38 pbm-agent[444]: 2022-09-15T14:49:53.000+0000 I got - Pastebin.com](https://pastebin.com/Hc97dEyi)

**Configs:**  
uname:

```auto
Linux ip-10-0-14-38 5.13.0-1019-aws #21~20.04.1-Ubuntu SMP Wed Mar 16 11:54:08 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

```

* * *

pbm version:

```auto
Version: 1.6.1
Platform: linux/amd64
GitCommit: aebbc24a03f21f9ddee6ebb72f510ff1d1c58fdc
GitBranch: release-1.6.1
BuildTime: 2021-11-03_12:12_UTC
GoVersion: go1.16.9

```

* * *

pbm-agent config:

```auto
pitr:
  enabled: true
  oplogSpanMin: 10
storage:
  type: s3
  s3:
    region: eu-north-1
    bucket: private-bucket-for-backups
    prefix: data/pbm/backup/meta
    credentials:
      access-key-id: <access key>
      secret-access-key: <secret key>

```

* * *

pbm list --full:

```auto
Backup snapshots:
  2022-09-15T13:03:14Z [complete: 2022-09-15T13:03:19]
  2022-09-15T14:00:10Z [complete: 2022-09-15T14:00:17]
  2022-09-15T14:21:48Z [complete: 2022-09-15T14:21:56]

PITR <off>:
  2022-09-15T13:03:20 - 2022-09-15T14:22:17

  meta-rs0: [{ 2022-09-15T13:03:19 - 2022-09-15T14:22:17}]

```

* * *

mongod config:

```auto
storage:
  oplogMinRetentionHours: 1
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27017
  bindIp: 0.0.0.0

processManagement:
  timeZoneInfo: /usr/share/zoneinfo

replication:
    replSetName: "meta-rs0"
security:
    authorization: enabled
    keyFile: /opt/mongo_rs_key_file

```

* * *

MongoDB:  
**No errors in MongoDB logs, just common info records**

---

<div class="post-metadata">

**Author:** ![sohahm](https://avatars.discourse-cdn.com/v4/letter/s/bc79bd/32.png) [@sohahm](https://forums.percona.com/u/sohahm)\
**Post date:** [June 13, 2023, 6:20pm UTC](https://forums.percona.com/t/pbm-restore-failed-context-deadline-exceeded/17494/2 "2023-06-13T18:20:38Z")

</div>

> [@Dmitry\_Tulnov](#):
>
> Sep 15 14:49:53 ip-10-0-14-38 pbm-agent[444]: 2022-09-15T14:49:53.000+0000 I got - [Pastebin.com](http://Pastebin.com)

I am having the same issue. Were you able to resolve it? Can you share your findings?  
For me it restores the entire cluster but fails in the end while trying to restore local.oplog.rs.bson.gz

---

<div class="post-metadata">

**Author:** ![Dmytro\_Zghoba](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/dmytro_zghoba/32/4721_2.png) [@Dmytro\_Zghoba](https://forums.percona.com/u/Dmytro_Zghoba)\
**Post date:** [June 14, 2023, 7:07am UTC](https://forums.percona.com/t/pbm-restore-failed-context-deadline-exceeded/17494/3 "2023-06-14T07:07:51Z")

</div>

Hi @sohahm ,

What PBM version do you use? If it was a bug, it could be fixed in the latest version.

But in the original description, I see a timeout during the download - network issue.

---

<div class="post-metadata">

**Author:** ![sujithtee](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sujithtee/32/18989_2.png) [@sujithtee](https://forums.percona.com/u/sujithtee)\
**Post date:** [June 20, 2023, 11:38am UTC](https://forums.percona.com/t/pbm-restore-failed-context-deadline-exceeded/17494/4 "2023-06-20T11:38:47Z")

</div>

I am facing the same issue while downloading from GCS bucket during restoration.