# Question regarding pbm\_config.yaml for aws S3 storage

**URL:** <https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996>\
**Category:** Percona Backup for MongoDB\
**Tags:** mongodb\
**Created:** [May 1, 2024, 8:33pm UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996 "2024-05-01T20:33:15Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jean66W](https://avatars.discourse-cdn.com/v4/letter/j/ce73a5/32.png) [@Jean66W](https://forums.percona.com/u/Jean66W)\
**Post date:** [May 1, 2024, 8:33pm UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996/1 "2024-05-01T20:33:15Z")

</div>

Hello,

I want to configure pbm-agent to backup to S3 with server-side encryption with Amazon S3 managed keys(SSE-S3) , don’t want to use custom key. If I am doing aws s3 cp command, the option is “–sse”.

How to configure above in pbm\_config.yamp file? I did it, but it shows error.

storage:  
type: s3  
s3:  
region: us-east-1  
bucket: xxxxxxxxxx  
prefix: data/pbm/backup  
serverSideEncryption:  
sseAlgorithm: aws:kms

After restarting pbm-agent, it shows the config like below:

[root@ip-10-88-3-46 etc]# pbm config  
pitr:  
enabled: false  
oplogSpanMin: 0  
compression: s2  
storage:  
type: s3  
s3:  
provider: aws  
region: us-east-1  
forcePathStyle: true  
bucket: xxxxxxxxx  
prefix: data/pbm/backup  
credentials: {}  
serverSideEncryption:  
sseAlgorithm: aws:kms  
kmsKeyID: “”  
sseCustomerAlgorithm: “”  
sseCustomerKey: “”  
maxUploadParts: 10000  
storageClass: STANDARD  
insecureSkipTLSVerify: false  
backup:  
oplogSpanMin: 0  
compression: s2

[root@ip-10-88-3-46 etc]# pbm logs  
2024-05-01T20:26:46Z E [rs0/10.88.3.46:27017] [agentCheckup] check storage connection: storage: no init file, attempt to create failed: upload to S3: KMS.ValidationException: 2 validation errors detected: Value ‘’ at ‘keyId’ failed to satisfy constraint: Member must satisfy regular expression pattern: ^\p{ASCII}+$; Value ‘’ at ‘keyId’ failed to satisfy constraint: Member must have length greater than or equal to 1  
status code: 400, request id: Z954FFXT09TRQQF9, host id: FJ/4Xz+JmbztxNy2Bq4tVs1UJM+sYwM38XG0ZwQ16BxZjijKiVZiDrHy+LEpyZR5hjrdz5QlsNE=  
2024-05-01T20:26:46Z E [rs0/10.88.1.179:27017] [agentCheckup] check storage connection: storage: no init file, attempt to create failed: upload to S3: KMS.ValidationException: 2 validation errors detected: Value ‘’ at ‘keyId’ failed to satisfy constraint: Member must satisfy regular expression pattern: ^\p{ASCII}+$; Value ‘’ at ‘keyId’ failed to satisfy constraint: Member must have length greater than or equal to 1

Can you please assist to get it working ? Thanks!!!

---

<div class="post-metadata">

**Author:** ![Jean66W](https://avatars.discourse-cdn.com/v4/letter/j/ce73a5/32.png) [@Jean66W](https://forums.percona.com/u/Jean66W)\
**Post date:** [May 2, 2024, 1:36pm UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996/3 "2024-05-02T13:36:33Z")

</div>

Thanks! I made change and restart pbm-agent, now the log shows permission errors:

[root@ip-10-88-1-11 etc]# pbm config  
pitr:  
enabled: false  
oplogSpanMin: 0  
compression: s2  
storage:  
type: s3  
s3:  
provider: aws  
region: us-east-1  
forcePathStyle: true  
bucket: xxxxxx  
prefix: data/pbm/backup  
credentials: {}  
maxUploadParts: 10000  
storageClass: STANDARD  
insecureSkipTLSVerify: false  
backup:  
oplogSpanMin: 0  
compression: s2  
[root@ip-10-88-1-11 etc]#

[root@ip-10-88-1-11 etc]# pbm logs  
2024-05-02T13:31:59Z E [rs0/10.88.3.28:27017] [agentCheckup] check storage connection: storage: no init file, attempt to create failed: upload to S3: AccessDenied: Access Denied  
status code: 403, request id: WV9N008X68XQM7KR, host id: 3jfy00hreZ+moM1mkFRFfJAb3WeJljfjUX5KTNj+uo2zbR0oh0mCQUkF/P0zkdtTT06VRYDmadE=  
2024-05-02T13:32:00Z E [rs0/10.88.2.25:27017] [agentCheckup] check storage connection: storage: no init file, attempt to create failed: upload to S3: AccessDenied: Access Denied

I have EC2 instance profile with read/write policy to the backup bucket. I can run aws cli command to upload file to S3 without error. What permission do I need to grant ? can you show me example ?

[root@ip-10-88-1-11 etc]# touch test-file  
[root@ip-10-88-1-11 etc]# aws s3 cp test-file s3://xxxxxx/data/pbm/backup --sse  
upload: ./test-file to s3://770079253716-mysql-backup/data/pbm/backup  
[root@ip-10-88-1-11 etc]#

---

<div class="post-metadata">

**Author:** ![Jean66W](https://avatars.discourse-cdn.com/v4/letter/j/ce73a5/32.png) [@Jean66W](https://forums.percona.com/u/Jean66W)\
**Post date:** [May 7, 2024, 5:55pm UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996/4 "2024-05-07T17:55:42Z")

</div>

@David_Clark David, Do you know how to fix above error ?

[root@ip-10-88-1-11 etc]# pbm logs  
2024-05-02T13:31:59Z E [rs0/10.88.3.28:27017] [agentCheckup] check storage connection: storage: no init file, attempt to create failed: upload to S3: AccessDenied: Access Denied  
status code: 403, request id: WV9N008X68XQM7KR, host id: 3jfy00hreZ+moM1mkFRFfJAb3WeJljfjUX5KTNj+uo2zbR0oh0mCQUkF/P0zkdtTT06VRYDmadE=  
2024-05-02T13:32:00Z E [rs0/10.88.2.25:27017] [agentCheckup] check storage connection: storage: no init file, attempt to create failed: upload to S3: AccessDenied: Access Denied

---

<div class="post-metadata">

**Author:** ![Jean66W](https://avatars.discourse-cdn.com/v4/letter/j/ce73a5/32.png) [@Jean66W](https://forums.percona.com/u/Jean66W)\
**Post date:** [May 10, 2024, 1:59pm UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996/6 "2024-05-10T13:59:08Z")

</div>

@David_Clark I follow this page to setup S3 bucket permissions

> **[Percona Backup for MongoDB - Overview](https://docs.percona.com/percona-backup-mongodb/details/storage-configuration.html#permissions-setup)**
>
> Percona Backup for MongoDB saves your files to a directory. Using pbm list, a user can scan this directory to find existing
> backups even if they never used pbm on their computer before.

I can run aws s3 cp command to upload files from EC2 to S3 bucket without issue. That’s why I am not sure what’s wrong here. Please see the following S3 policy.

---

<div class="post-metadata">

**Author:** ![Jean66W](https://avatars.discourse-cdn.com/v4/letter/j/ce73a5/32.png) [@Jean66W](https://forums.percona.com/u/Jean66W)\
**Post date:** [May 10, 2024, 2:01pm UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996/7 "2024-05-10T14:01:19Z")

</div>

I have IAM role profile defined to allow S3:\* action, mongo is running on EC2.

```auto
{
    "Version": "2012-10-17",
    "Statement": [
       {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::mongo-backup"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:DeleteObject"
            ],
            "Resource": "arn:aws:s3:::mongo-backup/*"
        }
    ]
}

```

---

<div class="post-metadata">

**Author:** ![Sandra\_Romanchenko](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sandra_romanchenko/32/84_2.png) [@Sandra\_Romanchenko](https://forums.percona.com/u/Sandra_Romanchenko)\
**Post date:** [August 16, 2024, 4:39pm UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996/8 "2024-08-16T16:39:06Z")

</div>

Hi @Jean66W

The provided permissions should be enough for PBM to work, however PBM currently supports only SSE-C and SSE-KMS methods.

> **[Percona Backup for MongoDB - Overview](https://docs.percona.com/percona-backup-mongodb/details/storage-configuration.html#server-side-encryption)**
>
> Percona Backup for MongoDB saves your files to a directory. Using pbm list, a user can scan this directory to find existing
> backups even if they never used pbm on their computer before.

Do I understand correctly that your bucket has some kind of policy which denies object upload unless server-side encryption is requested (like it’s provided in doc below)?

> **[Using server-side encryption with Amazon S3 managed keys (SSE-S3) - Amazon...](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingServerSideEncryption.html)**
>
> With server-side encryption, Amazon S3 manages encryption and decryption for you.

---

<div class="post-metadata">

**Author:** ![Sandra\_Romanchenko](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sandra_romanchenko/32/84_2.png) [@Sandra\_Romanchenko](https://forums.percona.com/u/Sandra_Romanchenko)\
**Post date:** [September 2, 2024, 10:50am UTC](https://forums.percona.com/t/question-regarding-pbm-config-yaml-for-aws-s3-storage/29996/9 "2024-09-02T10:50:42Z")

</div>

Hi @Jean66W

Please be informed that starting from 2.6.0 version PBM supports server-side encryption with Amazon S3 managed keys (SSE-S3), please find the details how to configure this in our docs:

> **[Percona Backup for MongoDB - Overview](https://docs.percona.com/percona-backup-mongodb/details/storage-configuration.html#sse-s3-encryption)**
>
> Percona Backup for MongoDB saves your files to a directory. Using pbm list, a user can scan this directory to find existing
> backups even if they never used pbm on their computer before.