Percona Backup to S3 - getting multipart error

Hi, I have to backup my DB and send the backup to S3 and while doing that Im getting
“An error occurred (InvalidArgument) when calling the UploadPart operation: Part number must be an integer between 1 and 10000, inclusive.”

Im using xtrabackup 2.4.18 and encrypting it and directly streaming it to S3 bucket.

xtrabackup --no-timestamp | gpg (parameters) | aws s3 cp - s3://bucket.

I was able to perform this test on small dataset of about 40G. and with 2.5TB the above operation is failing with the uploadpart error. Can someone advice here.

Thanks.

Hi there
Do you need to configure AWS for the size of the chunks? https://docs.aws.amazon.com/cli/latest/topic/s3-config.html#multipart-chunksize

You might also explore xbcloud in case that works for you although there may be an issue with encryption in that solution… https://www.percona.com/doc/percona-xtrabackup/2.3/xbcloud/xbcloud.html