If the S3 bucket being backed up to is configured with Object Lock settings (GOVERNANCE in my case) the S3 file uploads fail with:
021-06-09 06:00:42.678 INFO: [SST script] 210609 06:00:32 xbcloud: Successfully connected.
2021-06-09 06:00:42.678 INFO: [SST script] 210609 06:00:42 xbcloud: Failed to upload object <redacted>/myqsl-2021-06-09-06:00:17-full/sys/sys_config.ibd.00000000000000000001. Error message: Content-MD5 HTTP header is required for Put Object requests with Object Lock parameters
How do we specify to the backup system to ensure it adds the Content-MD5 header to the requests.
The calculation logic appears present in: https://github.com/percona/percona-xtrabackup/blob/8.0/storage/innobase/xtrabackup/src/xbcloud/s3.cc
I’d also like to set the --header="x-amz-acl: bucket-owner-full-control" somehow. Don’t see anything in the documentation that is very clear about how to go about that.
1 Like
Hi @Gerwin_van_de_Steeg
How do we specify to the backup system to ensure it adds the Content-MD5 header to the requests.
The calculation logic appears present in: https://github.com/percona/percona-xtrabackup/blob/8.0/storage/innobase/xtrabackup/src/xbcloud/s3.cc
by doing a walk through the code, I can see we only append it when we are using sign V2, you can force it by setting --s3-api-version=2
I’d also like to set the --header="x-amz-acl: bucket-owner-full-control" somehow.
You are in the right path, setting --header option will append whatever header string you add to s3 requests. Are you not seeing it been set if you run xbcloud with --verbose option ?
1 Like
Hi Marcello,
The entire application is being deployed through the use of the helm charts for the PXC-Operator and the PXC-Database. Neither of which grant any form of control over setting additional command line options.
So unless they can be set through some form of configuration options, it is just not feasible to use this product.
1 Like
We have a task in our backlog to add custom flags for xtrabackup/xbstream/xbcloud: [K8SPXC-200] Add support for custom options for xtrabackup, xbstream, xbcloud - Percona JIRA
It was never a pressing issue as for most of the cases defaults are enough. We will see if this can be added easily.
@Gerwin_van_de_Steeg do you believe it would cover your use case?
1 Like