# Xtrabackup with xbstream S3 endlessly prints “log scanned up to”

**URL:** <https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333>\
**Category:** Percona XtraBackup\
**Tags:** mysql, percona\
**Created:** [May 26, 2023, 12:45pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333 "2023-05-26T12:45:58Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ben\_Chiciudean](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ben_chiciudean/32/10759_2.png) [@Ben\_Chiciudean](https://forums.percona.com/u/Ben_Chiciudean)\
**Post date:** [May 26, 2023, 12:45pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/1 "2023-05-26T12:45:58Z")

</div>

I’m running this command:

```auto
xtrabackup --backup --compress --stream=xbstream --datadir=/var/lib/mysql/ --target-dir=/percona/backups \
--user=root --password=password | \
xbcloud put --storage=s3 \
--s3-endpoint='s3.amazonaws.com' \
--s3-bucket='percona-backups' \
$(date -I)-full_backup

```

If I run just xtrabackup without xbcloud it works fine, and the backup is saved locally. I have access to IAM Profile for AWS S3 with correct role, I tested it.

---

<div class="post-metadata">

**Author:** ![Ben\_Chiciudean](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ben_chiciudean/32/10759_2.png) [@Ben\_Chiciudean](https://forums.percona.com/u/Ben_Chiciudean)\
**Post date:** [May 26, 2023, 2:12pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/2 "2023-05-26T14:12:54Z")

</div>

if I try to do `xbcloud get s3://cashli-dev-percona-backups/test.txt` it just never returns the response on stdout. I have internet blocked on the instance but the s3 endpoints can be access by services, because it’s enabled on endpoints and also `aws s3 ls s3://your-bucket-name` works fine

---

<div class="post-metadata">

**Author:** ![Marcelo\_Altmann](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/marcelo_altmann/32/11299_2.png) [@Marcelo\_Altmann](https://forums.percona.com/u/Marcelo_Altmann)\
**Post date:** [May 26, 2023, 2:20pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/3 "2023-05-26T14:20:01Z")

</div>

Hi @Ben_Chiciudean , seems like you are having a firewall issue. xbcloud will have to make redirects if aws gives it a different endpoint. Can you please run xbcloud get command passing --verbose flag and share the results here.

You might want to obfuscate sensitive information such as your access and secret keys

Thanks

---

<div class="post-metadata">

**Author:** ![Ben\_Chiciudean](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ben_chiciudean/32/10759_2.png) [@Ben\_Chiciudean](https://forums.percona.com/u/Ben_Chiciudean)\
**Post date:** [May 26, 2023, 2:26pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/4 "2023-05-26T14:26:05Z")

</div>

this is with --verbose flag:

```auto
* Trying 169.254.169.254...
* TCP_NODELAY set
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> PUT /latest/api/token HTTP/1.1
Host: 169.254.169.254
Accept: */*
Accept-Encoding: gzip
X-aws-ec2-metadata-token-ttl-seconds: 21600
Content-Length: 0

```

I run it with `xbcloud get --verbose s3://my-bucket/test.txt` it gets stuck at the Content-Legth

---

<div class="post-metadata">

**Author:** ![Ben\_Chiciudean](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ben_chiciudean/32/10759_2.png) [@Ben\_Chiciudean](https://forums.percona.com/u/Ben_Chiciudean)\
**Post date:** [May 26, 2023, 2:27pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/5 "2023-05-26T14:27:35Z")

</div>

If I do:

```auto
curl -X GET http://169.254.169.254/latest/meta-data/iam/info

```

It retrieves the instance profile arn correctly but the `/latest/api/token` no response

---

<div class="post-metadata">

**Author:** ![Marcelo\_Altmann](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/marcelo_altmann/32/11299_2.png) [@Marcelo\_Altmann](https://forums.percona.com/u/Marcelo_Altmann)\
**Post date:** [May 26, 2023, 2:37pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/6 "2023-05-26T14:37:39Z")

</div>

Hi Ben,

This is how tokens via IAM works, We generate a temporary token, this token is used to fetch instance profile and retrieve the token used to authenticate into S3. Can you please those commands manually:

```auto
#Generate a token
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`

#Check if we have any instance profile (You should see your IAM role here)
curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/

#Retirve the instance profile data ( adjust YOUR_INSTANCE_PROFILE to the output from previous command)
curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/iam/security-credentials/YOUR_INSTANCE_PROFILE

```

Please send me the output of those

---

<div class="post-metadata">

**Author:** ![Ben\_Chiciudean](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ben_chiciudean/32/10759_2.png) [@Ben\_Chiciudean](https://forums.percona.com/u/Ben_Chiciudean)\
**Post date:** [May 26, 2023, 2:50pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/7 "2023-05-26T14:50:44Z")

</div>

it won’t generate the token:

```auto
bash-4.4$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
  % Total % Received % Xferd Average Speed Time Time Time Current
                                 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- 0:01:29 --:--:-- 0^C

```

how does it work for aws s3 cli?

---

<div class="post-metadata">

**Author:** ![Ben\_Chiciudean](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ben_chiciudean/32/10759_2.png) [@Ben\_Chiciudean](https://forums.percona.com/u/Ben_Chiciudean)\
**Post date:** [May 26, 2023, 2:59pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/8 "2023-05-26T14:59:53Z")

</div>

> [@Ben\_Chiciudean](#):
>
> `/latest/api/token`

I must be using `IMDSv1` for the EC2 Instance

---

<div class="post-metadata">

**Author:** ![Ben\_Chiciudean](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ben_chiciudean/32/10759_2.png) [@Ben\_Chiciudean](https://forums.percona.com/u/Ben_Chiciudean)\
**Post date:** [May 26, 2023, 4:08pm UTC](https://forums.percona.com/t/xtrabackup-with-xbstream-s3-endlessly-prints-log-scanned-up-to/22333/9 "2023-05-26T16:08:39Z")

</div>

I got it working by increasing the hop limit for metadata service:

```auto
aws ec2 modify-instance-metadata-options \
    --instance-id <instance_id> \
    --http-put-response-hop-limit 2 \
    --http-endpoint enabled

```

this made sure that I was able to access the `IMDSv2` from the docker container