# Innobackupex / xbstream (--parallel)

**URL:** <https://forums.percona.com/t/innobackupex-xbstream-parallel/26240>\
**Category:** Percona XtraBackup\
**Created:** [October 30, 2023, 12:00pm UTC](https://forums.percona.com/t/innobackupex-xbstream-parallel/26240 "2023-10-30T12:00:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Apostolos\_Karavias](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Apostolos\_Karavias](https://forums.percona.com/u/Apostolos_Karavias)\
**Post date:** [October 30, 2023, 12:00pm UTC](https://forums.percona.com/t/innobackupex-xbstream-parallel/26240/1 "2023-10-30T12:00:01Z")

</div>

Hi everyone,

I’ve setup a demo and the procedure is to retrieve the backup and automate the restore to validate that my backup is right and accesible.

I want to make the process as fast as possible, so I tried to use the **–parallel** (and use memory for example on **–apply-log** ). My question is on how to calculate the appropriate value of **–parallel** option? Is it based on CPU usage? Currently on a VPS with 6 cores I get the following using htop, while using _ **–parallel=32** _.

 ![htop](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/9/9b3cdf3d25b7331ada23ecbaf165ce1fd62841e0.png)

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![Marcelo\_Altmann](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/marcelo_altmann/32/11299_2.png) [@Marcelo\_Altmann](https://forums.percona.com/u/Marcelo_Altmann)\
**Post date:** [October 30, 2023, 12:32pm UTC](https://forums.percona.com/t/innobackupex-xbstream-parallel/26240/2 "2023-10-30T12:32:22Z")

</div>

Hi @Apostolos_Karavias

–parallel in the context of xbstream is the number of parallel files that can be processed at the same time.  
This will be IO-bound. However, xbstream is reading data from STDIN, which means the parallel throughput comes from the counterpart (most likely xbcloud).

You need xbcloud and our network capable of retrieving multiple file chunks in parallel and sending them to xbstream in order for parallel to give you any advantage.

---

<div class="post-metadata">

**Author:** ![Apostolos\_Karavias](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Apostolos\_Karavias](https://forums.percona.com/u/Apostolos_Karavias)\
**Post date:** [October 30, 2023, 12:37pm UTC](https://forums.percona.com/t/innobackupex-xbstream-parallel/26240/3 "2023-10-30T12:37:42Z")

</div>

Hi @Marcelo_Altmann thanks for replying. Currently the xbstream file is located locally on the machine, so I guess it’s just IO limited from the disk perspective? Meaning it’s safe to increase the number and check the results in terms of speed.

---

<div class="post-metadata">

**Author:** ![Marcelo\_Altmann](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/marcelo_altmann/32/11299_2.png) [@Marcelo\_Altmann](https://forums.percona.com/u/Marcelo_Altmann)\
**Post date:** [October 30, 2023, 1:57pm UTC](https://forums.percona.com/t/innobackupex-xbstream-parallel/26240/4 "2023-10-30T13:57:36Z")

</div>

Hi @Apostolos_Karavias ,

Correct. In your case, you might see a lot of benefits from parallel if your disk is slow in writing. Each thread on xbstream does the following in a loop:

1. Take an exclusive lock on reading from STDIN
2. Read a chunk of data from STDIN to memory
3. Release the lock on STDIN. (At this point another thread can acquire the lock on STDIN and read a new chunk)
4. Write the memory chunk to disk.

If step 4 is slow, you will have a lot of benefits from having a higher number of parallel threads, if the write to disk is fast, having the number of parallel threads like 32 most likely will not give you any benefit.

I would start by measuring restore time with 1 thread, then increase it to 2, 3, … until you reach a steady figure in terms of duration of restore.

If you have things like compression and encryption this might change the picture a bit.

---

<div class="post-metadata">

**Author:** ![Apostolos\_Karavias](https://avatars.discourse-cdn.com/v4/letter/a/dbc845/32.png) [@Apostolos\_Karavias](https://forums.percona.com/u/Apostolos_Karavias)\
**Post date:** [October 30, 2023, 1:59pm UTC](https://forums.percona.com/t/innobackupex-xbstream-parallel/26240/5 "2023-10-30T13:59:27Z")

</div>

Thank you for clearing up this one! I’m gonna try increasing and see the results.