# what is "Provider paused at "mean **URL:** \ **Category:** Percona XtraDB Cluster 5.x\ **Created:** [August 23, 2017, 4:33am UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824 "2017-08-23T04:33:13Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![dixon](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@dixon](https://forums.percona.com/u/dixon)\ **Post date:** [August 23, 2017, 4:33am UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824/1 "2017-08-23T04:33:13Z")
hi,all,when we use percona cluster in our pruduction box,we away got this msg in error log.what is it mean? 2017-08-20 04:02:30 29346 [Note] WSREP: Provider resumed. 2017-08-21 04:02:36 29346 [Note] WSREP: Provider paused at 71ff9701-81a2-11e7-aa25-5f865e134a58:313477 (287475) 2017-08-21 04:02:42 29346 [Note] WSREP: resuming provider at 287475 2017-08-21 04:02:42 29346 [Note] WSREP: Provider resumed. 2017-08-22 04:03:48 29346 [Note] WSREP: Provider paused at 71ff9701-81a2-11e7-aa25-5f865e134a58:389024 (364778) 2017-08-22 04:03:54 29346 [Note] WSREP: resuming provider at 364778 2017-08-22 04:03:54 29346 [Note] WSREP: Provider resumed. 2017-08-23 04:04:15 29346 [Note] WSREP: Provider paused at 71ff9701-81a2-11e7-aa25-5f865e134a58:461257 (438699) 2017-08-23 04:04:20 29346 [Note] WSREP: resuming provider at 438699 2017-08-23 04:04:20 29346 [Note] WSREP: Provider resumed. ---
**Author:** ![miguelangelnieto](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@miguelangelnieto](https://forums.percona.com/u/miguelangelnieto)\ **Post date:** [August 24, 2017, 7:02am UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824/2 "2017-08-24T07:02:23Z")
Hello, That could be caused by flow control. Please read the following link: [url][http://galeracluster.com/documentation-webpages/nodestates.html[/url]](http://galeracluster.com/documentation-webpages/nodestates.html%5B/url%5D) [url][http://galeracluster.com/documentation-webpages/managingfc.html[/url]](http://galeracluster.com/documentation-webpages/managingfc.html%5B/url%5D) So, a node could have a large queue of transactions to apply and therefore is asking the other nodes to pause. ---
**Author:** ![dixon](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@dixon](https://forums.percona.com/u/dixon)\ **Post date:** [August 24, 2017, 8:17pm UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824/3 "2017-08-24T20:17:43Z")
> [@miguelangelnieto;49469](#): > > Hello, > > That could be caused by flow control. Please read the following link: > > [http://galeracluster.com/documentation-webpages/nodestates.html](http://galeracluster.com/documentation-webpages/nodestates.html) > [http://galeracluster.com/documentation-webpages/managingfc.html](http://galeracluster.com/documentation-webpages/managingfc.html) > > So, a node could have a large queue of transactions to apply and therefore is asking the other nodes to pause. ------------------------------±-------------------------------------+ | wsrep\_local\_state\_uuid | 71ff9701-81a2-11e7-aa25-5f865e134a58 | | wsrep\_protocol\_version | 5 | | wsrep\_last\_committed | 602801 | | wsrep\_replicated | 302309 | | wsrep\_replicated\_bytes | 7776478820 | | wsrep\_repl\_keys | 17804807 | | wsrep\_repl\_keys\_bytes | 149561272 | | wsrep\_repl\_data\_bytes | 7101450919 | | wsrep\_repl\_other\_bytes | 0 | | wsrep\_received | 281437 | | wsrep\_received\_bytes | 2788159496 | | wsrep\_local\_commits | 302288 | | wsrep\_local\_cert\_failures | 15 | | wsrep\_local\_replays | 0 | | wsrep\_local\_send\_queue | 0 | | wsrep\_local\_send\_queue\_avg | 0.000122 | | wsrep\_local\_recv\_queue | 0 | | wsrep\_local\_recv\_queue\_avg | 62.377498 | | wsrep\_local\_cached\_downto | 476116 | | wsrep\_flow\_control\_paused\_ns | 626083319 | | wsrep\_flow\_control\_paused | 0.000001 | | wsrep\_flow\_control\_sent | 0 | | wsrep\_flow\_control\_recv | 5 | | wsrep\_cert\_deps\_distance | 48.080979 | | wsrep\_apply\_oooe | 0.012610 | | wsrep\_apply\_oool | 0.000310 | | wsrep\_apply\_window | 1.073725 | | wsrep\_commit\_oooe | 0.000000 | | wsrep\_commit\_oool | 0.000062 | | wsrep\_commit\_window | 1.056073 | | wsrep\_local\_state | 4 | | wsrep\_local\_state\_comment | Synced | | wsrep\_cert\_index\_size | 733 | | wsrep\_causal\_reads | 64131943 | | wsrep\_cert\_interval | 0.004100 | | wsrep\_incoming\_addresses | , | | wsrep\_cluster\_conf\_id | 9 | | wsrep\_cluster\_size | 3 | | wsrep\_cluster\_state\_uuid | 71ff9701-81a2-11e7-aa25-5f865e134a58 | | wsrep\_cluster\_status | Primary | | wsrep\_connected | ON | | wsrep\_local\_bf\_aborts | 307 | | wsrep\_local\_index | 1 | | wsrep\_provider\_name | Galera | | wsrep\_provider\_vendor | Codership Oy \ | | wsrep\_provider\_version | 3.5(r178) | | wsrep\_ready | ON ---
**Author:** ![dixon](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@dixon](https://forums.percona.com/u/dixon)\ **Post date:** [August 24, 2017, 8:19pm UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824/4 "2017-08-24T20:19:23Z")
> [@miguelangelnieto;49469](#): > > Hello, > > That could be caused by flow control. Please read the following link: > > [http://galeracluster.com/documentation-webpages/nodestates.html](http://galeracluster.com/documentation-webpages/nodestates.html) > [http://galeracluster.com/documentation-webpages/managingfc.html](http://galeracluster.com/documentation-webpages/managingfc.html) > > So, a node could have a large queue of transactions to apply and therefore is asking the other nodes to pause. i have post our cluster status pamerater,can you have a look it,is my cluster going to bad ??? ---
**Author:** ![miguelangelnieto](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@miguelangelnieto](https://forums.percona.com/u/miguelangelnieto)\ **Post date:** [September 5, 2017, 2:32am UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824/5 "2017-09-05T02:32:53Z")
From the status I see that flow control happens in your cluster: | wsrep\_local\_recv\_queue\_avg | 62.377498 | | wsrep\_local\_cached\_downto | 476116 | | wsrep\_flow\_control\_paused\_ns | 626083319 | | wsrep\_flow\_control\_paused | 0.000001 | | wsrep\_flow\_control\_sent | 0 | | wsrep\_flow\_control\_recv | 5 | and also, that since you are writing to multiple nodes at the same time there are many deadlocks errors: | wsrep\_local\_bf\_aborts | 307 | So, I would recommend you to write on a single node and also increase gcs.fc\_limit in case you have it configured to its default value. That variable defines the number of transactions in the queue that will trigger flow control. The default number is not usually good for production. ---
**Author:** ![dixon](https://avatars.discourse-cdn.com/v4/letter/d/c0e974/32.png) [@dixon](https://forums.percona.com/u/dixon)\ **Post date:** [September 5, 2017, 3:18am UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824/6 "2017-09-05T03:18:08Z")
> [@miguelangelnieto;49537](#): > > From the status I see that flow control happens in your cluster: > > | wsrep\_local\_recv\_queue\_avg | 62.377498 | > | wsrep\_local\_cached\_downto | 476116 | > | wsrep\_flow\_control\_paused\_ns | 626083319 | > | wsrep\_flow\_control\_paused | 0.000001 | > | wsrep\_flow\_control\_sent | 0 | > | wsrep\_flow\_control\_recv | 5 | > > and also, that since you are writing to multiple nodes at the same time there are many deadlocks errors: > > | wsrep\_local\_bf\_aborts | 307 | > > So, I would recommend you to write on a single node and also increase gcs.fc\_limit in case you have it configured to its default value. That variable defines the number of transactions in the queue that will trigger flow control. The default number is not usually good for production. did flow control happens will cause cluster going to bad?how to optimize it? ---
**Author:** ![miguelangelnieto](https://avatars.discourse-cdn.com/v4/letter/m/b9e5f3/32.png) [@miguelangelnieto](https://forums.percona.com/u/miguelangelnieto)\ **Post date:** [September 19, 2017, 1:47am UTC](https://forums.percona.com/t/what-is-provider-paused-at-mean/5824/7 "2017-09-19T01:47:32Z")
When flow control happens the nodes pause and stop applying transactions, so the throughput is lower. To reduce it, you can configure the thresholds that trigger the flow control as explained in the documentation links I shared. If your servers are VM, you can also increase the resources. In a nutshell, flow control means “I cannot apply the changes as fast as I receive them”. Think that it is something similar to regular async replication. I recommend you again to read the previous two links, everything is explained there.