PXC - wsrep_local_recv_queue_avg

[dannyg]

Hi,

i’ve started playing around with PXC and I’m testing it in an environment very similar to my production environment, we have 3 nodes in cluster and I’m trying to closely monitor the cluster status so i can understand when things go wrong.

my issue is with :

wsrep_local_recv_queue | 0
wsrep_local_recv_queue_avg | 0.205483

wsrep_local_recv_queue_avg, this value ranges anywhere between 0.205483 - 0.506762 on my nodes, is this good ? is this bad ? i understand that this means that the avg queue is bigger then what the node could apply, but whats a good avg ? is 0.0 even realistic ? currently i have 2 replication threads configured and everything is within a local LAN, machines have 64GB of RAM and SSD drives.

not sure what more can i do to lower the average…

Thanks,
Danny.

[mirfan]

wsrep_local_send_queue_avg should be as close to zero. If the value of wsrep_local_recv_queue_avg is higher then it leads towards slow node in the network and the node is overloaded. I would suggest to check this for more info http://www.codership.com/wiki/doku.php?id=monitoring&s[]=wsrep&s[]=flow&s[]=control&s[]=sent and http://www.percona.com/doc/percona-xtradb-cluster/5.5/wsrep-status-index.html

[dannyg]

Hi Mirfan,

I’m talking about local recv queue not send queue, what’s concerning here is that i constantly see 0 on the recv queue but the avg is always bigger then 0:

| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_avg | 0.002475 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_avg | 0.321252 |
| wsrep_local_cached_downto | 7788205 |
| wsrep_flow_control_paused_ns | 22231467821 |
| wsrep_flow_control_paused | 0.010708 |
| wsrep_flow_control_sent | 31 |
| wsrep_flow_control_recv | 37

this is of course causing flow control but i’m never able to see my wsrep_local_recv_queue go above 0, so how the the avg be bigger ?

–Danny.

[mirfan]

sorry, it was typo i mean wsrep_local_recv_queue_avg instead wsrep_local_send_queue_avg
when looking into wsrep_local_recv_queue_avg is non-zero means that there a queue often, may be not large one and wsrep_local_recv_queue is absolute value during show global status while _avg is the average queue in between show global status runs.