# A lot of count of threads in mongo from operator deployment

**URL:** <https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623>\
**Category:** Percona Operator for MongoDB\
**Tags:** community, percona, mongodb\
**Created:** [October 18, 2021, 12:41pm UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623 "2021-10-18T12:41:04Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![nikita.savchenko](https://avatars.discourse-cdn.com/v4/letter/n/7feea3/32.png) [@nikita.savchenko](https://forums.percona.com/u/nikita.savchenko)\
**Post date:** [October 18, 2021, 12:41pm UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/1 "2021-10-18T12:41:04Z")

</div>

Hi. I researched problem from [this topic](https://forums.percona.com/t/a-large-number-of-threads-and-the-maxincomingconnections-option/12571/3) and found another problem - a lot of count of threads in mongod process, up to 30k. I researched this problem and found out if I drop operator deployment than count of threads decrease down to 150-200 threads. But I don’t know wny it is happen.  
I take yaml-file for deploy percona-operator deployment from [github](https://github.com/percona/percona-server-mongodb-operator/blob/main/deploy/bundle.yaml) and I don’t change it for my deploy.  
And I deploy mongodb by [cr.yaml](https://github.com/percona/percona-server-mongodb-operator/blob/main/deploy/cr.yaml) and I change in this file settings for gettings replicaset - limits, requests and sharding = false.  
Probably, I should change any settings for operator deployment, but I do not know which ones.

 ![before_delete_operator](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/0/08c2598e3db322642f3e015fc475624bc1373f07.png)  
 ![after_delete_operator](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/9/9429c7b94776cbbbee54f228f8b17de58b9694bc.png)

---

<div class="post-metadata">

**Author:** ![nikita.savchenko](https://avatars.discourse-cdn.com/v4/letter/n/7feea3/32.png) [@nikita.savchenko](https://forums.percona.com/u/nikita.savchenko)\
**Post date:** [October 20, 2021, 8:32am UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/2 "2021-10-20T08:32:39Z")

</div>

Added output of **pt-k8s-debug-collector** and screenshots of threads and pods at the moment  
[pt-k8s-debug-collector\_output.log](https://forums.percona.com/uploads/short-url/orjCCIeJe9gRdKWNM89i1jRXcrD.log) (11.6 KB)

 ![pods](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/a/a15eb2dc56c2614599ed5d6be945b913637813ac.png)  
 ![threads](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/0/0d02f51875384b29752a2cd28d1ceb7753a33917.png)  
When I got output, my cluster was no load.

---

<div class="post-metadata">

**Author:** ![nikita.savchenko](https://avatars.discourse-cdn.com/v4/letter/n/7feea3/32.png) [@nikita.savchenko](https://forums.percona.com/u/nikita.savchenko)\
**Post date:** [November 1, 2021, 11:38am UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/3 "2021-11-01T11:38:26Z")

</div>

Added archive with data from minikube cluster. I noticed same problem there.  
[Results](https://drive.google.com/drive/folders/1Hvy-S1NNHSIp0bdBK8zo7H_uijdho9pG)  
I will add result from cloud cluster later.

---

<div class="post-metadata">

**Author:** ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)\
**Post date:** [November 8, 2021, 9:27am UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/4 "2021-11-08T09:27:18Z")

</div>

Hey @nikita.savchenko , I cannot reproduce it.

1. Take 1.10 branch from github
2. Deploy bundle.yaml
3. Deploy cr.yaml with disabled sharding
4. If I login into any pod I do not see more than 90 threads and this number is not growing over time.

Is there anything else that I should do?  
Is the cluster healthy or you see some container restarts (I’m implying some health checks failures)?

P.S. tried it for 1.11 (main branch) - cannot reproduce it as well

---

<div class="post-metadata">

**Author:** ![nikita.savchenko](https://avatars.discourse-cdn.com/v4/letter/n/7feea3/32.png) [@nikita.savchenko](https://forums.percona.com/u/nikita.savchenko)\
**Post date:** [November 16, 2021, 8:37am UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/5 "2021-11-16T08:37:19Z")

</div>

Hi, @Sergey_Pronin ! Thanks for answer! It’s strange that you don’t have the same problem, but I do. Unfortunately, I’m busy now and can retry problem later.  
Tell me, please, how did you check the number of threads?  
I do the same sequence of actions and I see increasing of threads count by linux top inside containers mongod and by pmm-monitoring (increasing count of connections).

---

<div class="post-metadata">

**Author:** ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)\
**Post date:** [November 16, 2021, 12:45pm UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/6 "2021-11-16T12:45:35Z")

</div>

@nikita.savchenko as I mentioned - just regular deployment with defaults. Nothing fancy.  
Do you have pmm enabled in the CR? Anything else enabled to that I can try this out?

---

<div class="post-metadata">

**Author:** ![nikita.savchenko](https://avatars.discourse-cdn.com/v4/letter/n/7feea3/32.png) [@nikita.savchenko](https://forums.percona.com/u/nikita.savchenko)\
**Post date:** [November 22, 2021, 10:46am UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/7 "2021-11-22T10:46:03Z")

</div>

Hi, @Sergey_Pronin!  
Results of last retrying [here](https://drive.google.com/file/d/1oTPZZwFq_YYW-nzR0EudxaT-a6s4yKXV/view?usp=sharing)  
Pmm monitoring is not enabled, but I added output from db.serverStatus().connections on master-replica.

---

<div class="post-metadata">

**Author:** ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)\
**Post date:** [December 1, 2021, 1:09pm UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/8 "2021-12-01T13:09:09Z")

</div>

I was able to reproduce it.  
It happens when setFCV flag is set to true. Not sure why you set it to true and not sure why is it causing this yet.

```auto
  upgradeOptions:
    versionServiceEndpoint: https://check.percona.com
    apply: 4.4-recommended
    schedule: "0 2 * * *"
    setFCV: true

```

setFCV flag is used only for major version upgrade, so I’m not sure how it can be causing this.

Here is the ticket to get to the root of it: [[K8SPSMDB-602] setFCV flag causes a thread leak - Percona JIRA](https://jira.percona.com/browse/K8SPSMDB-602)

---

<div class="post-metadata">

**Author:** ![Viktor\_Kuzmin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/viktor_kuzmin/32/5821_2.png) [@Viktor\_Kuzmin](https://forums.percona.com/u/Viktor_Kuzmin)\
**Post date:** [February 9, 2022, 5:04am UTC](https://forums.percona.com/t/a-lot-of-count-of-threads-in-mongo-from-operator-deployment/12623/9 "2022-02-09T05:04:47Z")

</div>

Probably this one is related to one more bug: [K8SPSMDB-650: Reconcile should not be scheduled until operator has work to do by kvaster · Pull Request #880 · percona/percona-server-mongodb-operator · GitHub](https://github.com/percona/percona-server-mongodb-operator/pull/880)
