Cannot find metric names: error when searching for metricIDs in the current indexdb: the number of matching timeseries exceeds 30000;

Percona Monitoring and Management (PMM)
1

We are receiving the below errors from the Victoriametrics log file, and the MySQL Table Details dashboard is not working.

/home/builder/rpm/BUILD/VictoriaMetrics-pmm-6401-v1.89.1/app/vmselect/main.go:519 error in “/prometheus/api/v1/series?end=1691506272&match%5B%5D=mysql_info_schema_table_version&start=1691463072”: cannot fetch time series for “filters=[{name="mysql_info_schema_table_version"}], timeRange=[2023-08-08T02:51:12Z…2023-08-08T14:51:12Z]”: cannot find metric names: error when searching for metricIDs in the current indexdb: the number of matching timeseries exceeds 30000; either narrow down the search or increase -search.max* command-line flag values at vmselect; see VictoriaMetrics · The High Performance Open Source Time Series Database & Monitoring Solution

2

We don’t see any servers in list, Can someone please help me?

image
image1848×723 53.9 KB

3

Can someone please help me with the above issue?

4

Can someone please help me with the above issue?

5

Hello!

Have you found a solution to this problem? I also ran into her.

My configs

    vmselect:
      image:
        tag: v1.87.4-cluster
      extraArgs:
        search.maxSamplesPerQuery: "300000000000"
        search.maxSamplesPerSeries: "100000000000"
        search.maxSeries: "2000000000"
        search.maxQueryDuration: "600s"
        dedup.minScrapeInterval: "30s"
        search.maxQueueDuration: "20s"
        search.maxConcurrentRequests: "16"
        cacheDataPath: ""

vmstorage:
      image:
        tag: v1.87.4-cluster
      extraArgs:
        dedup.minScrapeInterval: "30s"
        search.maxUniqueTimeseries: "30000000"
      replicaCount: 2
      storageDataPath: "/vm-data"
      storage:
6

@apaicer You might check your database server’s table counts.

https://jira.percona.com/browse/PMM-12396

My Scenario I have used --disable-tablestats-limit=5000 same for all the servers, and recently I have added a few servers, and one of the servers has 6000+ tables, and I have used the same --disable-tablestats-limit=5000 value for that server also, After receiving these errors, the MySQL Table Details dashboard is not working.

EX: test123 server has 100 tables.

Then try the --disable-tablestats-limit=90 value and then add it to the PMM server. So, with this, we can reproduce the issue.

Temporary solution: Find the DB servers that have table counts greater than the --disable-tablestats-limit value, remove them from the PMM, and then increase the table stats value for the DB servers that have more than the current table count.

1 Like