Modifying victoriametrics configs

Hi. I use AWS AMI-based installation (PMM version 2.29.1) and I’m looking for some proper and reliable way of modifying underlying pmm-managed victoriametrics config. E.g. I want to modify search.maxSamplesPerQuery property. I can obviously modify /etc/supervisord.d/victoriametrics.ini directly, but I suspect it will be overwritten during update/pmm restart.

1 Like

Hello @Artem_Timchenko1 ,
you can do it by passing environment variable into our docker container with prefix VM_. Here you can see how it works VictoriaMetrics · The High Performance Open Source Time Series Database & Monitoring Solution for Victoria metrics.

Please let me know if it works for you.

1 Like

Unfortunately I use AWS AMI, from the AWS Marketplace, so no Docker there under the hood.

1 Like

Good point but all is not lost! I think you could set the variables globally in /etc/profile.d/victoriametrics.sh (new file)

with export VM_search_maxSamplesPerQuery=x as the only line in the file then restart your PMM instance so the variable will be set and VM will read it on next start. (you can do it by hand if you like by running:

. /etc/profiled.d/victoriametrics.sh
supervisorctl restart victoriametrics

but you will need ssh/shell access to your AMI instance.

1 Like

This doesn’t work, most probably due to all the processes being daemons, while /etc/profile.d seems to be used only for login sessions.

I did some tests, will left it here, maybe would be useful for someone.
The most reliable way is to add these VM_ variables to /etc/supervisord.conf, to environmant option in [supervisord] section. This will expose these vars to all supervisord-managed services, but it shouldn’t be a big issue. I haven’t found this file to be significantly modified by pmm-update ansible playbook right now.

Another way, which is kind of baggy and hacky is based on a supervisor bag, which allow to merge configurations for programs with the same name. For example, there is a supervisord file to manage VM /etc/supervisord.d/victoriametrics.ini, which has program definition [program:victoriametrics]. We can create some file like /etc/supervisord.d/victoriametrics-env.ini and define the next config there:
[program:victoriametrics]
environment = VM_search_maxSamplesPerQuery=‘XXX’
As a result both program definitions will be merged. But as I mentioned, it’s a bug in supervisor which is discussed here. Even though it hasn’t been fixed for 7 years already, there is no guarantee things will remain the same :slight_smile:

2 Likes