Not the answer you need?
Register and ask your own question!

Bloom filter not being used in MyRocks

VamsiVamsi Current User Role Contributor
hi Folks,
show engine RocksDB status shows these values for bloom filter related counters:

rocksdb.bloom.filter.useful COUNT : 0

rocksdb.bloom.filter.full.positive COUNT : 192

rocksdb.bloom.filter.full.true.positive COUNT : 192

rocksdb.bloom.filter.prefix.checked COUNT : 0

rocksdb.bloom.filter.prefix.useful COUNT : 0

We have a PK that has 5 columns and we always have equality predicates on at least first 2 columns (sometimes first 4 columns as well). 

rocksdb_default_cf_options used are: write_buffer_size=256m;target_file_size_base=32m;max_bytes_for_level_base=512m;level0_file_num_compaction_trigger=4;level0_slowdown_writes_trigger=10;level0_stop_writes_trigger=15;max_write_buffer_number=4;compression_per_level=kLZ4Compression;bottommost_compression=kZSTD;compression_opts=-14:1:0;prefix_extractor=capped:12;block_based_table_factory={cache_index_and_filter_blocks=1;filter_policy=bloomfilter:10:false;whole_key_filtering=0};level_compaction_dynamic_level_bytes=true;optimize_filters_for_hits=true;compaction_pri=kMinOverlappingRatio |


What other config to set to make it use the bloom filter for PK prefix lookups? 

Thanks,

Vamsi


Answers

  • yves.trudeauyves.trudeau Percona Percona Staff Role
    Hi,
       How large are the first columns?  There is a 4 bytes index_id part in the prefix so this means only 8 bytes are left for the PK.
  • VamsiVamsi Current User Role Contributor
    We have 2 cases. In one case the first column is 8 bytes only. In other case it can be longer than 8 bytes. However, regardless of whether it is exactly 8 bytes or longer, the rocksdb.bloom.filter.prefix.* counters should not be at 0, right? In either case, at least some non-zero values should be there?
  • yves.trudeauyves.trudeau Percona Percona Staff Role
    I would assume they shouldn't be 0 but while I know exactly when some InnoDB counters are updated, in MyRocks I far less sure.  Basic question.... was the bloomfilter setting added after the table was created?  If not, you should try to manually compact it.  The only production system I have access to using MyRocks uses the whole_key_filtering:

    | Rocksdb_bloom_filter_full_positive                 | 3933673684        |


  • VamsiVamsi Current User Role Contributor
    I see. Thanks for your responses, but even after a few days we did not see the counters change though lot of compactions were happening. So we think either my config settings are not complete, OR there is some bug causing prefix bloom to be not used. 


Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.