Hello,
I’m looking for some suggestions. I have a 50G database that grows on average about 1-2gigs of information a day.
I’m looking for some suggestion on how to improve performance on some new systems i have. I have run some preformance tests with mysql bench to get an idea of how they are performing, I was wondering if anyone had any additional suggestions on what i can tune in the configuration of mysql.
Prsently were running MySQL 4.0.24 with CentOS 4.4 X86_64.
The system i ran this test on has the following:
[root@cf-db7 sql-bench]# cat /proc/cpuinfo | grep processor
processor : 0
processor : 1
processor : 2
processor : 3
CPU info:
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU 5130 @ 2.00GHz
stepping : 6
cpu MHz : 1995.004
cache size : 4096 KB
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl tm2 cx16 xtpr
bogomips : 3990.05
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
[root@cf-db7 sql-bench]# cat /proc/meminfo
MemTotal: 16410608 kB
MemFree: 12325424 kB
Buffers: 166224 kB
Cached: 3346656 kB
[root@cf-db7 sql-bench]#
Here is my my.cnf file as well:
[root@cf-db7 sql-bench]# cat /etc/my.cnf
Example MySQL config file for very large systems.
This is for a large system with memory of 1G-2G where the system runs mainly
MySQL.
You can copy this file to
/etc/my.cnf to set global options,
mysql-data-dir/my.cnf to set server-specific options (in this
installation this directory is /usr/local/mysql/data) or
~/.my.cnf to set user-specific options.
In this file, you can use all long options that a program supports.
If you want to know which options a program supports, run the program
with the “–help” option.
The following options will be passed to all MySQL clients
[client]
#password = your_password
port = 3306
socket = /tmp/mysql.sock
Here follows entries for some specific programs
The MySQL server
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-locking
key_buffer = 4096M
max_allowed_packet = 32M
table_cache = 1024
sort_buffer_size = 4M
read_buffer_size = 4M
myisam_sort_buffer_size = 128M
thread_cache = 16
query_cache_size = 72M
Try number of CPU’s*2 for thread_concurrency
thread_concurrency = 8
max_connections=2000
wait_timeout=300
interactive_timeout=1200
long_query_time=3
basedir=/usr/local/mysql
Don’t listen on a TCP/IP port at all. This can be a security enhancement,
if all processes that need to connect to mysqld run on the same host.
All interaction with mysqld must be made via Unix sockets or named pipes.
Note that using this option without enabling named pipes on Windows
(via the “enable-named-pipe” option) will render mysqld useless!
#skip-networking
Replication Master Server (default)
binary logging is required for replication
#log-bin
required unique id between 1 and 2^32 - 1
defaults to 1 if master-host is not set
but will not function as a master if omitted
server-id = 1
Replication Slave (comment out master section to use this)
To configure this host as a replication slave, you can choose between
two methods :
1) Use the CHANGE MASTER TO command (fully described in our manual) -
the syntax is:
CHANGE MASTER TO MASTER_HOST=, MASTER_PORT=,
MASTER_USER=, MASTER_PASSWORD= ;
where you replace , , by quoted strings and
by the master’s port number (3306 by default).
Example:
CHANGE MASTER TO MASTER_HOST=‘125.564.12.1’, MASTER_PORT=3306,
MASTER_USER=‘joe’, MASTER_PASSWORD=‘secret’;
OR
2) Set the variables below. However, in case you choose this method, then
start replication for the first time (even unsuccessfully, for example
if you mistyped the password in master-password and the slave fails to
connect), the slave will create a master.info file, and any later
change in this file to the variables’ values below will be ignored and
overridden by the content of the master.info file, unless you shutdown
the slave server, delete master.info and restart the slaver server.
For that reason, you may want to leave the lines below untouched
(commented) and instead use CHANGE MASTER TO (see above)
required unique id between 2 and 2^32 - 1
(and different from the master)
defaults to 2 if master-host is set
but will not function as a slave if omitted
#server-id = 2
The replication master for this slave - required
#master-host =
The username the slave will use for authentication when connecting
to the master - required
#master-user =
The password the slave will authenticate with when connecting to
the master - required
#master-password =
The port the master is listening on.
optional - defaults to 3306
#master-port =
binary logging - not required for slaves, but recommended
#log-bin
Point the following paths to different dedicated disks
#tmpdir = /tmp/
#log-update = /path-to-dedicated-directory/hostname
Uncomment the following if you are using BDB tables
#bdb_cache_size = 384M
#bdb_max_lock = 100000
Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /usr/local/mysql/data/
#innodb_data_file_path = ibdata1:2000M;ibdata2:10M:autoextend
#innodb_log_group_home_dir = /usr/local/mysql/data/
#innodb_log_arch_dir = /usr/local/mysql/data/
You can set …_buffer_pool_size up to 50 - 80 %
of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 384M
#innodb_additional_mem_pool_size = 20M
Set …_log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 100M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50
[mysqldump]
quick
max_allowed_packet = 16M
[mysql]
no-auto-rehash
Remove the next comment character if you are not familiar with SQL
#safe-updates
[isamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
[myisamchk]
key_buffer = 256M
sort_buffer_size = 256M
read_buffer = 2M
write_buffer = 2M
[mysqlhotcopy]
interactive-timeout
[root@cf-db7 sql-bench]#
Finally here is the mysql bench output:
Benchmark DBD suite: 2.15
Date of test: 2007-04-13 17:15:49
Running tests on: Linux 2.6.9-42.0.10.ELsmp x86_64
Arguments: --socket=/tmp/mysql.sock
Comments:
Limits from:
Server version: MySQL 4.0.24 standard
Optimization: None
Hardware:
alter-table: Total time: 13 wallclock secs ( 0.02 usr 0.02 sys + 0.00 cusr 0.00 csys = 0.04 CPU)
ATIS: Total time: 5 wallclock secs ( 3.31 usr 0.88 sys + 0.00 cusr 0.00 csys = 4.19 CPU)
big-tables: Total time: 5 wallclock secs ( 1.78 usr 1.60 sys + 0.00 cusr 0.00 csys = 3.38 CPU)
connect: Total time: 51 wallclock secs (12.94 usr 13.27 sys + 0.00 cusr 0.00 csys = 26.21 CPU)
create: Total time: 231 wallclock secs ( 1.13 usr 0.53 sys + 0.00 cusr 0.00 csys = 1.66 CPU)
insert: Total time: 438 wallclock secs (165.75 usr 57.61 sys + 0.00 cusr 0.00 csys = 223.36 CPU)
select: Total time: 53 wallclock secs (16.92 usr 4.79 sys + 0.00 cusr 0.00 csys = 21.71 CPU)
transactions: Test skipped because the database doesn’t support transactions
wisconsin: Total time: 4 wallclock secs ( 1.19 usr 0.56 sys + 0.00 cusr 0.00 csys = 1.75 CPU)
All 9 test executed successfully
Totals per operation:
Operation seconds usr sys cpu tests
alter_table_add 6.00 0.00 0.01 0.01 100
alter_table_drop 5.00 0.01 0.00 0.01 91
connect 3.00 2.16 0.44 2.60 10000
connect+select_1_row 4.00 2.22 0.69 2.91 10000
connect+select_simple 4.00 2.24 0.55 2.79 10000
count 6.00 0.02 0.00 0.02 100
count_distinct 0.00 0.08 0.02 0.10 1000
count_distinct_2 0.00 0.08 0.02 0.10 1000
count_distinct_big 5.00 3.01 0.80 3.81 120
count_distinct_group 0.00 0.38 0.10 0.48 1000
count_distinct_group_on_key 0.00 0.07 0.04 0.11 1000
count_distinct_group_on_key_parts 1.00 0.38 0.09 0.47 1000
count_distinct_key_prefix 0.00 0.07 0.01 0.08 1000
count_group_on_key_parts 0.00 0.35 0.11 0.46 1000
count_on_key 15.00 3.17 0.91 4.08 50100
create+drop 64.00 0.32 0.09 0.41 10000
create_MANY_tables 99.00 0.30 0.07 0.37 10000
create_index 1.00 0.00 0.00 0.00 8
create_key+drop 65.00 0.36 0.10 0.46 10000
create_table 1.00 0.00 0.00 0.00 31
delete_all_many_keys 12.00 0.00 0.01 0.01 1
delete_big 0.00 0.00 0.00 0.00 1
delete_big_many_keys 12.00 0.00 0.01 0.01 128
delete_key 1.00 0.07 0.11 0.18 10000
delete_range 2.00 0.00 0.00 0.00 12
drop_index 1.00 0.00 0.00 0.00 8
drop_table 0.00 0.00 0.00 0.00 28
drop_table_when_MANY_tables 2.00 0.06 0.14 0.20 10000
insert 20.00 2.54 4.61 7.15 350768
insert_duplicates 5.00 1.00 1.44 2.44 100000
insert_key 21.00 1.66 1.05 2.71 100000
insert_many_fields 1.00 0.08 0.02 0.10 2000
insert_select_1_key 1.00 0.00 0.00 0.00 1
insert_select_2_keys 1.00 0.00 0.00 0.00 1
min_max 2.00 0.01 0.00 0.01 60
min_max_on_key 8.00 5.39 1.69 7.08 85000
multiple_value_insert 1.00 0.14 0.00 0.14 100000
order_by_big 15.00 8.78 2.49 11.27 10
order_by_big_key 13.00 9.07 2.52 11.59 10
order_by_big_key2 12.00 8.76 2.50 11.26 10
order_by_big_key_desc 12.00 9.14 2.49 11.63 10
order_by_big_key_diff 13.00 8.81 2.48 11.29 10
order_by_big_key_prefix 11.00 8.78 2.49 11.27 10
order_by_key2_diff 1.00 0.80 0.19 0.99 500
order_by_key_prefix 1.00 0.41 0.10 0.51 500
order_by_range 1.00 0.42 0.10 0.52 500
outer_join 1.00 0.00 0.00 0.00 10
outer_join_found 1.00 0.00 0.00 0.00 10
outer_join_not_found 0.00 0.00 0.00 0.00 500
outer_join_on_key 0.00 0.00 0.00 0.00 10
select_1_row 7.00 1.00 1.83 2.83 100000
select_1_row_cache 4.00 0.78 2.17 2.95 100000
select_2_rows 8.00 1.06 2.19 3.25 100000
select_big 11.00 8.82 2.48 11.30 80
select_big_str 4.00 1.35 0.62 1.97 10000
select_cache 1.00 0.63 0.22 0.85 10000
select_cache2 29.00 0.75 0.11 0.86 10000
select_column+column 8.00 0.92 1.69 2.61 100000
select_diff_key 33.00 0.05 0.01 0.06 500
select_distinct 1.00 0.63 0.16 0.79 800
select_group 1.00 0.43 0.10 0.53 2911
select_group_when_MANY_tables 1.00 0.09 0.13 0.22 10000
select_join 1.00 0.20 0.05 0.25 100
select_key 28.00 14.69 4.07 18.76 200000
select_key2 29.00 14.21 3.99 18.20 200000
select_key2_return_key 29.00 13.54 4.01 17.55 200000
select_key2_return_prim 29.00 14.19 3.95 18.14 200000
select_key_prefix 29.00 14.16 4.09 18.25 200000
select_key_prefix_join 2.00 1.69 0.44 2.13 100
select_key_return_key 29.00 13.86 4.11 17.97 200000
select_many_fields 4.00 1.70 1.58 3.28 2000
select_range 5.00 3.39 0.88 4.27 410
select_range_key2 2.00 1.54 0.44 1.98 25010
select_range_prefix 2.00 1.53 0.40 1.93 25010
select_simple 4.00 0.49 1.38 1.87 100000
select_simple_cache 4.00 0.72 1.71 2.43 100000
select_simple_join 0.00 0.22 0.06 0.28 500
update_big 7.00 0.00 0.00 0.00 10
update_of_key 4.00 0.49 0.73 1.22 50000
update_of_key_big 3.00 0.01 0.00 0.01 501
update_of_primary_key_many_keys 8.00 0.01 0.00 0.01 256
update_with_key 19.00 2.48 4.49 6.97 300000
update_with_key_prefix 7.00 1.52 1.55 3.07 100000
wisc_benchmark 2.00 1.00 0.18 1.18 114
TOTALS 805.00 199.29 78.31 277.60 3225950
[root@cf-db7 sql-bench]#