I am in a process of using a 15 Terabyte of database on MYSQL 5.1
This is not a true OLTP nor a datawarehosuing stuff. We need to query say 20 times an hours only.
The way i am planning to design this is
- We’ll have 10 nodes and each node will have 5-6 databases
- Engine would by MyISAM and we are planning to use Partitioning and it will be partitioned on a field A which is main searchable field.
- This is going to be a semi normalized database with 10 tables.
- Data load would be done by load data infile(We have a complete .txt file 3 GB in size, we are writteing a C parser which will parse the file and create 10 files(one for each table)
What consideration should we need to take to carry out such huge database.
- Search is the major critaria in this application
** Data loading:- is this thr right approach (Data load would be a daily affair)
*** What change are important as far as My.ini is concern(For cache and memeory ) point of view. All nodes have a shared SAN drive, and memeory on each node is minimum 2 GB.
Any help owuld really make my life easier.