We are designing a new database (mysql likely) for a log view project , i.e. ingesting large amount of logs (from cpu chips test logs) into the database and massage the data and make it more useful, couple of extra needs is the new table columns may need to add from time to time and the logs keep comings and need to parse and load. What primary design and tools and scrips to do you suggest? THX
1 Like
Every application is unique, so you will end up writing your own scripts to ingest the data from your cpu test logs and format it into SQL for inserting into the database. For databases that are mostly writes, I would recommend you use Percona Server for MySQL and use the MyRocks storage engine which is optimized for write-intensive applications, such as storing logs.
1 Like