How long doest it take to insert 100MB

[perumal]

Hi,

I wrote a php program to read and parse apache log file.The size of the log file is 100MB.

From that i collected 70MB datas.

I tried to insert this data into mysql database ,it takes hardly 15 minutes to finish the job.

Is this timing is correct?

or

ist possible to reduce the time?


please help me..........

Thanks
Perumal.

[HardCoded]

You can try taking all indexes off when doing the insert. Still gonna take a long time to re-index though when you're done. That's quite a bit of data. But not as long, for sure.

You could also try piping the logs directly into the databse directly through a PHP script. Something like

CustomLog "|/path/to/logger.php" combined

An advantage to this plan is that you probably already have most or all of your regex done.

I've never tried it so I could be blowing smoke out my ass, but it looks worth looking into.