abudabit
02-10-2006, 07:20 PM
Hi all,
I am very very new to using the full text searches with sql (mysql, Isam). I have a behind the scenes script I use to clean up redundant items by running full text searches using part of one entry as the needle. Anyways, it runs fine except everyone in a while my server hangs up. This will run about 20k searches a night in a table of 100k rows (although it crashed when I had only 15k rows so that probably isn't the issue).
So I had a few theories about what is crashing it, tell me if one of these seems possible:
1. Since it uses a random part of the entry, special characters that are screwing up the boolean are getting in there and causing some sort of endless cycle (is there a way to disable boolean?). I am fixing this now by using regex to select only words with letters, but I won't find out if it works because the crashes are so random.
2. Since this cycles through a lot of times (I use javascript to wait 30 minutes and run 20 iterations so it doesn't try them all at once) maybe there is a build up of memory leakage.
3. Full text searches just use too many resources.
Is there any way to narrow down what it might be? Also since my website is database heavy, should I tweak my system resource allocation or should I just keep the default settings? Has anyone heard of common problems with full text searches which cause freezes?
I am very very new to using the full text searches with sql (mysql, Isam). I have a behind the scenes script I use to clean up redundant items by running full text searches using part of one entry as the needle. Anyways, it runs fine except everyone in a while my server hangs up. This will run about 20k searches a night in a table of 100k rows (although it crashed when I had only 15k rows so that probably isn't the issue).
So I had a few theories about what is crashing it, tell me if one of these seems possible:
1. Since it uses a random part of the entry, special characters that are screwing up the boolean are getting in there and causing some sort of endless cycle (is there a way to disable boolean?). I am fixing this now by using regex to select only words with letters, but I won't find out if it works because the crashes are so random.
2. Since this cycles through a lot of times (I use javascript to wait 30 minutes and run 20 iterations so it doesn't try them all at once) maybe there is a build up of memory leakage.
3. Full text searches just use too many resources.
Is there any way to narrow down what it might be? Also since my website is database heavy, should I tweak my system resource allocation or should I just keep the default settings? Has anyone heard of common problems with full text searches which cause freezes?