Why does the search function suddenly stop working?

Socializing and general posts on wide-ranging topics. Remember, it's Poststructural!
Post Reply
User avatar
s1m0n
Posts: 10069
Joined: Wed Oct 06, 2004 12:17 am
antispam: No
Please enter the next number in sequence: 10
Location: The Inside Passage

Why does the search function suddenly stop working?

Post by s1m0n »

I occassionly go and reread old threads from the past. Does anyone know why, when old keyword searches (as evinced in my browser's picklist) once reliably summoned an old thread that they should suddenly stop working, when I know for a fact that the thread contains exactly the words I'm searching for?
And now there was no doubt that the trees were really moving - moving in and out through one another as if in a complicated country dance. ('And I suppose,' thought Lucy, 'when trees dance, it must be a very, very country dance indeed.')

C.S. Lewis
User avatar
s1m0n
Posts: 10069
Joined: Wed Oct 06, 2004 12:17 am
antispam: No
Please enter the next number in sequence: 10
Location: The Inside Passage

Re: Why does the search function suddenly stop working?

Post by s1m0n »

I mean, our search function really sucks. It gives up WAY too soon. If it has to return a string of the 30, or 50, best matches to your query, why doesn't it? Why does it return zero if it can't return less than 20? Or 15? The results are sometimes 2 pages, but never more than that. As far as I'm concernerned, if you write an SQL query that's too vague, you should get hundreds of responses, not zero. With hundreds, you can refine. With zero, you're skunked. You have no idea what you got wrong, or how to fix it. What jerk decided that phpBB users wanted to know nothing rather than something? It can't be to save load on the dbase. I mean, it's running the searches, and compiling the results, so there's no less draw on the search engine. All it's doing is counting the results at the last step, and if it's more than X (15 or 20, but I haven't counted) it refuses to show the results. This sux.
And now there was no doubt that the trees were really moving - moving in and out through one another as if in a complicated country dance. ('And I suppose,' thought Lucy, 'when trees dance, it must be a very, very country dance indeed.')

C.S. Lewis
User avatar
s1m0n
Posts: 10069
Joined: Wed Oct 06, 2004 12:17 am
antispam: No
Please enter the next number in sequence: 10
Location: The Inside Passage

Re: Why does the search function suddenly stop working?

Post by s1m0n »

I suppose that before it ever runs a real query, it can run a fast check on on the indexes and decide that your query isn't likely to return less than 20 results, and give up then. That'll save search engine overhead, but SEO, these days, is cheap. It now costs approximately nothing to actually run the query and to give you real results.

I wonder if that's a change that the latest version of phpBB might have made?
And now there was no doubt that the trees were really moving - moving in and out through one another as if in a complicated country dance. ('And I suppose,' thought Lucy, 'when trees dance, it must be a very, very country dance indeed.')

C.S. Lewis
User avatar
Nanohedron
Moderatorer
Posts: 38212
Joined: Wed Dec 18, 2002 6:00 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: Been a fluter, citternist, and uilleann piper; committed now to the way of the harp.

Oh, yeah: also a mod here, not a spammer. A matter of opinion, perhaps.
Location: Lefse country

Re: Why does the search function suddenly stop working?

Post by Nanohedron »

Wish I could tell you. TBH, I only use the search function if I have one - maybe two - keywords that have a high probability of producing what I'm looking for. It seems to work best that way. I've also found using Google to be fruitful, typing in something like, "chiff & fipple overornamentation" to see what pops up.

I have no way of knowing if this is outdated or simply may be called substandard. My own active presence online is pretty much mostly C&F, so I'm not all that familiar with other styles of search function. I have to say, though, that when I have used them, I was no more, and even less satisfied, with the result. So as far as I'm concerned, from what I can see, ALL search functions are more disappointing than not. I'll admit that the search function style here seems a bit... um... rustic, perhaps, but that's all I can be led to say about it.
"If you take music out of this world, you will have nothing but a ball of fire." - Tribal musician
User avatar
benhall.1
Moderator
Posts: 14797
Joined: Wed Jan 14, 2009 5:21 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: I'm a fiddler and, latterly, a fluter. I love the flute. I wish I'd always played it. I love the whistle as well. I'm blessed in having really lovely instruments for all of my musical interests.
Location: Unimportant island off the great mainland of Europe

Re: Why does the search function suddenly stop working?

Post by benhall.1 »

The search function here really is poor. There's not a lot we can do about it. It's partly because of the size of the database. The other issue is that, ideally, it should be re-indexed regularly. But last time I did it, it took me several days, during which time it stopped whenever my machine stopped. That is not supposed to happen, but happen it did. There's also a chance that it may fail - since you have to delete the previous index before you start, I'm not sure where that would leave us. It's stressful so I don't do it often.
User avatar
s1m0n
Posts: 10069
Joined: Wed Oct 06, 2004 12:17 am
antispam: No
Please enter the next number in sequence: 10
Location: The Inside Passage

Re: Why does the search function suddenly stop working?

Post by s1m0n »

Ah. Thanks for the responses, and thank you for your effort, Ben.
And now there was no doubt that the trees were really moving - moving in and out through one another as if in a complicated country dance. ('And I suppose,' thought Lucy, 'when trees dance, it must be a very, very country dance indeed.')

C.S. Lewis
User avatar
s1m0n
Posts: 10069
Joined: Wed Oct 06, 2004 12:17 am
antispam: No
Please enter the next number in sequence: 10
Location: The Inside Passage

Re: Why does the search function suddenly stop working?

Post by s1m0n »

I suppose if it's an indexing issue then a phpBB upgrade isn't likely to help.
And now there was no doubt that the trees were really moving - moving in and out through one another as if in a complicated country dance. ('And I suppose,' thought Lucy, 'when trees dance, it must be a very, very country dance indeed.')

C.S. Lewis
User avatar
benhall.1
Moderator
Posts: 14797
Joined: Wed Jan 14, 2009 5:21 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: I'm a fiddler and, latterly, a fluter. I love the flute. I wish I'd always played it. I love the whistle as well. I'm blessed in having really lovely instruments for all of my musical interests.
Location: Unimportant island off the great mainland of Europe

Re: Why does the search function suddenly stop working?

Post by benhall.1 »

s1m0n wrote:I suppose if it's an indexing issue then a phpBB upgrade isn't likely to help.
Actually, no. In fact, after the last upgrade, I found that I had to re-index in any case. It really stresses me out.

However, I will have a look at this over the weekend. Can I just flag up - and I'll do this more publicly if/when I get round to doing this exercise again - that I will need everyone - absolutely everyone - NOT to use the search function once I start re-indexing. It doesn't do any harm, but I found by actual test (getting Dale to search things while I was doing the re-indexing) that anyone at all searching while the process was going on sowed the already glacial progress almost to a standstill.

If I can get to the end of the exercise, you will find a significant improvement, which will last months before it starts to deteriorate again, so you may find that it's a worthwhile exercise.

Not everyone uses the search function. But really is useful to be able to do it, and I feel that it's worthwhile to make it available whenever possible. Thanks for flagging up that I need to look at it again, s1m0n.
Tunborough
Posts: 1419
Joined: Sun Dec 05, 2010 2:59 pm
antispam: No
Please enter the next number in sequence: 10
Location: Southwestern Ontario

Re: Why does the search function suddenly stop working?

Post by Tunborough »

Do we know where all the time is being spent? Sure it's a lot of data, but when you talk about it taking days, and a glacial pace, it sounds like there's some bottle-neck we need to look at. If re-indexing stops when your computer stops, I wonder what communication is happening between your computer and the server in the process. Have you noticed your internet usage spiking when you re-index? AaronFW has taken a look at the phpBB code. Perhaps he would have something to suggest.
User avatar
s1m0n
Posts: 10069
Joined: Wed Oct 06, 2004 12:17 am
antispam: No
Please enter the next number in sequence: 10
Location: The Inside Passage

Re: Why does the search function suddenly stop working?

Post by s1m0n »

Is the dbase behind this website MySQL? Or something similar? I've lost all the php chops I once had, but I expect my SQL instincts* are intact. If you need an assistant, or even a delegate, in this process, I might be able to assist.

*It's stabler
And now there was no doubt that the trees were really moving - moving in and out through one another as if in a complicated country dance. ('And I suppose,' thought Lucy, 'when trees dance, it must be a very, very country dance indeed.')

C.S. Lewis
User avatar
benhall.1
Moderator
Posts: 14797
Joined: Wed Jan 14, 2009 5:21 pm
antispam: No
Please enter the next number in sequence: 8
Tell us something.: I'm a fiddler and, latterly, a fluter. I love the flute. I wish I'd always played it. I love the whistle as well. I'm blessed in having really lovely instruments for all of my musical interests.
Location: Unimportant island off the great mainland of Europe

Re: Why does the search function suddenly stop working?

Post by benhall.1 »

s1m0n wrote:Is the dbase behind this website MySQL? Or something similar? I've lost all the php chops I once had, but I expect my SQL instincts* are intact. If you need an assistant, or even a delegate, in this process, I might be able to assist.

*It's stabler
The database is MySQL. There are two options for indexing, provided by the phpBB platform: "Fulltext MySQL" and "Fulltext Native (Active)". I have looked this up in the past, and have also tried both versions. For some reason, the "Fulltext MySQL" simply doesn't work on our site. Answers that I have found on phpBB forums in past searches have suggested that many other people have found the same, so it is recommended that the other version is used. One of the big problems with the "Native" version is that it needs to be updated, preferably fairly often. But that just isn't practical, given how long it takes to do.

I'm going to start soon, in any case ...
Post Reply