New search engine live!

Posts   
 
    
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39774
Joined: 17-Aug-2003
# Posted on: 15-May-2005 18:02:13   

The new search engine is now live.

It indexes in real-time. Please inform us with any issues you run into: support AT llblgen.com.

Searches should now be really fast (1 second or so).

Frans Bouma | Lead developer LLBLGen Pro
Marcus avatar
Marcus
User
Posts: 747
Joined: 23-Apr-2004
# Posted on: 07-Jun-2005 10:06:27   

Otis wrote:

The new search engine is now live. It indexes in real-time. Please inform us with any issues you run into: support AT llblgen.com. Searches should now be really fast (1 second or so).

Frans,

Maybe this is just me... Either I have a very bad memory or the way my memory works is wierd confused but I'm finding some difficulties with the new search... I tend not to remember which english words were used in the context of a post I have previously read.

I was looking for "Saving entities with a Many to Many (m:n)" relationship yesterday, but no matter what combination of search words I used, I couldn't bring up anything relevant. "m:n" is seems to be ignored. "to" is ignored and hence "Many to Many" results in just "many". "Save" is ignored etc... Using quotes to indicate an exact phrase also doesn't help...

Today I was looking for an example of Stored Procedure that you posted (which I hope you can point me to simple_smile ). I remembered that it was a fine example of how stored procedures should be written and wanted to use it as a reference. I think it was something to do with this a procedure that was used in this forum and was dated back a few years ago... The only thing I could clearly rememeber about it was that it must have contained "@@ERROR". But "@@ERROR" returns nothing.

The other thing I'm always searching for is code snippets... There are some great little snippets on the forum about how to get LLBLGen to behave in a certain way... I search for stuff is to use code keywords. This is an example of something I was trying to find again...

FolderLinkEntity folderLinkEntity = new FolderLinkEntity();
// Trick to get over problem updating PKs in LLBLGEN
folderLinkEntity.Fields["ChildFolderID"].ForcedCurrentValueWrite(req.FolderID, req.FolderID);
folderLinkEntity.Fields["ChildFolderID"].IsChanged = true;
folderLinkEntity.Fields["ParentFolderID"].ForcedCurrentValueWrite(req.NewParentFolderID, req.OldParentFolderID);
folderLinkEntity.Fields["ParentFolderID"].IsChanged = true;

I remembered it contained "ForcedCurrentValueWrite", but the search doesn't include code sections...

While I really appreciate the speed of the new search, I am finding more and more that I need to revert to using Outlook to search the RSS feeds that I pull once a day from the forum...

Again, maybe this is just me simple_smile , but if there is anything you could do to make the search a little more "inclusive" it would be appreciated.

Marcus avatar
Marcus
User
Posts: 747
Joined: 23-Apr-2004
# Posted on: 07-Jun-2005 10:15:28   

Marcus wrote:

Today I was looking for an example of Stored Procedure that you posted (which I hope you can point me to simple_smile ).

I found it!!! smile (using Outlook disappointed )

http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=3208&StartAtMessage=0#17746

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39774
Joined: 17-Aug-2003
# Posted on: 07-Jun-2005 10:36:00   

Thanks for the feedback, Marcus!

I'll try to update the search with a more flexible search option. The problem is though that with 16,000+ messages, an index is a requirement, 'LIKE' isnt suitable anymore. With an index, the problem is that if you don't want to pollute the index with a lot of data no-one's interested in, you've filter out things. For example, all words starting with a non-character (like @@ERROR) are ignored. So a way to perform a LIKE query should still be possible, but on one single forum instead.

I scanned the list of ignored words over and over again, but apparently I've missed 'save'. It shouldn't be on the ignore list. I'll try to update the list (which isn't a walk in the park, as it has to re-parse all messages, and takes a dual proc server about 8 minutes).

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39774
Joined: 17-Aug-2003
# Posted on: 26-Jun-2005 18:25:24   

I've re-indexed the search index with a much less strict noise list. I wrote an on-line re-indexer, but when I tried it on the ISP server, they apparently had build in a timeout for a page (re-indexing takes about 16 minutes) so I had to export/import all data, which took a lot of time, hence the longer period of the close down of the forum.

Also added is a shortcut for searching using an URL. I'll use that one to search on the forum using pre-baked search terms, which can help users to find things with one click of an URL. ('knowledge base' idea. )

I've now also stored the parsed XML for each message. This in the future can open up a re-indexing on other areas of the parsed XML than just plain text. For now the search engine searches just in the plain text blocks, not in the code blocks. The searchword list is now big: 23,867 words. The word-message index is now: 475,396 records. The number of threads is now: 3,158, the number of messages is now: 17,599

Frans Bouma | Lead developer LLBLGen Pro