Integration with Lucene

Posts   
 
    
luisfco
User
Posts: 1
Joined: 01-Jun-2009
# Posted on: 01-Jun-2009 19:06:44   

HI

We are very interested in using LLBLGen in some proyects but we have some questions regardin the use of Lucene with LLBLGen. We dont want/cant use SQLServer FTS, because we also want to use Compact edition, and other databases(Firebird, Access).

  • Is there a way to add Lucene serch functinality to LLBLGen?
  • If it is not available in the current LLBLGen version are there any plans to include it?

In case it is not included and wont be added. What could e the best option to integrate it?: - With a Sotored Procedure (using NET code inside the SP, not TSQL), that performs the search and returns a recordset/table with the results, this recordset/table can used to JOIN with some tables - The other option would be to integrate Lucene directly in the DB (instead of using file structure to store the index, the index will be stored in DB tables), then maybe I can perform the search in the index and again use JOIN to get the row from other tables. How easy can this (using a table/recordset to join with other tables and return the results to LLBLGen) be done? - Any other possible options?

Saludos, Luis

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39797
Joined: 17-Aug-2003
# Posted on: 02-Jun-2009 10:31:09   

Integration with Lucene isn't planned. The main problem is that we don't have any experience with Lucene, so for example how is lucene integrated with sqlserver or other databases in such a way that when you update a row, lucene is updating its store as well? To me, that's external, so when you update a row, you also have to update lucene.

To filter a database on results coming from an external source, you either have to be able to use that source in a SQL query (IMHO only possible with external stored procs in the case of lucene) or you've to pre-filter first using lucene, i.e.: first ask lucene which IDs match a filter for example, then use those IDs to obtain a range of entities. that last part is doable with llblgen pro (it supports range filtering based on a set of values), though you have to do the lucene updates yourselves as well as the filtering on lucene. So for example when an entity is saved (you can intercept that in llblgen pro) you can for example call a Lucene api method to store that info: ID + searchable field values, which you then can filter on later on. That's the only way I see how this will work.

Frans Bouma | Lead developer LLBLGen Pro
gcause
User
Posts: 6
Joined: 06-May-2009
# Posted on: 07-Jul-2009 03:01:50   

Otis wrote:

Integration with Lucene isn't planned. The main problem is that we don't have any experience with Lucene, so for example how is lucene integrated with sqlserver or other databases in such a way that when you update a row, lucene is updating its store as well? To me, that's external, so when you update a row, you also have to update lucene.

To filter a database on results coming from an external source, you either have to be able to use that source in a SQL query (IMHO only possible with external stored procs in the case of lucene) or you've to pre-filter first using lucene, i.e.: first ask lucene which IDs match a filter for example, then use those IDs to obtain a range of entities. that last part is doable with llblgen pro (it supports range filtering based on a set of values), though you have to do the lucene updates yourselves as well as the filtering on lucene. So for example when an entity is saved (you can intercept that in llblgen pro) you can for example call a Lucene api method to store that info: ID + searchable field values, which you then can filter on later on. That's the only way I see how this will work.

We are definitely interested in getting Lucene working with LLBGen Pro and we are going to have to devise some special mechanism unless you have added the indexing methods you were talking about last year in the forum as potential v3 features.

NHibernate already has such integration.

http://blogs.intesoft.net/post/2008/03/NHibernateSearch-using-Lucene-NET-Full-Text-Index-Part1.aspx

http://ayende.com/Blog/archive/2007/03/18/Googlize-your-entities-NHibernate--Lucene.NET-Integration.aspx

http://morten.lyhr.dk/2008/03/nhibernates-search-feature.html

We are using an ORACLE10g database so we can use SQL Server's built in FTS.

Is there anything in LLBGen Pro that will help us implement full integration with Lucene easily?