Hi Llbl people,

I've spent some time wandering in the forum and I'm still quite confused about the best way to use LLBL with web services generally, and in the case of our project specifically

Let me first explain my problematic:

I'm starting for my company a research-based project related to grid computing. Our product will be aimed at groups of collaborating entities, working on the same topic, monitoring their data in heterogeneous DB sources with similar Schemas, and wishing to share some global views on their data, without the use of a central repository. For instance our first client should be the French national pesticides observatory, a federation of laboratories monitoring pesticides all over the country, wishing to aggregate their data for national scale statistics.

Grid technology had been successfully applied to distributed computing (remember (DNA exploration or SETI at home). We'll use the same kind of technos applied to data: Where processing grid computing emulates a single virtual processing unit based on several distributed physical units, data grid computing emulates a single virtual DB Source based on several distributed heterogeneous sources. A framework for grid computing was recently implemented in .Net: it's called WSRF.Net and is downloadable at www.ws-rf.net

WSRF.Net offers an extension to .Net services based on Wse2 and allows creating state-full services, and exposing resources like processes, or DB Sources.

Ok here we are: Our plan was to take advantage of the nice object abstraction of db querying offered by LLBLGen: generate a “connector” component for each DB, exposing its schema, and remotely accessible through some kind of grid based mediation.

There should be some kind of centralised DB (decentralised in middle term, “ontology” based, ideally in the long term) storing pair correspondences between related schemas, so that starting from a common abstracted schema, the virtual query specified by a user can be transformed into a concrete query for each source.

Some central web services would deal with registering the connectors, their schema and accessible views and pair-wise correspondences (those have to be defined manually)

LLBL seems to offer the perfect abstraction for that kind of task. The actual query doesn’t have to exist until some Object based message reaches the connector, which can then build accordingly the Predicate buckets and use the DQE the normal way. And then the result to be XML Serialized and gets back on wires to the central services.

Still, I have two issues with how to properly do that.

First, I don’t think it being realistic to reference the connectors’ DBGeneric projects and rebuild the central services code each time a connector is inserted or modified, nor to import the assembly nor use any MarshallbyRef kind of mechanism to be able to use the filled collections once retrieved; the ReadXml() methods involve that the assemblies for the entities are available.

So, in order to keep our central part flexible, generic and unaware of specific assemblies we have to: - First find a nice way to retrieve and store individual schemas from the connectors, and encode/generate the resultset fields structure/prefetch paths + Predicate Buckets to be passed in the connectors - Then deal with the xml coming back, by rewriting the ReadXml(), or keeping only to custom TypeLists casted into DataTables wrapped in DataSets for instance.

However, I find it frustrating not to be able to bring the ORM framework further inside the system than the connectors, because most of it is actually completely SchemaGeneric.

Then, thinking more about it, we increasingly have the idea of replacing the standard generated code with a generic layer, holding the same inheritances and implementations from the ORM framework objects, but keeping the Persistence and structural info apart, for example in some kind of XML schema, that can be passed over web services and transformed.

I might be wrong, but it seems to me the ORM framework is so nicely packed with Interfaces and factory patterns that this should not be unconceivable, and relieve us from the build/assembly dependency.

So what do you guys think? Maybe it’s great oversee. But still, if we want to go in the direction of best flexibility, what would you advise as for the ORM support to insert in the central services / connectors.

And then specifically for you Frans, does all I’ve described pose any problem concerning licensing? To some extend, with a little exaggeration, I could summarise my last idea with: “We’d like to adopt the dynamicity of ORM and DQE, and get rid of the static side of the generator”, which is where you keep your private framework.

Hope I haven’t been too confused, on top of being much too long. Thanks anyway for reading so far, and for any ideas to help us clarify all this.

Jessynoo wrote:

Still, I have two issues with how to properly do that.

First, I don’t think it being realistic to reference the connectors’ DBGeneric projects and rebuild the central services code each time a connector is inserted or modified, nor to import the assembly nor use any MarshallbyRef kind of mechanism to be able to use the filled collections once retrieved; the ReadXml() methods involve that the assemblies for the entities are available.

So, in order to keep our central part flexible, generic and unaware of specific assemblies we have to: - First find a nice way to retrieve and store individual schemas from the connectors, and encode/generate the resultset fields structure/prefetch paths + Predicate Buckets to be passed in the connectors - Then deal with the xml coming back, by rewriting the ReadXml(), or keeping only to custom TypeLists casted into DataTables wrapped in DataSets for instance.

However, I find it frustrating not to be able to bring the ORM framework further inside the system than the connectors, because most of it is actually completely SchemaGeneric.

It of course depends on the type of clients to be used with the services. If the clients should be anything, from a pda based java-app to a C++ app to a .NET app, you have to stick with generic data-containers which are consumable by any of these platforms. So in general that comes down to very plain XML. So the usage of the entity objects ends more or less right below the surface of the service in that case, where the surface will transform the entities into the xml consumable by the clients.

If the clients are just .NET clients, I wouldn't bother with xml but go for remoting and use the entities on teh client as well, but it might be that you need to abstract it away anyway due to project restrictions.

One way of making it easier for you could be that you use dynamic lists on the server. Dynamic lists are lists in datatables which are constructed in code from entity fields of related entities and you simply use typedlist fetch logic to read the data. This way you can utilize the power of the relational model, build on top of the generated code (i.e. not doing a lot of work yourself) and have the flexibility of the XML export routines of the dataset, which have consumer routines on java as well for example.

And then specifically for you Frans, does all I’ve described pose any problem concerning licensing? To some extend, with a little exaggeration, I could summarise my last idea with: “We’d like to adopt the dynamicity of ORM and DQE, and get rid of the static side of the generator”, which is where you keep your private framework. Hope I haven’t been too confused, on top of being much too long. Thanks anyway for reading so far, and for any ideas to help us clarify all this.

You're free to do what you want with the generated code, i.e. write your own templates which replace what we ship with LLBLGen Pro and run the runtime libs with the generated code you create with these templates. As long as you don't sell it as a different O/R mapper but that's logical