Multiple DB Support

Posts   
 
    
KS
User
Posts: 55
Joined: 08-Aug-2006
# Posted on: 20-Jul-2009 17:34:25   

Hi Frans/LLBL Guys,

After successfully completing my last project using LLBL at my last company, recently I moved onto a newer company. I have taken up Architect role in a big IT project in a new company. As any project start includes evaluation of technologies and tools, we are also in same process. Part of it is to identify whether to ORM and if yes LLBL or NHibernate. I was part of similar evaluation at last job so not really concerned about that. However certain scenarios/requirements are different and hence I am posting questions , any suggestionare welcome(keep it in mind, it is high volume enterprise web app with millions of hit a day)

  1. <LLBL Quote> Support for multiple catalogs in a single project (SqlServer) You can add more than one catalog to an LLBLGen Pro project, and create relations between the entities of the different catalogs. The generated code then lets you work with the catalogs as if they form one catalog. </LLBL Quote>

Does that mean Multiple catalogs on 1 DB server only ? Or there's also support for Multiple DB Server (like creating relation between entities of two different entities on two different catalog which are on different servers ) ?

  1. Can Entity be exposed as part of interface of Webservice(completely xml serializable) ?

  2. This is more of an general architecture question. Our web app is all Queryable and No CRUD(or minimal CRUD). Can that be considered as an Indicator to NOT TO use an ORM as ORM's main strength is to save time and effort on trivial CRUD. I can imagine lot of queries in our App might be complicated joins including multiple tables.

  3. Again, this is more of generic architecture question. We will be implementing MVP(no MVC) architecture. Any suggestion for comsuming LLBL Entity all the way in View(Supervisor) or View should completely be abstracted(Passive) ? Any thoughts/suggestions are welcome.

  4. Last and most important when v3 is coming out and will there be support for v2 to v3 project conversion ? Also does porting to newer version will include substantial effort ?

Thanks KS

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39749
Joined: 17-Aug-2003
# Posted on: 20-Jul-2009 18:39:45   

KS wrote:

Hi Frans/LLBL Guys,

After successfully completing my last project using LLBL at my last company, recently I moved onto a newer company. I have taken up Architect role in a big IT project in a new company.

Congratulations with the promotion! simple_smile

As any project start includes evaluation of technologies and tools, we are also in same process. Part of it is to identify whether to ORM and if yes LLBL or NHibernate. I was part of similar evaluation at last job so not really concerned about that. However certain scenarios/requirements are different and hence I am posting questions , any suggestionare welcome(keep it in mind, it is high volume enterprise web app with millions of hit a day)

before I go further: the more hits a website gets, the less it's a concern for the data-access technology: the more hits, the more you want to cache higher up in the webapplication, as even processing data returned by the data-access technology (be it from its own cache or from the db) is too slow compared to caching processed data (i.e. rendered webcontent)

  1. <LLBL Quote> Support for multiple catalogs in a single project (SqlServer) You can add more than one catalog to an LLBLGen Pro project, and create relations between the entities of the different catalogs. The generated code then lets you work with the catalogs as if they form one catalog. </LLBL Quote>

Does that mean Multiple catalogs on 1 DB server only ? Or there's also support for Multiple DB Server (like creating relation between entities of two different entities on two different catalog which are on different servers ) ?

Single server, as in: you have to be able to reach both using the same open connection. NHibernate doesn't support multiple catalogs, although it does support multiple schemas. Multiple servers isn't doable as you then can't join data. Linked servers is also not supported as names are different with named servers (and meta-data retrieval is not doable), but I think it's also not really an option in high performance scenarios.

  1. Can Entity be exposed as part of interface of Webservice(completely xml serializable) ?

yes, WCF, webservice, remoting you name it. Full change tracking included. See documentation of Adapter -> distributed scenarios and our examples.

  1. This is more of an general architecture question. Our web app is all Queryable and No CRUD(or minimal CRUD). Can that be considered as an Indicator to NOT TO use an ORM as ORM's main strength is to save time and effort on trivial CRUD. I can imagine lot of queries in our App might be complicated joins including multiple tables.

What do you mean with Queryable? Queryable<T> ? Or do you mean: the user can define wahtever query he wants?

  1. Again, this is more of generic architecture question. We will be implementing MVP(no MVC) architecture. Any suggestion for comsuming LLBL Entity all the way in View(Supervisor) or View should completely be abstracted(Passive) ? Any thoughts/suggestions are welcome.

What I heard (no expert in this) is that when you do MVP/MVC, it's almost a given that you create a new set of classes for the presentation only, which could be DTO's for example, which you generate with DTO templates (there are several available on the forum, search for "DTO templates") which make creating themselves from entities back/forth easier.

  1. Last and most important when v3 is coming out and will there be support for v2 to v3 project conversion ? Also does porting to newer version will include substantial effort ?

v3 will go beta before the end of the year as it looks now. Yes there will be conversion of v2 to v3 projects as in: the v2 .lgp file is converted to the v3 xml based project file using a template. V3 has a fully new designer and some new features for the runtime (like DDD value types and support for mixing inheritance types in 1 hierarchy), and the runtime /templates will be adjusted a bit to support these features (and move code from templates to runtime) but it won't be that massive as with v2.5: v3.0 is mainly about the designer, managing the project, creating new entities etc..

Frans Bouma | Lead developer LLBLGen Pro
KS
User
Posts: 55
Joined: 08-Aug-2006
# Posted on: 20-Jul-2009 20:57:15   

Otis wrote:

KS wrote:

Hi Frans/LLBL Guys,

After successfully completing my last project using LLBL at my last company, recently I moved onto a newer company. I have taken up Architect role in a big IT project in a new company.

Congratulations with the promotion! simple_smile

Thanks

As any project start includes evaluation of technologies and tools, we are also in same process. Part of it is to identify whether to ORM and if yes LLBL or NHibernate. I was part of similar evaluation at last job so not really concerned about that. However certain scenarios/requirements are different and hence I am posting questions , any suggestionare welcome(keep it in mind, it is high volume enterprise web app with millions of hit a day)

before I go further: the more hits a website gets, the less it's a concern for the data-access technology: the more hits, the more you want to cache higher up in the webapplication, as even processing data returned by the data-access technology (be it from its own cache or from the db) is too slow compared to caching processed data (i.e. rendered webcontent)

  1. <LLBL Quote> Support for multiple catalogs in a single project (SqlServer) You can add more than one catalog to an LLBLGen Pro project, and create relations between the entities of the different catalogs. The generated code then lets you work with the catalogs as if they form one catalog. </LLBL Quote>

Does that mean Multiple catalogs on 1 DB server only ? Or there's also support for Multiple DB Server (like creating relation between entities of two different entities on two different catalog which are on different servers ) ?

Single server, as in: you have to be able to reach both using the same open connection. NHibernate doesn't support multiple catalogs, although it does support multiple schemas. Multiple servers isn't doable as you then can't join data. Linked servers is also not supported as names are different with named servers (and meta-data retrieval is not doable), but I think it's also not really an option in high performance scenarios.

Fair enough

  1. Can Entity be exposed as part of interface of Webservice(completely xml serializable) ?

yes, WCF, webservice, remoting you name it. Full change tracking included. See documentation of Adapter -> distributed scenarios and our examples.

I thought so.

  1. This is more of an general architecture question. Our web app is all Queryable and No CRUD(or minimal CRUD). Can that be considered as an Indicator to NOT TO use an ORM as ORM's main strength is to save time and effort on trivial CRUD. I can imagine lot of queries in our App might be complicated joins including multiple tables.

What do you mean with Queryable? Queryable<T> ? Or do you mean: the user can define wahtever query he wants?

By Queryable I mean there no OLTP (no CRUD) involded, think of it as read-only/searchable/information web app(there will be a better tech term for it simple_smile ). No Users wouldn't define there own queries, however they would be different View of with Variety of searchable criterias. Keeping that in mind, do you still think ORM is still a good choice ?

  1. Again, this is more of generic architecture question. We will be implementing MVP(no MVC) architecture. Any suggestion for comsuming LLBL Entity all the way in View(Supervisor) or View should completely be abstracted(Passive) ? Any thoughts/suggestions are welcome.

What I heard (no expert in this) is that when you do MVP/MVC, it's almost a given that you create a new set of classes for the presentation only, which could be DTO's for example, which you generate with DTO templates (there are several available on the forum, search for "DTO templates") which make creating themselves from entities back/forth easier.

I thought of that, in fact you trackback using my userid on this forum, you will find I modified and posted back DTO template(as we used it in my last project). However once you do that you loose out of binding, in memory filter, paging etc support. Also by adding 1 more layer of transformation you get some more performance hit(even though it's minimal). That doesn't mean I am disagreeing to you, I am just trying to think out loud and weigh each option. Going with you suggestion, what we get is key objective of MVP/MVC => Clear Separation of Concerns.

  1. Last and most important when v3 is coming out and will there be support for v2 to v3 project conversion ? Also does porting to newer version will include substantial effort ?

v3 will go beta before the end of the year as it looks now. Yes there will be conversion of v2 to v3 projects as in: the v2 .lgp file is converted to the v3 xml based project file using a template. V3 has a fully new designer and some new features for the runtime (like DDD value types and support for mixing inheritance types in 1 hierarchy), and the runtime /templates will be adjusted a bit to support these features (and move code from templates to runtime) but it won't be that massive as with v2.5: v3.0 is mainly about the designer, managing the project, creating new entities etc..

Looking forward to v3, finally lgp is making way for xml mapping, I am sure it would have been tough for you simple_smile

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39749
Joined: 17-Aug-2003
# Posted on: 21-Jul-2009 11:14:23   

KS wrote:

  1. This is more of an general architecture question. Our web app is all Queryable and No CRUD(or minimal CRUD). Can that be considered as an Indicator to NOT TO use an ORM as ORM's main strength is to save time and effort on trivial CRUD. I can imagine lot of queries in our App might be complicated joins including multiple tables.

What do you mean with Queryable? Queryable<T> ? Or do you mean: the user can define wahtever query he wants?

By Queryable I mean there no OLTP (no CRUD) involded, think of it as read-only/searchable/information web app(there will be a better tech term for it simple_smile ). No Users wouldn't define there own queries, however they would be different View of with Variety of searchable criterias. Keeping that in mind, do you still think ORM is still a good choice ?

Sure, it allows you to easy write the real queries to perform and to store data back into objects you can process in your pipeline back to the user/client, using typed objects.

  1. Again, this is more of generic architecture question. We will be implementing MVP(no MVC) architecture. Any suggestion for comsuming LLBL Entity all the way in View(Supervisor) or View should completely be abstracted(Passive) ? Any thoughts/suggestions are welcome.

What I heard (no expert in this) is that when you do MVP/MVC, it's almost a given that you create a new set of classes for the presentation only, which could be DTO's for example, which you generate with DTO templates (there are several available on the forum, search for "DTO templates") which make creating themselves from entities back/forth easier.

I thought of that, in fact you trackback using my userid on this forum, you will find I modified and posted back DTO template(as we used it in my last project). However once you do that you loose out of binding, in memory filter, paging etc support. Also by adding 1 more layer of transformation you get some more performance hit(even though it's minimal). That doesn't mean I am disagreeing to you, I am just trying to think out loud and weigh each option. Going with you suggestion, what we get is key objective of MVP/MVC => Clear Separation of Concerns.

I'm not a supporter of creating new layers of abstraction with new classes just because someone said that or cooked up a list of rules everyone should use, and thus I'm not a person who will recommend writing new classes for just the UI, simply because re-using existing classes which already do the job just fine is easier and easier to maintain as well, as you then don't have to maintain two sets of classes for the same thing and also not projection code back/forth.

Our entity classes support rich databinding for the reason that it makes life easier. I always have to laugh when I hear the stories from pundits who refuse to use databinding and do everything themselves using new sets of classes, projections and what not, to avoid... what exactly? All they do is creating more complexity.

Separation of concerns is a myth btw. There's a great scientific paper about it: http://citeseer.ist.psu.edu/tarr99degrees.html called N Degrees of Separation: Multi-Dimensional Separation of Concerns by Tarr, Ossher, Harrison. It's a fundamental paper about how to look at things and prioritize what you're doing. It's basicly saying that there's no such thing as separation of concerns: you always mix them in one way or the other: what one is after is that there's just 1 dominant concern. But which one is dominant? That depends on how you look at the class. For example, one could say that if you implement 2 or more interfaces ('SOLID' promotes that), the class already has two or more concerns which are dominant, as you can look at the class from two or more perspectives.

So if I were you, I'd use what works, and works well and go with that. The more layers you put on top of eachother to avoid complexity and make things less maintainable exactly do what you try to avoid: make things more complex and less maintainable because there's much more code.

That's not to say things shouldn't be split up across logical classes, they should. It's what our runtime framework does (if possible, databinding requires things to be integrated): validation, authorization etc., it's all supported through the same class (entity) but delegated to separate classes for the real work.

  1. Last and most important when v3 is coming out and will there be support for v2 to v3 project conversion ? Also does porting to newer version will include substantial effort ?

v3 will go beta before the end of the year as it looks now. Yes there will be conversion of v2 to v3 projects as in: the v2 .lgp file is converted to the v3 xml based project file using a template. V3 has a fully new designer and some new features for the runtime (like DDD value types and support for mixing inheritance types in 1 hierarchy), and the runtime /templates will be adjusted a bit to support these features (and move code from templates to runtime) but it won't be that massive as with v2.5: v3.0 is mainly about the designer, managing the project, creating new entities etc..

Looking forward to v3, finally lgp is making way for xml mapping, I am sure it would have been tough for you simple_smile

The binary file crap you mean? Well, yes and no. For v1 and 2, it was the easiest way to serialize the entire object graph and load it back in one way. Adding xml serialization to that was too painful, as things like that have to be integrated from the get-go or you're in for a big problem later on.

For v3, a couple of things had to be written first, as those things had to be everywhere and it should work very well: a solid graph class which had its algorithms implemented well, undo-redo behavior on a transparent level and xml serialization/deserialization. Using extension methods I can now serialize/deserialize the project stuff very easily. Undo / redo is literally everywhere (see my blogpost about the undo/redo paradox for an idea) and transparent: I can do whatever I want with the object graph and it's all undoable and redoable and I don't have to pain myself setting up the right commands nested into eachother, that's automatic. simple_smile (this is shipped with v3 in the library 'Algorithmia', every v3 licensee can use that code in their own projects).

It works with observers through events, and as validation is part of it like it is in the runtime (using validators plugged in), I can do databinding with my classes without any problem and everythings works: the editors for entity fields, mappings etc. are very small, as databinding (winforms) does a lot, combined with the event handlers. Seriously, if I had to write new classes for the UI, using MVC, it would have been a lot more work, especially without databinding or other nice things. Now I can leverage what's build into the classes that form the model: if I change a field in a grid, it raises events and as the grid modifies the property of an object in the model, other observers (like project explorers, other UI elements, or graphs) get notified and act accordingly. If I had to write new classes for the UI, this would have been a true pain because every change to an instance of these classes had to be reflected to the real object, which would have been a redirection without any reason: it doesn't make things easier, it doesn't require less code ... on the contrary.

So before you start, think about what you really need and build that and re-use what's already there. All those extra classes ... it doesn't make things easier.

Frans Bouma | Lead developer LLBLGen Pro