How to implement select saving through db views

Posts   
 
    
Posts: 134
Joined: 04-Mar-2005
# Posted on: 06-Apr-2005 20:48:54   

I've searched the forum for something like this but I've determined that my problem must be unique (yeah, right! wink ).

I have several different permutations of the same problem, but they all boil down to having a portion of the data stored in an accesible, but readonly, location; and another portion of the data stored in "my" database and being updatable.

I intially thought of creating DB views to pull the data together (in some cases it's extra fields, that are being added in the view, in other cases it's extra rows; but in both bases the "extra" data is the editable stuff), then create an entity on that view (the beta rocks!); and handle the updating of the table via a custom save to the table that underlies the "extra" data.

I'm now having second thoughts and I'm wondering whether it wouldn't be better to create the tables as separate entities and then merge them together in my DAL so that they appear as one entity at levels above that.

But that's as far as I got... confused

Does anyone have suggestions or recommendations or, better still, "I've done this exact thing and here's the code I used"? (A guy can hope!).

Thanks in advance

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 07-Apr-2005 09:52:54   

What are the reasons you have second thoughts, as these would give more insight in why you think differently now and perhaps these second thoughts can be addressed simple_smile

Frans Bouma | Lead developer LLBLGen Pro
Posts: 134
Joined: 04-Mar-2005
# Posted on: 07-Apr-2005 15:17:25   

My second thoughts are just around the amount of work involved in option 1 (setting up DB views and writing custom save routines) and whether option 2 (handling it in classes) mightn't be a more elegant option.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 07-Apr-2005 20:24:06   

ChicagoKiwi wrote:

My second thoughts are just around the amount of work involved in option 1 (setting up DB views and writing custom save routines) and whether option 2 (handling it in classes) mightn't be a more elegant option.

The views indeed require more work, though are more efficient, as working with sets of data in C#/VB.NET can be slow sometimes (slower than a db can do it). It thus depends on what amount of data you have to work with simple_smile

Frans Bouma | Lead developer LLBLGen Pro
Posts: 134
Joined: 04-Mar-2005
# Posted on: 08-Apr-2005 14:35:07   

The amount of data that I'll be dealing with will never be large (less than 100 entities in a collection at any one time, say - obviously the table underneath are bigger though) however I need to implement some pretty extensive searching (not full-text, just on lots of properties that could return results from one of both of my underlying tables).

Does this suggest one of my possible solutions (or another) over the other?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 09-Apr-2005 11:51:42   

ChicagoKiwi wrote:

The amount of data that I'll be dealing with will never be large (less than 100 entities in a collection at any one time, say - obviously the table underneath are bigger though)

Never underestimate the amount of data an application has to work with, though that's just a general FYI wink

however I need to implement some pretty extensive searching (not full-text, just on lots of properties that could return results from one of both of my underlying tables).

Does this suggest one of my possible solutions (or another) over the other?

Searching should be done on the db server, though I don't see why views would make that faster, as a view is considered as a select statement which is mapped into the query at runtime by teh database server anyway, so I think whatever you choose will perform equally.

Frans Bouma | Lead developer LLBLGen Pro
Posts: 134
Joined: 04-Mar-2005
# Posted on: 11-Apr-2005 14:42:29   

Otis wrote:

ChicagoKiwi wrote:

The amount of data that I'll be dealing with will never be large (less than 100 entities in a collection at any one time, say - obviously the table underneath are bigger though)

Never underestimate the amount of data an application has to work with, though that's just a general FYI wink .

Actually on this one I'm lucky - our archaic order system only allows 99 lines per contract... Nevertheless the point is well made and taken - thanks.

however I need to implement some pretty extensive searching (not full-text, just on lots of properties that could return results from one of both of my underlying tables).

Does this suggest one of my possible solutions (or another) over the other?

Searching should be done on the db server, though I don't see why views would make that faster, as a view is considered as a select statement which is mapped into the query at runtime by the database server anyway, so I think whatever you choose will perform equally.

I wasn't suggesting that I was going to pull the entire table contents and then filter in .NET. I was just trying to round out some of the functionality I need.

It seems that you don't have any strong opinion either way on this. Does anyone else?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 11-Apr-2005 18:56:58   

ChicagoKiwi wrote:

Otis wrote:

ChicagoKiwi wrote:

The amount of data that I'll be dealing with will never be large (less than 100 entities in a collection at any one time, say - obviously the table underneath are bigger though)

Never underestimate the amount of data an application has to work with, though that's just a general FYI wink .

Actually on this one I'm lucky - our archaic order system only allows 99 lines per contract...

Cool, an example how legacy stuff can be great! simple_smile

It seems that you don't have any strong opinion either way on this. Does anyone else?

I think a 'strong' opinion for either situation is not that possible, as it depends on the data amount whats best. (IMHO)

Frans Bouma | Lead developer LLBLGen Pro
Posts: 134
Joined: 04-Mar-2005
# Posted on: 18-Aug-2005 23:34:12   

After putting this one on the back burner for a while I now need to solve the problem (damn these projects that actually need to be completed!).

I ended up setting up views which are a union of data from two sources: one which is editable and one which is not, and I have a flag to indicate which is which.

So at save time I pass the entityToBeSaved to a helper class which knows which entities fall into this category. For those entities that fall into this category I copy each field to another entity based directly on the editable portion of the data and pass back the new entityToBeSaved rather than the one based on the view. That much works.

The problem is that I now have an entity which is outOfSync and can't be used (the one that didn't get saved). When adding I need to discard this entity and refetch the entity based on the view using the PK from the other entity I just saved (the PK is from an Oracle sequence).

So my question is: how can I snag the id of the new entity sometime after the UoW commit and then refetch the entity? Or is there a better way to go about this?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 19-Aug-2005 09:34:58   

You can read the new PK value of the new entity after the commit, then set the PK of the old entity with that value and refetch that with FetchEntity. That's the easiest way to do it I think.

Frans Bouma | Lead developer LLBLGen Pro
Posts: 134
Joined: 04-Mar-2005
# Posted on: 19-Aug-2005 14:34:29   

Otis wrote:

You can read the new PK value of the new entity after the commit, then set the PK of the old entity with that value and refetch that with FetchEntity. That's the easiest way to do it I think.

Forgive me if I'm being slow... How do I get a reference to the entity after a commit? The UnitOfWork is empty after the commit. Could I use the AddCallBack function here (although there's no PostEntityInsert slot) or should this only be used for calling StoredProcs? confused

Posts: 134
Joined: 04-Mar-2005
# Posted on: 19-Aug-2005 16:51:17   

ChicagoKiwi wrote:

Forgive me if I'm being slow...

I was.flushed I grabbed the uow GetEntityElementsToInsert and GetEntityElementsToUpdate before the commit and then iterated through the collection _after _the commit. The entities then have the keys filled and I can refetch my original entity.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 20-Aug-2005 12:15:28   

ChicagoKiwi wrote:

ChicagoKiwi wrote:

Forgive me if I'm being slow...

I was.flushed I grabbed the uow GetEntityElementsToInsert and GetEntityElementsToUpdate before the commit and then iterated through the collection _after _the commit. The entities then have the keys filled and I can refetch my original entity.

You know, doing some unittests on the inheritance code I ran into the same issue smile I used a uow to save the test entities, but when I created a delete uow from that same uow it was always empty, of course, as the commit cleared the uow.

the GetEntityElementsToInsert is indeed the right collection. simple_smile BEFORE the commit of course wink

Frans Bouma | Lead developer LLBLGen Pro
Posts: 134
Joined: 04-Mar-2005
# Posted on: 25-Aug-2005 18:37:00   

I've run into a further problem on this issue. The GetEntityElementsToInsert and GetEntityElementsToUpdate methods only return the elements/entities that have been explicitly added to the UoW, they don't include items that are going to be saved through recursion. Is there any way of getting at these elements?
It would be nice if the UoW had a method that flattened the recurse tree and made all the elements available. (e.g. if a ProjectEntity is in the UoW with a related SubprojectEntity and recurse = true calling the flatten method will add the SubprojectEntity to the UoW directly, thus exposing it via GetEntityElementsTo..., and set recurse on the ProjectEntity to false).

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 26-Aug-2005 10:54:01   

ChicagoKiwi wrote:

I've run into a further problem on this issue. The GetEntityElementsToInsert and GetEntityElementsToUpdate methods only return the elements/entities that have been explicitly added to the UoW, they don't include items that are going to be saved through recursion. Is there any way of getting at these elements?
It would be nice if the UoW had a method that flattened the recurse tree and made all the elements available. (e.g. if a ProjectEntity is in the UoW with a related SubprojectEntity and recurse = true calling the flatten method will add the SubprojectEntity to the UoW directly, thus exposing it via GetEntityElementsTo..., and set recurse on the ProjectEntity to false).

They're not rolled out until save time, correct. The uow simply calls a recursive save on the entities added to the uow if the save is recursive. The reason they're not rolled out when they're added is that the operation can be time consuming (relatively) if you have a large graph, and in the case of abandonement of the uow, that's thus a waste of performance.

Frans Bouma | Lead developer LLBLGen Pro
Posts: 134
Joined: 04-Mar-2005
# Posted on: 26-Aug-2005 14:39:23   

Otis wrote:

ChicagoKiwi wrote:

I've run into a further problem on this issue. The GetEntityElementsToInsert and GetEntityElementsToUpdate methods only return the elements/entities that have been explicitly added to the UoW, they don't include items that are going to be saved through recursion. Is there any way of getting at these elements?
It would be nice if the UoW had a method that flattened the recurse tree and made all the elements available. (e.g. if a ProjectEntity is in the UoW with a related SubprojectEntity and recurse = true calling the flatten method will add the SubprojectEntity to the UoW directly, thus exposing it via GetEntityElementsTo..., and set recurse on the ProjectEntity to false).

They're not rolled out until save time, correct. The uow simply calls a recursive save on the entities added to the uow if the save is recursive. The reason they're not rolled out when they're added is that the operation can be time consuming (relatively) if you have a large graph, and in the case of abandonement of the uow, that's thus a waste of performance.

So the answer's 'no'? wink If this 'flatten' method were implemented it would be give the user/coder control over when the recursion was done. I agree that if the UoW is discarded it would be less efficient, but giving the option to recurse at a time other than within the commit would be a great bonus - at least to me! Apart from when a UoW is discarded I can't see a whole lot of difference:

calling: UoW.Commit() continues to act as it does now

or

calling: UoW.Flatten() recurses and adds all items to the UoW ... UoW.Commit() has nothing to recurse (since it's already been done) and commits the elements.

Or am I being overly simplistic and do I just need to ensure that all my UoW.AddForSave calls are explicit rather than relying on recursion to pick them up?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39933
Joined: 17-Aug-2003
# Posted on: 27-Aug-2005 11:45:39   

ChicagoKiwi wrote:

Otis wrote:

ChicagoKiwi wrote:

I've run into a further problem on this issue. The GetEntityElementsToInsert and GetEntityElementsToUpdate methods only return the elements/entities that have been explicitly added to the UoW, they don't include items that are going to be saved through recursion. Is there any way of getting at these elements?
It would be nice if the UoW had a method that flattened the recurse tree and made all the elements available. (e.g. if a ProjectEntity is in the UoW with a related SubprojectEntity and recurse = true calling the flatten method will add the SubprojectEntity to the UoW directly, thus exposing it via GetEntityElementsTo..., and set recurse on the ProjectEntity to false).

They're not rolled out until save time, correct. The uow simply calls a recursive save on the entities added to the uow if the save is recursive. The reason they're not rolled out when they're added is that the operation can be time consuming (relatively) if you have a large graph, and in the case of abandonement of the uow, that's thus a waste of performance.

So the answer's 'no'? wink If this 'flatten' method were implemented it would be give the user/coder control over when the recursion was done. I agree that if the UoW is discarded it would be less efficient, but giving the option to recurse at a time other than within the commit would be a great bonus - at least to me! Apart from when a UoW is discarded I can't see a whole lot of difference:

calling: UoW.Commit() continues to act as it does now

or

calling: UoW.Flatten() recurses and adds all items to the UoW ... UoW.Commit() has nothing to recurse (since it's already been done) and commits the elements.

Or am I being overly simplistic and do I just need to ensure that all my UoW.AddForSave calls are explicit rather than relying on recursion to pick them up?

What you need is a routine which gives you a list of entities which will be affected by a recursive save of a given entity X. This code is currently in the SaveEntity() routine (it calls itself recursively), and uses 3 routines in the current entity: GetDependentRelatedEntities, GetDependingRelatedEntities and GetMemberEntityCollections. For each entry in GetDependentRelatedEntities, it calls save again prior to act on the current entity. It makes sure the current entity is in a list of 'already in progress', so when you for example save 'order' and it depends on 'customer' and customer is saved, and it then processes all depending entities (like order), it ends up with order again, which will then cause a loop.

If you take a look at the SaveEntity() routine in DataAccessAdapter, you can easily spot the mechanism. After 1.0.2005.1 goes into beta, please remind me to write this routine for you, as I've more time then.

Frans Bouma | Lead developer LLBLGen Pro