DetermineActionQueues

Posts   
 
    
mikeg22
User
Posts: 411
Joined: 30-Jun-2005
# Posted on: 03-May-2006 01:08:21   

I'm having big performance problems saving large EntityCollections. The DetermineActionQueues(entityCollectionToSave,...) is taking a looooong time to finish, and it looks like it is because of two things. First,


ArrayList sortedGraphList = this.ProduceTopologyOrderedList(entityToSave);
if(inQueue.ContainsKey(entityToSave.ObjectID))
{
    return;
}

executes for every entity in the collection, so a Topo list is created for every entity in the collection, even if that entity is already in "inQueue"

next,


inQueue.Add(toSave.ObjectID, null);

happens only for "dirty" entities, so entities in the root entity collection that are not dirty do not get added, even if they have been traversed already by a previous call to DetermineActionQueues.

I'm guessing InQueue is supposed to be a quick lookup for entities that are already in "some kind" of queue, so putting them in only for dirty entities seems fair so that they don't get added more than once. However, wouldn't keeping a hashtable of traversed entities be useful so that DetermineActionQueues(entityCollectionToSave,...) doesn't end up repeating traversal code in the case where entities in entityCollectionToSave are part of the same graph? Because of how it works now, I have 1200 entities in a collection I am saving, and they are all part of the same Topo list, and the Topo list gets created for each and every one of them inside DetermineActionQueues(entityToSave,...). This takes an unbelievable amount of time at the moment cry

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39928
Joined: 17-Aug-2003
# Posted on: 03-May-2006 09:03:11   

I'll look into it. What you suggest could indeed help a lot. The inQueue is a lookup for entities which are already in the save queue, so they're not added again.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39928
Joined: 17-Aug-2003
# Posted on: 04-May-2006 11:57:50   

Ok, I added the code, it was a few lines, and I didn't even had to break any public method signature smile .

I also added the same test code to the UnitOfWork, as it too suffers from this.

Is it possible for you to test this new build out to see if it fixes your problem? All my unittests succeed so I think the code seems solid simple_smile .

It should work as planned, as the graph sorted takes into account any entity in the graph, so if an entity is in a graph, it will end up in the sorted list, and it can't be in 2 graphs, as the graphs it connects then form 1 graph, not two.

I store, in the loop that's running over the sorted list, all objectids in a hashtable, which is returned as well, so I only process entities which are really in the graph of the entity sorted, so I can't think of a way how this can leave any entity uncovered, i.o.w.: it should save all entities correctly and be much much faster simple_smile

Frans Bouma | Lead developer LLBLGen Pro
mikeg22
User
Posts: 411
Joined: 30-Jun-2005
# Posted on: 04-May-2006 19:20:59   

Yeah, I can test it. Are you going to email the class files?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39928
Joined: 17-Aug-2003
# Posted on: 05-May-2006 08:31:26   

I mailed them to the email address you specified in your profile. Please let me know if they work or don't work.

Frans Bouma | Lead developer LLBLGen Pro
mikeg22
User
Posts: 411
Joined: 30-Jun-2005
# Posted on: 05-May-2006 19:07:38   

Otis wrote:

I mailed them to the email address you specified in your profile. Please let me know if they work or don't work.

Spent the last day trying to fix our Janus Grid bug but I'll test the new dll this morning simple_smile

mikeg22
User
Posts: 411
Joined: 30-Jun-2005
# Posted on: 05-May-2006 19:52:21   

That fixed the problem!

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39928
Joined: 17-Aug-2003
# Posted on: 05-May-2006 20:46:42   

Cool simple_smile Thanks for testing simple_smile

Frans Bouma | Lead developer LLBLGen Pro