Hi,

I've written some code that gets a huge list of "readings" in the database. It needs to perform some simple migration (changing a "Float" value in each reading).

The problem is that in a client database, there are 66,000 of these readings. Loading the entity collection is OK (takes about a minute), and performing the simple loop through to do the actual migration is OK (takes 1 - 2 minutes), but saving the entire entity collection kills aspnet_wp.exe. This is the error that happens about 5 minutes after I call SaveEntityCollection:

aspnet_wp.exe (PID: 3160) was recycled because memory consumption exceeded the 613 MB (60 percent of available RAM).

Sooooo, I need to work out why all the memory is consumed by the save, and if there are any alternatives that I can use while saving to not use so much memory...

More information follows:

I am fetching the entity collection using a pre-fetch path. Readings belong to "Monitoring Records", so MonRecords 1:N Readings. I save the collection with this command:

adaptor.SaveEntityCollection(readings, false, true);

I am currently using ASP.NET 2, with SQL Server 2000, and the latest version of LLBL (1.0.2005.1 Final, December 2005).

Is there a way to save the entity collection without killing memory?

You're saving all entities in a single transaction. You're also recursing through the graph. What happens is that everything related to the transaction is kept in memory till it's committed.

You might also want to upgrade to the latest runtime build for 1.0.2005.1, which has been released on July 19th 2006. The graph sorting algorithm has been rewritten in 1.0.2005.1 after it was released to make it more flexible and faster.

Could you first check if the graph sorter is the problem? Do this by using a UnitOfWork and adding the collection to the unitofwork, and then call the method Uow.ConstructSaveProcessQueues.

if that succeeds fairly quickly (should perhaps take a few seconds) then you can proceed by traversing the queues one by one without a transaction by using the uow.GetInsertQueue and the uow.GetUpdateQueue results. So saving the entities without a transaction, nonrecursively.

Thanks Otis

Yep, it took a second, so I'll try using the UOW to save without transactions!

Matt.

Hi,

Just got back to this now. I think I may be suffering from Friday "doh-ness", but I'm unsure how to save these Entities individually without the transaction. Here's my code thus far:

            UnitOfWork2 uow = new UnitOfWork2();
            uow.AddCollectionForSave(readings,false,true);
            uow.ConstructSaveProcessQueues();

           System.Collections.ArrayList al = uow.GetUpdateQueue();
            foreach (ActionQueueElement element in al)
            {
                   // What goes here?
            }

Hi,

I guess you can write :

adapter.SaveEntity(element.Entity);

Ah, almost...

adaptor.SaveEntity((IEntity2) element.Entity);

I didn't realise IEntityCore and IEntity2 belonged to the same family.

OK, so this code compiles. But, it still wont work. It stilll consumes aboutover half a gig of memory and then falls over. My guess is that this code still uses a transaction?

I have upgraded to the latest runtime DLL's. I know that this generally will consume a lot of memory given what I am loading into an entity collection, but its just the saving that kills it.

MattWoberts wrote:

Ah, almost...

adaptor.SaveEntity((IEntity2) element.Entity);

I didn't realise IEntityCore and IEntity2 belonged to the same family.

OK, so this code compiles. But, it still wont work. It stilll consumes aboutover half a gig of memory and then falls over. My guess is that this code still uses a transaction?

I have upgraded to the latest runtime DLL's. I know that this generally will consume a lot of memory given what I am loading into an entity collection, but its just the saving that kills it.

Specify false for recurse. Adapter sets recurse to true. It's essential you use false for recurse so you won't get a transaction.

Thanks again Frans, that seems to have sorted it.

I'll have to read up on transactions in LLBLGen, my understanding was that "recurse" controlled whether LLBLGen saved any related entities (got through a prefetch path) with the entity being saved, I didn't know it had any bearing on whether or not transactions would be used.

Matt

LLBLGen Pro by design starts a transaction if there are 2 or more queries involved in a single DML call. So if you save something recursively, a transaction is started if there's no transaction. Saving an entity collection: ditto. Also, if you save an entity which is in a hierarchy of type Targetperentity, it starts a transaction, as it needs 2 or more inserts.

It doesn't start a transaction for single statement queries, e.g. a single insert, as the single insert is a transaction inside the DB.

So doing it without a transaction did solve it?

Sure did