Any kind of SQL batching at all in Self Servicing?

Posts   
 
    
gfunny
User
Posts: 20
Joined: 21-Apr-2005
# Posted on: 14-Oct-2010 20:50:11   

I'm looking for performance gains for batching SQL Load or Insert/Updates - I'm surprised that this would not be available in any form? Pretty please - I've read some threads but most date back to 2005--2006.

MTrinder
User
Posts: 1461
Joined: 08-Oct-2008
# Posted on: 14-Oct-2010 21:51:59   

What sort of volumes are you talking about ? All inserts and updates are done in one batch anyway - how do you envisage that splitting them into multiple batches will help performance?

Matt

gfunny
User
Posts: 20
Joined: 21-Apr-2005
# Posted on: 14-Oct-2010 22:00:55   

I have a process that occurs - 5 entities are inserted into database. It would be nice to send all 5 over to be saved at once - wiithout having to reset the connection 4 times, and repopulating each entity after each save b/c of the new id that has to be retrieved to be used as a foriegn key on the next entity that is saved. This process can be called over 100 times in 1 execution, so the connection resets and retrieval for foreign key ids would add up. Trying to find ways to make the process faster.

MTrinder
User
Posts: 1461
Joined: 08-Oct-2008
# Posted on: 14-Oct-2010 22:05:05   

So are the enties all in PK-Fk relationships - something along the lines of Customer -> Order -> OrderLine ->Product ....?

If this is the case you should not need to save them one at a time - as long as LLBLGen knows about the relationships (ie it has picked them up in the designer and created properties for the related entities) calling a Save on any of the enties with the recursive parameter set to "true" will cause all of the entities to be saved in the correct order, will all of the PK-FK values set correctly.

If you show us some of the code you are using we can take a closer look at how you are doing things.

Matt

gfunny
User
Posts: 20
Joined: 21-Apr-2005
# Posted on: 14-Oct-2010 22:07:42   

Ok, so it won't repopulate each entity with a select statement? And is it sent over in 1 batch or is it recursive (4-5 connection resets)

thanks for your response!

MTrinder
User
Posts: 1461
Joined: 08-Oct-2008
# Posted on: 14-Oct-2010 22:17:55   

The repopulation is based on the value of the other parameter (refetch after save). If this is false it won't run the additional SELECT to repopulate the entities, although it will still retrieve and populate the PK-FK values.

It's not sent in one batch to the server, as the work to populate and set the PK/FK values is done in code, but it is all done in the context of one connection - so no resets...

Matt

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39905
Joined: 17-Aug-2003
# Posted on: 15-Oct-2010 11:15:02   

Batching isn't implemented because it interferes with other functionality we have in the runtime, which works on a per-entity basis, like auditing, authorization and concurrency checks etc.. This means that if batching is used, these features won't work so we don't implement batching. So having more features actually hurts in this case, which is in a way ironic, but unfortunately reality. We did look at implementing it with private reflection hacks like NHibernate imlements it (as the SQL Server batching logic is internal to the SQL client so only MS can use it) but we couldn't integrate it with the rest of the pipeline, as we need feedback whether a single entity was updated/inserted properly, even though you execute them in a 'batch' at a higher level (e.g. unit of work, collection save).

If you're looking for bulk-inserts, it's best to look for the bulk copy import feature of SQL Server, because that import system bypasses the SQL interpreter, making it much faster than executing insert statements.

Frans Bouma | Lead developer LLBLGen Pro
gfunny
User
Posts: 20
Joined: 21-Apr-2005
# Posted on: 18-Oct-2010 05:09:05   

Thanks for your responses.

Is there anyway to set an entity MarkSavedEntitiesAsFetched instead of at a global level?

Is there any performance gain by simply putting everything in a UnitofWork?

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 18-Oct-2010 07:04:46   

gfunny wrote:

Is there anyway to set an entity MarkSavedEntitiesAsFetched instead of at a global level?

theEntity.Fields.State = EntityState.Fetched;

gfunny wrote:

Is there any performance gain by simply putting everything in a UnitofWork?

No. I mean LLBLGen is already in a performance fashion. You can UnitOfWork if you need to group the actions in one object and maybe pass it along screens or serialize it to be saved in another application stage.

David Elizondo | LLBLGen Support Team
gfunny
User
Posts: 20
Joined: 21-Apr-2005
# Posted on: 18-Oct-2010 07:06:50   

Thanks! U look like a Jedi Night on Tattooine

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 18-Oct-2010 07:10:55   

gfunny wrote:

Thanks! U look like a Jedi Night on Tattooine

Haha. Thanks

David Elizondo | LLBLGen Support Team