LLBGen - multithreaded prefetch paths

Posts   
 
    
miloszes
User
Posts: 222
Joined: 03-Apr-2007
# Posted on: 24-Jun-2010 13:35:59   

Hi,

is it possible (or it's already do by LLBLGen Runtime) to load related tables in the multithreaded environment?

For example if I Fetch a Parent with many children prefetch paths, are they (children) fetched using threading? So the query to different tables are run in separate threads? If not, is it possible to change that behavior and how?

What about linq-2-LLBLGen ? Does a parrallel LINQ work in the mentioned situation ?

Kind Regards, MiloszeS

arschr
User
Posts: 894
Joined: 14-Dec-2003
# Posted on: 24-Jun-2010 14:30:58   

Multi threaded prefetch path fetches is an open feature request.

miloszes
User
Posts: 222
Joined: 03-Apr-2007
# Posted on: 24-Jun-2010 14:40:24   

Great.

Do you have a RoadMap containing the mentioned feature? I'm curious when more/less it could be available.

Regards.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39903
Joined: 17-Aug-2003
# Posted on: 24-Jun-2010 16:12:04   

We've investigated this, but it's really not that efficient. The problem is that a prefetch path's child nodes depend on the data of the parent. So a prefetch path is really fetched sequentially. Merging the data using separate threads sounds tempting, but it actually gives a lot of overhead, as a parent can't be shared among multiple threads so the parent can't be a merge target for multiple children, which is actually the scenario you run into. So this requires locking which is actually slower.

Add to that that many applications are webapplications, which means you want to handle multiple requests at once instead of clog the server with a single request using parallel threads.

Frans Bouma | Lead developer LLBLGen Pro
miloszes
User
Posts: 222
Joined: 03-Apr-2007
# Posted on: 28-Jun-2010 09:46:55   

Otis wrote:

We've investigated this, but it's really not that efficient. The problem is that a prefetch path's child nodes depend on the data of the parent. So a prefetch path is really fetched sequentially. Merging the data using separate threads sounds tempting, but it actually gives a lot of overhead, as a parent can't be shared among multiple threads so the parent can't be a merge target for multiple children, which is actually the scenario you run into. So this requires locking which is actually slower.

Add to that that many applications are webapplications, which means you want to handle multiple requests at once instead of clog the server with a single request using parallel threads.

Yes, I agree with you. But not every scenario is same. I have a specific project and have a few places in which multi threading could give a real boost. If you could schedule the feature with low priority it would be very good for me. If it's not possible nor real I can also live without it - we'll make some workarounds.

Regards, MiloszeS