Huge Prefetch Path

Posts   
 
    
JoseMartin
User
Posts: 3
Joined: 27-Feb-2019
# Posted on: 27-Feb-2019 16:46:53   

Hello LLBLGen Team:

I have the following requirement: We desire fetch a specific entitity and get all entities relations for that entity, in other words I wish build a graph of all relations. My approach is using the prefetchs paths in order to build a dynamic prefetch through reflection.

This is my code:

var prefetchPath = new PrefetchPath2((int)EntityType.CustomerEntity); [b]<-- ROOT 
[/b]
var customerEntityType = typeof(CustomerEntity);

foreach (var propertyPrefetch in customerEntityType.GetProperties(BindingFlags.Public | BindingFlags.Static).Where(x => x.Name.StartsWith("PrefetchPath")).ToList())
            {
                var prefetchPathElement = (IPrefetchPathElement2)propertyPrefetch.GetValue(null, null);
                var entityTypeToFetch = (EntityType)Enum.ToObject(typeof(EntityType), prefetchPathElement.ToFetchEntityType);

                    var prefetchElement = prefetchPath.Add(prefetchPathElement);

                    BuildPrefetchPaths(entityTypeToFetch.ToString(), prefetchElement); [b]<--CALL TO RECURSIVE METHOD[/b]
                }
            }

Recursive Method:


private void BuildPrefetchPaths(string originEntity, IPrefetchPathElement2 prefetchPathElement)
        {
            var factoryEntity = _factoryManager.Factories.First(f => f.ForEntityName.Equals(originEntity));
            var instanceEntity = factoryEntity.Create();
            var typeEntity = instanceEntity.GetType();

            foreach (var prefetchProperty in typeEntity.GetProperties(BindingFlags.Public | BindingFlags.Static).Where(x => x.Name.StartsWith("Prefetch")).ToList())
            {
                var prefetchPropertyToAdd = (IPrefetchPathElement2)prefetchProperty.GetValue(null, null);

                    EntityType entityTypeToFetch = (EntityType)Enum.ToObject(typeof(EntityType), prefetchPropertyToAdd.ToFetchEntityType);

                        var prefetchElement2 = prefetchPathElement.SubPath.Add(prefetchPropertyToAdd);

                        BuildPrefetchPaths(entityTypeToFetch.ToString(), prefetchElement2);
            }
        }

The big problem is does not work because when I call the recursive method of the call to DataAdapter is not retrieve any information and does not throw exception.

It is some strange because if I don't do the call to recursive method inside the method called 'BuildPrefetchPaths' working well. I don't known if exists a restriction for a huge prefetch path.

Can you help me with some comments or ideas? The version that we use is LLBLGen is 4.2 and as additional information the database target have 97 tables.

Walaa avatar
Walaa
Support Team
Posts: 14946
Joined: 21-Aug-2005
# Posted on: 27-Feb-2019 20:05:49   

For any recursive code, there has to be a stopping condition. What's the stopping condition in your case?

JoseMartin
User
Posts: 3
Joined: 27-Feb-2019
# Posted on: 28-Feb-2019 19:16:41   

Sorry for not including the part where I handle my base case that gives exit to the recursion, I did not include it to focus on the code problem that I have.

When I get out of that code function I get a big prefetch path and when I run this function where I get the collection it does not do anything.

using (var adapter = new DataAccessAdapter(_connectionString, false, CatalogNameUsage.Clear, string.Empty))
{
    var parameters = new QueryParameters
    {
        PrefetchPathToUse = prefetchPath,
        CollectionToFetch = customers,
        FilterToUse = filter,
        AllowDuplicates = false
    };

    await adapter.FetchEntityCollectionAsync(parameters, cancellationToken: CancellationToken.None);
}

No throws exceptions or send any message, any ideas?

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 01-Mar-2019 07:01:14   

Your recursive method, it will build a very long graph with redundant paths, as in:

Customer Order Customer etc...

Be careful of that. Please examine the graph in debug to see whether it make sense. Also please check this link... and go to the "Common mistakes" section.

In general I would recommend to write a prefetchPath graph manually, not using recursive functions where you loose control on what exactly is being fetched. Not to mention the possible performance issues in your application (memory, network, etc).

David Elizondo | LLBLGen Support Team
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 01-Mar-2019 10:42:10   

Indeed. If you fetch the related entities of an entity, and the related entities of those entities etc. you end up potentially fetching all entities in the database.

If it's one level deep, sure, but if it's potentially infinite in length, you could end up having all entities in the path, which might not what you want.

Frans Bouma | Lead developer LLBLGen Pro
JoseMartin
User
Posts: 3
Joined: 27-Feb-2019
# Posted on: 01-Mar-2019 17:27:32   

Hi, I was thinking about what you are saying, you are right, this can be a serious performance problem because I am bringing all the entities from the database. The problem that I have is that now the client is what requires that I can obtain from a root entity all its relations with other entities and in turn all the entities that are related to those entities.

Maybe it's not the best practice but that's what I'm asked for, from the side of the cyclic reference I have this controlled and I implement a control where I have a relationship like this A -> B and B -> A I take control so that again do not pass A -> B

I will have to inspect the graph that is generated but for what I see it is well built, only it is huge and at the moment of executing the filling of the collection it does not do anything.

I can not build this part manually because they want this to be that dynamic.

Have you ever faced a problem of these characteristics?

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 02-Mar-2019 06:59:15   

JoseMartin wrote:

Have you ever faced a problem of these characteristics?

Personally, no. As I said, I prefer the other way, it's not dynamically, but anyway if you modify the schema you also have to modify the llblgen project, and the code. So the dynamism is a perception.
If you have to do it recursively, stick with your code, just be careful of the possible performance issues and how to deal with them.

David Elizondo | LLBLGen Support Team