memoryleak with .GetValueOrDefault in a linq query

Posts   
 
    
tnero
User
Posts: 18
Joined: 09-Apr-2013
# Posted on: 10-May-2021 08:12:19   

Attached a project please look at Diagnostic Tools process memory in VS.

Using .GetvalueOrDefault() on a property causes memory leaks and is 2 to 3x slower.

Is this something that can be addressed? I'm on the hunt of some large new memory leak.

This is latest 5.8.1 (27 Apr 2021)

Leak:

        mylist = (from m in meta.TableLeak
                  select new MyDTO
                  {
                    ValueNull = m.ValueNull.GetValueOrDefault(),
                  }).ToList();

NoLeak:

        mylist = (from m in meta.TableLeak
                  select new MyDTO
                  {
                    ValueNull = m.ValueNull,
                  }).ToList();

PS. I don't see the attach file option - but this should be easily to reproduce.

Attachments
Filename File size Added on Approval
LLBLGenTest.zip 95,109 10-May-2021 08:12.35 Approved
tnero
User
Posts: 18
Joined: 09-Apr-2013
# Posted on: 10-May-2021 08:49:53   

Updated example:

Leak: ValueNull = m.ValueNull.GetValueOrDefault()

NoLeak: ValueNull = m.ValueNull ?? 0


    public class MyNotNullDTO
    {
      public decimal ValueNull { get; set; }
    }

    private static List<MyNotNullDTO> NoLeak()
    {
      List<MyNotNullDTO> mylist;

      using (var da = new DataAccessAdapter())
      {
        var meta = new LinqMetaData(da);

        mylist = (from m in meta.TableLeak
                  select new MyNotNullDTO
                  {                
                    ValueNull = m.ValueNull ?? 0
                  }).ToList();
      }

      return mylist;
    }

    private static List<MyNotNullDTO> Leak()
    {
      List<MyNotNullDTO> mylist;

      using (var da = new DataAccessAdapter())
      {
        var meta = new LinqMetaData(da);

        mylist = (from m in meta.TableLeak
                  select new MyNotNullDTO
                  {
                    ValueNull = m.ValueNull.GetValueOrDefault()
                  }).ToList();
      }

      return mylist;
    }
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 10-May-2021 09:40:07   

GetValueOrDefault is a method that's not converted to a faster alternative when fetching data. So for every row, the method is called, value is passed, result is coming back and then that value is used. The projection it's defined in is converted a lambda and compiled and then cached.

A tertiary operator (? : ) or ?? are converted to a faster alternative when projecting data at runtime (early null checks, faster paths when null occurs etc.).

You refer to a memory leak too. How did you measure that and did you use a profiler to see where the memory is kept referenced and what is kept in memory? A simple linq query isn't going to leak memory. Additionally, 'memory leaks' are only leaks if they end up in generation 2 and stay there in the GC, so the GC can't remove them

Frans Bouma | Lead developer LLBLGen Pro
tnero
User
Posts: 18
Joined: 09-Apr-2013
# Posted on: 10-May-2021 10:00:36   

Thanks for the feedback Otis - I will use the tertiary operator in future.

As for the memory leak - I have been trying to decode memory dumps from the live server and find it cryptic. LLBLGen is definitely not the cause on my live env. since its happening after merging 2 services together and LLBLGEN has stayed the same.

Just thought I would raise this issue.

thanks!!

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39588
Joined: 17-Aug-2003
# Posted on: 10-May-2021 11:27:33   

No problem in raising the issue simple_smile From our profiles on our system we can't see any memory leaks in any situation. There is memory consumption of course, and with every release we try to refactor code to bring it down but leaks aren't seen anywhere (it's always dropping to the level we started with).

Things to look into perhaps are: caches, do you cache object graphs? The one thing to realize is that if you fetch a customer + its orders for instance and keep one of the order instances in myCustomer.Orders in memory, the customer is kept in memory too, and thus all orders in the myCustomer.Orders collection as well! If your architecture requires you to this, investigate how you use the data cached and if single readonly sets (e.g. typedlists with poco objects, or derived models) aren't a better approach.

Using profilers is key to track down these memory leaks however. dotmemory or dottrace do excellent work in tracking down where things are allocated. Also, the memory profiler in visual studio is perhaps a bit arcane but with the right configuration (this really requires some time get it right tho) you can make it trace call chains and the bytes allocated along the call paths. This is key to get insight in what's happening along the way, and whether it is expected.

That last part is crucial: some things simply take performance and memory, and if you require more in a short period of time than the GC can clean up you will run out of memory, and it has nothing to do with a leak. So first try to investigate what you should expect, then profile to see what actually happens and then, if necessary, make changes so reality meets expectations simple_smile

Frans Bouma | Lead developer LLBLGen Pro