Linq error when left joining onto a group by sub query

Posts   
 
    
worldspawn avatar
worldspawn
User
Posts: 321
Joined: 26-Aug-2006
# Posted on: 10-Feb-2010 01:14:18   

Hello,

Using 2.6, Adapter.


LinqMetaData meta = new LinqMetaData(Adapter);

            var instituteContacts = from c in meta.InstituteContact
                                    group c by c.InstituteId into g
                                    select new { g.Key, Id = g.Min(p=>p.Id) };

            var foo = (from c in meta.Institute
                    join z in instituteContacts on c.Id equals z.Key into tmpICg
                    from icg in tmpICg.DefaultIfEmpty()
                    join x in meta.InstituteContact on icg.Id equals x.Id into tmpIC
                    from ic in tmpIC.DefaultIfEmpty()
                    select new Institute
                    {
                        InstituteId = c.Id,
                        Name = c.Name,
                        AgreementExpiry = c.AgreementExpiry,
                        ContactPerson = ic.ContactName,
                        Number = ic.Telephone,
                        EmailAddress = ic.Email
                    });
            foo.ToList();

There is a 1:M rel with institute and contact. What I am trying to achieve here is to get a list of institutes and also the contact details of the first contact associated with the institute if it's there.

Now this works fine with inner joins (except the result excludes any institutes that have no contact defined). But with the left joins it fails, not a SqlException either, no sql is executed.

[IndexOutOfRangeException: Index was outside the bounds of the array.] SD.LLBLGen.Pro.LinqSupportClasses.ValueListProjectionDefinition.AddProjectionListToProjection(ProjectionListExpression toAdd, ITemplateGroupSpecificCreator frameworkElementCreator, IElementCreatorCore generatedCodeElementCreator, MappingTracker trackedMappings) +647 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleProjectionExpression(ProjectionExpression expressionToHandle) +347 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +1058 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleGroupJoinExpression(GroupJoinExpression expressionToHandle) +153 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleGroupJoinExpression(GroupJoinExpression expressionToHandle) +35 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +646 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleAndProcessJoinExpressionSide(SetExpression side, Expression sideSelector, Expression& handledSide, Expression& handledSideSelector, String& aliasSide) +24 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleJoinExpression(JoinExpression expressionToHandle) +884 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +769 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleGroupJoinExpression(GroupJoinExpression expressionToHandle) +41 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleGroupJoinExpression(GroupJoinExpression expressionToHandle) +35 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +646 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleAndProcessJoinExpressionSide(SetExpression side, Expression sideSelector, Expression& handledSide, Expression& handledSideSelector, String& aliasSide) +24 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleJoinExpression(JoinExpression expressionToHandle) +884 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +769 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.HandleExpressionTree(Expression expression) +750 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.Execute(Expression expression) +13 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.System.Linq.IQueryProvider.Execute(Expression expression) +17 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProQuery1.Execute() +16 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProQuery1.System.Collections.Generic.IEnumerable<T>.GetEnumerator() +16 System.Collections.Generic.List1..ctor(IEnumerable1 collection) +7667686 System.Linq.Enumerable.ToList(IEnumerable1 source) +61 IAEC.SMS.DomainServices.InstituteService.List() in D:\Projects\IAEC\Main\Source\IAEC Student Management System v1.0\app\IAEC.SMS.DomainServices\InstituteService.cs:49 IAEC.SMS.Web.Controllers.InstituteController.InstituteList(Int32 pageSize, Int32 pageIndex, String sortBy, Boolean sortDirection) in D:\Projects\IAEC\Main\Source\IAEC Student Management System v1.0\app\IAEC.SMS.Web.Controllers\InstituteController.cs:50 lambda_method(ExecutionScope , ControllerBase , Object[] ) +233 System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +17 System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary2 parameters) +178 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary2 parameters) +24 System.Web.Mvc.&lt;&gt;c__DisplayClassa.&lt;InvokeActionMethodWithFilters&gt;b__7() +52 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func1 continuation) +254 System.Web.Mvc.<>c__DisplayClassc.<InvokeActionMethodWithFilters>b__9() +19 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(ControllerContext controllerContext, IList1 filters, ActionDescriptor actionDescriptor, IDictionary2 parameters) +192 System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +399 System.Web.Mvc.Controller.ExecuteCore() +126 System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) +27 System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext) +7 System.Web.Mvc.MvcHandler.ProcessRequest(HttpContextBase httpContext) +151 System.Web.Mvc.MvcHandler.ProcessRequest(HttpContext httpContext) +57 System.Web.Mvc.MvcHandler.System.Web.IHttpHandler.ProcessRequest(HttpContext httpContext) +7 System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +181 System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +75

I dont think this is one of those goofy things i try to do on occasion smile Seems like this should work. Or perhaps theres another way to implement the same thing.

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 10-Feb-2010 04:55:09   

What is your runtime library version? (http://llblgen.com/tinyforum/Messages.aspx?ThreadID=7725) Make sure you are using the latest.

Is there any generated sql?

David Elizondo | LLBLGen Support Team
worldspawn avatar
worldspawn
User
Posts: 321
Joined: 26-Aug-2006
# Posted on: 10-Feb-2010 05:29:51   

Hi daelmo,

new profile pic i see. U look very dark & mysterious now.

Runtime (ORM) Was 2.6.9.1005... i have now updated to the latest and no change. No sql is generated.

Walaa avatar
Walaa
Support Team
Posts: 14993
Joined: 21-Aug-2005
# Posted on: 10-Feb-2010 10:19:48   

There is a 1:M rel with institute and contact. What I am trying to achieve here is to get a list of institutes and also the contact details of the first contact associated with the institute if it's there

Why don't you use PrefetchPaths?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39866
Joined: 17-Aug-2003
# Posted on: 10-Feb-2010 10:45:50   

Could you cut the query down to the bare minimum which fails? It's now too complex to debug or to setup for debugging (that might sound weird but the expression tree is very deep)

Frans Bouma | Lead developer LLBLGen Pro
worldspawn avatar
worldspawn
User
Posts: 321
Joined: 26-Aug-2006
# Posted on: 10-Feb-2010 12:23:15   

I dont like to use prefetch paths unless i need to. In this case a join should work and it means one less query.

Also, (i'm guessing) because I am selecting into a custom class all the subsequent modifications i make on my IQueryable such as OrderBy, Take, Skip and Count seem to be done in memory. I see only the initial select being executed against sql, so I'm not sure what the behaviour of the prefetch will be. Maybe it will load the entire related table and look for references in mem, i'm pretty sure I have seen that happen before.

Frans, hows this?


var instituteContacts2 = from c in meta.InstituteContact
                                    group c by c.InstituteId into g
                                    select new { g.Key };

            var theGoonies = from c in meta.Institute
                             join z in instituteContacts2 on c.Id equals z.Key into tmpICg
                            from icg in tmpICg.DefaultIfEmpty()
                            select new { InstituteId = c.Id };

            theGoonies.ToList();

[IndexOutOfRangeException: Index was outside the bounds of the array.] SD.LLBLGen.Pro.LinqSupportClasses.ValueListProjectionDefinition.AddProjectionListToProjection(ProjectionListExpression toAdd, ITemplateGroupSpecificCreator frameworkElementCreator, IElementCreatorCore generatedCodeElementCreator, MappingTracker trackedMappings) +647 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleProjectionExpression(ProjectionExpression expressionToHandle) +347 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +1058 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleGroupJoinExpression(GroupJoinExpression expressionToHandle) +153 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleGroupJoinExpression(GroupJoinExpression expressionToHandle) +35 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +646 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleAndProcessJoinExpressionSide(SetExpression side, Expression sideSelector, Expression& handledSide, Expression& handledSideSelector, String& aliasSide) +24 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleJoinExpression(JoinExpression expressionToHandle) +884 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +769 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.QueryExpressionBuilder.HandleExpression(Expression expressionToHandle) +187 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.HandleExpressionTree(Expression expression) +750 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.Execute(Expression expression) +13 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.System.Linq.IQueryProvider.Execute(Expression expression) +17 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProQuery1.Execute() +16 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProQuery1.System.Collections.Generic.IEnumerable<T>.GetEnumerator() +16 System.Collections.Generic.List1..ctor(IEnumerable1 collection) +7667686 System.Linq.Enumerable.ToList(IEnumerable1 source) +61 IAEC.SMS.DomainServices.InstituteService.List() in D:\Projects\IAEC\Main\Source\IAEC Student Management System v1.0\app\IAEC.SMS.DomainServices\InstituteService.cs:40 IAEC.SMS.Web.Controllers.InstituteController.InstituteList(Int32 pageSize, Int32 pageIndex, String sortBy, Boolean sortDirection) in D:\Projects\IAEC\Main\Source\IAEC Student Management System v1.0\app\IAEC.SMS.Web.Controllers\InstituteController.cs:50 lambda_method(ExecutionScope , ControllerBase , Object[] ) +233 System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) +17 System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary2 parameters) +178 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary2 parameters) +24 System.Web.Mvc.&lt;&gt;c__DisplayClassa.&lt;InvokeActionMethodWithFilters&gt;b__7() +52 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodFilter(IActionFilter filter, ActionExecutingContext preContext, Func1 continuation) +254 System.Web.Mvc.<>c__DisplayClassc.<InvokeActionMethodWithFilters>b__9() +19 System.Web.Mvc.ControllerActionInvoker.InvokeActionMethodWithFilters(ControllerContext controllerContext, IList1 filters, ActionDescriptor actionDescriptor, IDictionary2 parameters) +192 System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName) +399 System.Web.Mvc.Controller.ExecuteCore() +126 System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext) +27 System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext) +7 System.Web.Mvc.MvcHandler.ProcessRequest(HttpContextBase httpContext) +151 System.Web.Mvc.MvcHandler.ProcessRequest(HttpContext httpContext) +57 System.Web.Mvc.MvcHandler.System.Web.IHttpHandler.ProcessRequest(HttpContext httpContext) +7 System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +181 System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +75

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39866
Joined: 17-Aug-2003
# Posted on: 10-Feb-2010 15:37:24   

Ok, will try to reproduce it AND will see if I can find a fix for it. DefaultIfEmpty()... my best friend. wink

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39866
Joined: 17-Aug-2003
# Posted on: 10-Feb-2010 17:05:19   

My query works:


LinqMetaData metaData = new LinqMetaData(adapter);
var customerIdsWithOrders = from o in metaData.Order
                            group o by o.CustomerId into g
                            select new { g.Key };

var q = from c in metaData.Customer
        join o in customerIdsWithOrders on c.CustomerId equals o.Key into co
        from x in co.DefaultIfEmpty()
        select new { Id = c.CustomerId };

foreach(var v in q)
{
    
}

It gives a proper query.


SELECT [LPA_L2].[CustomerID] AS [Id] 
FROM ( 
    (   SELECT [LPA_L4].[CustomerId] AS [Key] 
        FROM (
            SELECT [LPLA_2].[CustomerID] AS [CustomerId] 
            FROM [Northwind].[dbo].[Orders] [LPLA_2]  
            GROUP BY [LPLA_2].[CustomerID]
        ) [LPA_L4]
    ) [LPA_L1]  RIGHT JOIN [Northwind].[dbo].[Customers] [LPA_L2]  
    ON  [LPA_L2].[CustomerID] = [LPA_L1].[Key])

So, I'll attach the latest internal build to see if this works for you.

Frans Bouma | Lead developer LLBLGen Pro
worldspawn avatar
worldspawn
User
Posts: 321
Joined: 26-Aug-2006
# Posted on: 11-Feb-2010 00:05:10   

I remembered I have a lib folder with all 3rd party dlls in it and i reference them from there so i wasn't actually using the latest dlls :|

However after correcting that and copying the provided dll in nothing has changed. I've double checked and triple checked that the correct files are in the bin folder. Still I am getting the same error. cry

Is it possible a different runtime dll update is needed? ORM perhaps...?

Or could you suggest a different way to implement the same query (without prefetches)?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39866
Joined: 17-Aug-2003
# Posted on: 11-Feb-2010 09:43:56   

worldspawn wrote:

I remembered I have a lib folder with all 3rd party dlls in it and i reference them from there so i wasn't actually using the latest dlls :|

However after correcting that and copying the provided dll in nothing has changed. I've double checked and triple checked that the correct files are in the bin folder. Still I am getting the same error. cry

Is it possible a different runtime dll update is needed? ORM perhaps...?

Or could you suggest a different way to implement the same query (without prefetches)?

The group by could perhaps also be done with 'Distinct' but only the repro case query. The original query has an aggregate so you need the group by, no I don't know a different way to do this. The problem though is that unless we have a repro case query, it's not possible to fix anything. If you could reproduce this on adventureworks or northwind in the smallest possible way, please do so, so we can look into this. The error occurs inside the linq provider so it's not the ormsupportclasses

Frans Bouma | Lead developer LLBLGen Pro
worldspawn avatar
worldspawn
User
Posts: 321
Joined: 26-Aug-2006
# Posted on: 12-Feb-2010 05:07:22   

Hi Frans,

I tried to reproduce in AdventureWorks but couldn't. So I went looking for differences and the only one I could find is that my 'InstituteContact' table inherits from another table ('Address'). Sorry I forgot that critical piece of info... I wrote this DB 3 years ago.

I modified the AdventureWorks db to make StoreContact inherit from a new entity i called BaseTable and altered the LLBL project accordingly... sadly this still hasn't reproduced the error frowning

Could I just send you my DB and a console reproduction against that DB?

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39866
Joined: 17-Aug-2003
# Posted on: 12-Feb-2010 10:26:59   

worldspawn wrote:

Hi Frans,

I tried to reproduce in AdventureWorks but couldn't. So I went looking for differences and the only one I could find is that my 'InstituteContact' table inherits from another table ('Address'). Sorry I forgot that critical piece of info... I wrote this DB 3 years ago.

I modified the AdventureWorks db to make StoreContact inherit from a new entity i called BaseTable and altered the LLBL project accordingly... sadly this still hasn't reproduced the error frowning

Could I just send you my DB and a console reproduction against that DB?

Sure, send it to support AT llblgen DOT com. Make sure the db and repro app are usable. The db can be supplied in a script (just give the tables included) and supply the lgp file, as the crash happens when the query is generated so there's no data involved anyway. We'll then look into it. It will be monday though before we'll look into this.

Frans Bouma | Lead developer LLBLGen Pro