Is this kind of join possible?

Posts   
1  /  2
 
    
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 02-Jun-2008 23:35:07   

I was very excited when I was able to have my data layer pass back IQueryables mapped to my POCO domain classes and manipulate them, so I was hoping I could extend that to joins, but so far no dice.

For example, this works just fine and is very very cool:

public IQueryable<Category> GetCategories() { var db = new LinqMetaData(adapter); var query = from c in db.Category select new Category { Id = c.CategoryId, Name = c.Name }; return query; }

public IList<Category> GetCategoriesThatStartWithE() { var categoryQuery = from c in repo.GetCategories() where c.Name.StartsWith("E") select c;

IList<Category> categories = categoryQuery.ToList();
return categories;

}

However, when I try to join two IQueryables passed back from my data layer, I can't make it work. Here's an example of something I tried:

I have the following two methods defined in a ProductCatalogRepository class:

public IQueryable<ProductFamilyCategoryMap> GetProductFamilyCategoryMap() { var db = new LinqMetaData(adapter); var query = from m in db.ProductCategoryMap select new ProductFamilyCategoryMap { CategoryId = m.CategoryId, ProductFamilyId = m.ProductFamilyId }; return query; }

public IQueryable<ProductFamily> GetProductFamilies() { var db = new LinqMetaData(adapter); var query = from pf in db.ProductFamily select new ProductFamily { Id = pf.ProductFamilyId, Name = pf.Name }; return query; }

I'm selecting into domain objects that aren't entities.

ProductFamilyCategoryMap is a linking table to establish a many to many relationship between categories and product families.

I was hoping to be able to join those two IQueryables at my service layer something like this:

public IList<ProductFamily> GetProductFamiliesInACategory(Guid categoryId) { var productFamilyCategoryMap = repo.GetProductFamilyCategoryMap(); var productFamilies = repo.GetProductFamilies();

var joinedFamilies = from pf2 in productFamilies
                     join m2 in productFamilyCategoryMap on pf2.Id equals m2.ProductFamilyId
                     where m2.CategoryId == categoryId
                     select pf2;

return joinedFamilies.ToList();

}

Unfortunately, that example doesn't work, it fails with the following error:

ORMQueryConstructionException: No known database function mapping found for property 'CategoryId'] SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.MemberAccessEvaluator. HandleMemberIsPartOfEntityField(MemberExpression expressionToHandle, Expression memberContainer) +457 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.MemberAccessEvaluator. HandleMemberExpression(MemberExpression expressionToHandle) +162 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler.HandleExpression(Expression expressionToHandle) +2006 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleBinaryExpression(BinaryExpression expressionToHandle) +36 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleBinaryExpressionBooleanOperator(BinaryExpression expressionToHandle) +7 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleExpression(Expression expressionToHandle) +1724 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleLambdaExpression(LambdaExpression expressionToHandle) +32 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleExpression(Expression expressionToHandle) +2100 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleWhereExpression(WhereExpression expressionToHandle) +84 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleExpression(Expression expressionToHandle) +1383 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleSelectExpression(SelectExpression expressionToHandle, SelectExpression newInstance) +24 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleSelectExpression(SelectExpression expressionToHandle) +140 SD.LLBLGen.Pro.LinqSupportClasses.ExpressionHandlers.GenericExpressionHandler. HandleExpression(Expression expressionToHandle) +1245 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.HandleExpressionTree(Expression expression) +729 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.Execute(Expression expression) +10 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase. System.Linq.IQueryProvider.Execute(Expression expression) +14 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProQuery1. System.Collections.Generic.IEnumerable<T>.GetEnumerator() +36 System.Collections.Generic.List1..ctor(IEnumerable1 collection) +369 System.Linq.Enumerable.ToList(IEnumerable1 source) +54

I thought I'd simplify it by removing the where statement just to see if I could make it do anything, but that fails with this exception:

ArgumentException: An item with the same key has already been added.] System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) +48 System.Collections.Generic.Dictionary2.Insert(TKey key, TValue value, Boolean add) +2668392 SD.LLBLGen.Pro.ORMSupportClasses.DerivedTableTargetingFieldFinder..ctor(List1 knownDerivedTables, Boolean testOnFieldAliasAsWell) +342 SD.LLBLGen.Pro.ORMSupportClasses.PersistenceCore.FixupDerivedTableTargetingFields(List1 derivedTables, IEntityFieldCore[] fieldsToCheck, String selectListAlias, IPredicate filter, IRelationCollection relations, ISortExpression sorter, IGroupByCollection groupBy, IInheritanceInfoProvider infoProvider) +77 SD.LLBLGen.Pro.ORMSupportClasses.PersistenceCore.PreprocessQueryElements(IEntityFieldCore[] fields, String selectListAlias, IPredicate filter, IRelationCollection relations, ISortExpression sorter, IGroupByCollection groupByClause, IInheritanceInfoProvider infoProvider) +84 SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.CreateQueryFromElements(IEntityFields2 fieldCollectionToFetch, IRelationPredicateBucket filterBucket, Int32 maxNumberOfItemsToReturn, ISortExpression sortClauses, Boolean allowDuplicates, IGroupByCollection groupByClause, Int32 pageNumber, Int32 pageSize, IFieldPersistenceInfo[]& persistenceInfo, IRetrievalQuery& selectQuery) +186 SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.FetchProjection(List1 valueProjectors, IGeneralDataProjector projector, IEntityFields2 fields, IRelationPredicateBucket filter, Int32 maxNumberOfItemsToReturn, ISortExpression sortClauses, IGroupByCollection groupByClause, Boolean allowDuplicates, Int32 pageNumber, Int32 pageSize) +67 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProvider2.ExecuteValueListProjection(QueryExpression toExecute) +290 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.ExecuteExpression(Expression handledExpression) +172 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.Execute(Expression expression) +20 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProProviderBase.System.Linq.IQueryProvider.Execute(Expression expression) +14 SD.LLBLGen.Pro.LinqSupportClasses.LLBLGenProQuery1. System.Collections.Generic.IEnumerable<T>.GetEnumerator() +36 System.Collections.Generic.List1..ctor(IEnumerable1 collection) +369 System.Linq.Enumerable.ToList(IEnumerable1 source) +54

Is there any way to make this work? I would really, really like an ability to do this.

joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 00:01:19   

I wanted to add that my wanting to try this was inspired by something I saw Rob Conery do with LINQ to SQL, so the basic concept seems feasible, I've just had no luck translating the concept to LINQ to LLBLGen yet.

Here's the working LINQ to SQL example (more complicated because it's doing localization)

Repository Methods:

public IQueryable<ProductCategoryMap> GetProductCategoryMap() { return from pc in db.Categories_Products select new ProductCategoryMap { ProductID = pc.ProductID, CategoryID = pc.CategoryID }; }

public IQueryable<Product> GetProducts() {

var cultureDetail = from cd in db.ProductCultureDetails
                    where cd.Culture.LanguageCode == 
                    System.
                    Globalization.
                    CultureInfo.
                    CurrentUICulture.
                    TwoLetterISOLanguageName
                    select cd;


var defaultImages = from i in db.ProductImages
                    select i;

var result = from p in db.Products
             join detail in cultureDetail on p.ProductID equals detail.ProductID
             select new Product
             {
                 ID = p.ProductID,
                 Name = p.ProductName,
                 Description = detail.Description,
                 ShortDescription = detail.ShortDescription,
                 Price = detail.UnitPrice ?? p.BaseUnitPrice,
                 Manufacturer = p.Manufacturer,
                 ProductCode=p.ProductCode,
                 DefaultImagePath=(from i in defaultImages
                                       where i.ProductID==p.ProductID
                                       select i)
                                       .Take(1).SingleOrDefault().ThumbUrl
             };


return result;

}

Service Method:

public IList<Product> GetProductsByCategory(int categoryID) {

return (from p in _repository.GetProducts()
       join cp in _repository.GetProductCategoryMap() on p.ID equals cp.ProductID
       where cp.CategoryID == categoryID
       select p).ToList();

}

joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 03:33:01   

Additional information:

Using latest build: yes

no trace output before exception occurs

no fancy mappings or inheritances. Create database, point GUI at database, include all tables as entities, push generate. Adapter templates, C#, .NET 3.5 framework.

See attachment for database creation script.

Attachments
Filename File size Added on Approval
TestProject.sql 2,198 03-Jun-2008 03:33.23 Approved
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 09:58:45   

I think this was fixed a couple of days ago (not released yet) but I'll run some tests. (also with the where removed which seems a different error although it could be the same thing)

(edit) I can reproduce the error in DerivedTableTargetingFieldFinder, with:


var cq = from c in metaData.Customer
         select new { c.CustomerId, c.CompanyName };

var oq = from o in metaData.Order
         select new { o.OrderId, o.CustomerId, o.EmployeeId, o.OrderDate };

var q = from c in cq
        join o in oq on c.CustomerId equals o.CustomerId
        where o.EmployeeId == 3
        select c;

which is caused by the fact that 2 derived tables turn up to have the same alias. The queries above mimic your queries. That there are two different queryables joined together is not really important, as both are first merged into the expression tree and the whole tree is handled in 1 go. The first error you got was fixed a couple of days ago, where a compound key join didn't work.

Looking into it. I'll also make sure your example code works properly with the fix.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 12:33:17   

The derived table alias issue was caused by the fact that the right-side derived table was created with the alias of the left side (flushed ).

Once I got that figured out, I ran into the issue where the projection contained a reference to a joined set. This required that the SetReferenceExpression in the projection was replaced with a clone of the projection of the referenced set. Luckily this was already tracked in the handlers, so it was easy to obtain the projection and as there's already a cloner expression handler in the code, it was really a couple of lines of code to fix it. pfew. simple_smile

It looked like a horrific piece of code to add, but with these expression trees, it's often the case that a simple replacement of the expression into something which is already understood is much easier and fixes the same issue.

I'll now run all the unittests to see what stuff I broke and will see if I can make your code work too simple_smile

(edit). Hmm... takes longer than expected.

The main problem with all these things is that a subtree in the expression tree often has to be handled in the scope it is found in, e.g. a tree in a projection results in different things than when the same tree is in a filter lambda for example. It's a bit of a pain to find a given expression E in scope X, then handle E using generic code (which handles E in all cases), and do the right thing, thus determine that E is inside the scope of X. The problem is that E can be nested inside other expressions as well, which is still inside X, however when a new query is found, the scope of X ends...

Frans Bouma | Lead developer LLBLGen Pro
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 14:27:07   

Thanks for looking into this so quickly Frans.

Were you able to squash both of the issues? It sounds like you've at the very least made significant progress, but I wasn't sure if the "taking longer than expected" applied to squashing the bug or running the unit tests simple_smile

I ask not to rush you but, because if you did, I would like to know if there is any chance I could apply a patch to the source code that shipped with the beta and build some binaries to experiment with? I'd really like to be able to continue working on the data layer in that style to see if I run into any other fun issues.

Also, our explanation of the aliases made me remember a separate question I had: is it possible to start with a relationpredicatebucket and convert this into an IQueryable<T> that I could expose to higher layers?

I may be completely wrong, and maybe it wouldn't help with the linq provider at all, but it seems like if that could be made to work that you could situationally sidestep some complications in aliasing and other ways of building complex query graphs while maintaining the linq abstraction overall.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 14:59:15   

joliver wrote:

Thanks for looking into this so quickly Frans.

Were you able to squash both of the issues? It sounds like you've at the very least made significant progress, but I wasn't sure if the "taking longer than expected" applied to squashing the bug or running the unit tests simple_smile

Well, it's always a gamble if a fix won't break another thing. As with linq expression tree handling, if you handle an expression and replace it with the handled version for query A, it might be that the code for situation B doesn't expect the replacement and crashes. So it should be logical.

At the moment I'm almost there. The fields in the final projection still have the alias of the field they refer to which isn't correct. I hope to fix that real soon. simple_smile I could fix it by re-handling a clone of the original expression, which works for your query, but in an edge case with a groupby it doesn't, as it then tries to handle the groupby twice, injecting an expression twice into the projection (don't ask, Microsoft made a mess of some parts of the expression tree, at least the C# team did. VB.NET's compiler creates sometimes other trees, which are often easier to parse)

I ask not to rush you but, because if you did, I would like to know if there is any chance I could apply a patch to the source code that shipped with the beta and build some binaries to experiment with? I'd really like to be able to continue working on the data layer in that style to see if I run into any other fun issues.

If I fix it today, I'll release a new build.

Also, our explanation of the aliases made me remember a separate question I had: is it possible to start with a relationpredicatebucket and convert this into an IQueryable<T> that I could expose to higher layers?

No, unfortunately, that's not the case. Filters refer to other elements in the final query, and you don't know these aliases till the whole tree is parsed. Aliases are handed out at the start, to expression tree elements.

I may be completely wrong, and maybe it wouldn't help with the linq provider at all, but it seems like if that could be made to work that you could situationally sidestep some complications in aliasing and other ways of building complex query graphs while maintaining the linq abstraction overall.

The errors are often in the aliases, and it then looks like the code handling the aliases is broken. simple_smile

However, it's often caused by other things. For example, if you have a query Q with a select etc. and you refer to that by some expression, where it refers to a field in the projection of Q, it has to create a derived table field referencing the derived table created from Q, and use the table alias of Q, not the table alias of the field INSIDE Q it refers to. However, in the case of a join, the join isn't a separate query... well not always (but sometimes it is wink ) and it then has to refer to one side of the join for example. So it's often in the handling code of a given expression where things go wrong, teh aliases are just objects refering to some part of the tree, they're not really rocketscience. What has to be done carefully is the creation of new fields referring to parts of the query with the right alias. In the beginning this went wrong, now it is only in edge cases where the code doesnt' expect a given expression at that point, or it is handled too early or too late, which was then caused by code which was written using the knowledge of several months ago, and which wasn't updated always with knowledge gathered later on. (as writing a linq provider is really doing research -> try -> fail -> try again -> succeed -> move on -> try etc. simple_smile

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 15:26:22   

OK! simple_smile

One nice thing about expression trees is that the elements are immutable, so changing something is always creating a copy. simple_smile The generic traversers only need 1 override to create a new expression tree with the change you need. simple_smile

I got it fixed. I'll now see if your code also works. (the unittests succeed, but it might be your code is perhaps a different case... yes, with linq there's no end to the wicked set of cases one can run into wink )


[Test]
public void FetchOneSideOfJoinBetweenTwoSeparateQueries()
{
    using(DataAccessAdapter adapter = new DataAccessAdapter())
    {
        LinqMetaData metaData = new LinqMetaData(adapter);

        var cq = from c in metaData.Customer
                 select new { c.CustomerId, c.CompanyName };

        var oq = from o in metaData.Order
                 select new { o.OrderId, o.CustomerId, o.EmployeeId, o.OrderDate };

        var q = (from c in cq
                 join o in oq on c.CustomerId equals o.CustomerId
                 where o.EmployeeId == 3
                 select c).Distinct();

        int count = 0;
        foreach(var v in q)
        {
            count++;
        }
        Assert.AreEqual(60, count);
    }
}

[Test]
public void JoinBetweenTwoSeparateQueriesIntoNewAnonymousType()
{
    using(DataAccessAdapter adapter = new DataAccessAdapter())
    {
        LinqMetaData metaData = new LinqMetaData(adapter);

        var cq = from c in metaData.Customer
                 select new { c.CustomerId, c.CompanyName };

        var oq = from o in metaData.Order
                 select new { o.OrderId, o.CustomerId, o.EmployeeId, o.OrderDate };

        var q = from c in cq
                 join o in oq on c.CustomerId equals o.CustomerId
                 where o.EmployeeId == 3
                 select new { CustomerData = c, OrderData = o };

        int count = 0;
        foreach(var v in q)
        {
            count++;
        }
        Assert.AreEqual(115, count);        // join creates duplicates which can't be filtered out as orderid is present in final resultset.
    }
}

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 15:59:34   

Your db script doesnt have the culture info stuff. So Looking back at my unittest, it does what failed in your code, reproducing the other code snippet is a bit difficult without data + schema.

One thing to realize in your code is that you pass in the same adapter into various queries which you return. Do realize that you can't store these queries somewhere, they have to be consumed on the same thread. It's therefore better (unless you know the produced query will always be consumed by the same thread) to create a new adapter for each query produced.

The end-query will be executed on a single adapter so that's not a problem.

I'll attach the updated runtime lib + linq provider to this thread (it's binary but you could get started), and I'll release an updated build later on today.

Frans Bouma | Lead developer LLBLGen Pro
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 16:59:10   

Thanks for the new DLLs, I'm going to hook them into my project and take it for a spin shortly, I'll let you know how it goes.

I know that the queries can only be executed on the same adapter, I have a DataAccessContext deals with making sure the same adapter is handed to each repository for the lifecycle of a given web request. I need to make the implementation a little better to deal with the edge case of code I specifically do NOT want automatically enlisted in an ambient transaction, but on the whole I think it can be made to work.

Regarding the sample with localization, I didn't include scripts for that because it and that code are from a Linq to SQL project that I didn't author. However, I'll attach a backup of that database so that you can test that scenario out if you want to.

Thanks again for all your help.

joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 17:29:46   

I've tried out the new libraries and I got one test case working, but I'm still running into other areas that aren't working exactly as I'd like.

This method now passes:

    public IList<ProductFamily> GetProductFamiliesInACategory(Guid categoryId)
    {
        var allfamilies = repo.GetProductFamilies();
        var allmaps = repo.GetProductFamilyCategoryMaps();
        var families = from f in allfamilies
                       join m in allmaps on f.Id equals m.ProductFamilyId
                       select f;
        return families.ToList();
    }

These two variations do not:

  • Alternate join syntax, I could live without this if I had to

    public IList<ProductFamily> GetProductFamiliesInACategory2(Guid categoryId) { var allfamilies = repo.GetProductFamilies(); var allmaps = repo.GetProductFamilyCategoryMaps(); var families = from f in allfamilies from m in allmaps where f.Id == m.ProductFamilyId select f; return families.ToList(); }

  • Adding additional predicates to one of the joined tables, I really need this one

    public IList<ProductFamily> GetProductFamiliesInACategory3(Guid categoryId)
    {
        var allfamilies = repo.GetProductFamilies();
        var allmaps = repo.GetProductFamilyCategoryMaps();
        var families = from f in allfamilies
                       join m in allmaps on f.Id equals m.ProductFamilyId
                       where m.CategoryId == categoryId
                       select f;
        return families.ToList();
    }
    

I'll attach a zip file of the solution so you can poke around with it if you want, it's unfortunately not the simplest scenario possible to exhibit the problem (and I'm willing to simplify it if you want), but it's self contained with unit tests ready to be executed, so it might let you rapidly be able to reproduce the issue.

The test cases are in UnitTests.ProductCatalogServiceTestFixture in the unfortunately named ServiceMagicX test methods, I guess I'm stuck with that name now since I can't edit the zip file I already put up simple_smile

The other classes of note are BLL.ProductCatalogService and DAL.ProductCatalogRepository

My DataContext was in the middle of being changed, so I took it out of this solution, the repository is just hardcoded to create and use one adapter for it's lifespan, this is horribly broken, but I think it's good enough for the test cases.

EDIT: I was unable to get my zip file under 1 meg with the LLBLGen binaries included, so I deleted them. The project references will expect all of the SD.* dlls to be in the .\lib directory in the archive if you want to compile it.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 17:45:26   

joliver wrote:

I've tried out the new libraries and I got one test case working, but I'm still running into other areas that aren't working exactly as I'd like.

This method now passes:

    public IList<ProductFamily> GetProductFamiliesInACategory(Guid categoryId)
    {
        var allfamilies = repo.GetProductFamilies();
        var allmaps = repo.GetProductFamilyCategoryMaps();
        var families = from f in allfamilies
                       join m in allmaps on f.Id equals m.ProductFamilyId
                       select f;
        return families.ToList();
    }

These two variations do not:

  • Alternate join syntax, I could live without this if I had to

    public IList<ProductFamily> GetProductFamiliesInACategory2(Guid categoryId) { var allfamilies = repo.GetProductFamilies(); var allmaps = repo.GetProductFamilyCategoryMaps(); var families = from f in allfamilies from m in allmaps where f.Id == m.ProductFamilyId select f; return families.ToList(); }

With my repro case it works OK: var cq = from c in metaData.Customer select new { c.CustomerId, c.CompanyName };

var oq = from o in metaData.Order select new { o.OrderId, o.CustomerId, o.EmployeeId, o.OrderDate };

var q = (from c in cq from o in oq where c.CustomerId == o.CustomerId select c).Distinct();

Strange. I'll wait for your repro to see what's different simple_smile

  • Adding additional predicates to one of the joined tables, I really need this one

    public IList<ProductFamily> GetProductFamiliesInACategory3(Guid categoryId)
    {
        var allfamilies = repo.GetProductFamilies();
        var allmaps = repo.GetProductFamilyCategoryMaps();
        var families = from f in allfamilies
                       join m in allmaps on f.Id equals m.ProductFamilyId
                       where m.CategoryId == categoryId
                       select f;
        return families.ToList();
    }
    

That's equal to my unittest (see code in my previous post). Strange!

I'll attach a zip file of the solution so you can poke around with it if you want, it's unfortunately not the simplest scenario possible to exhibit the problem (and I'm willing to simplify it if you want), but it's self contained with unit tests ready to be executed, so it might let you rapidly be able to reproduce the issue.

The test cases are in UnitTests.ProductCatalogServiceTestFixture in the unfortunately named ServiceMagicX test methods, I guess I'm stuck with that name now since I can't edit the zip file I already put up simple_smile

The other classes of note are BLL.ProductCatalogService and DAL.ProductCatalogRepository

My DataContext was in the middle of being changed, so I took it out of this solution, the repository is just hardcoded to create and use one adapter for it's life span, this is horribly broken, but I think it's good enough for the test cases.

EDIT: I was unable to get my zip file under 1 meg with the LLBLGen binaries included, so I deleted them. The project references will expect all of the SD.* dlls to be in the .\lib directory in the archive if you want to compile it.

Please keep the queries as they are in your situation. Overly simplified stuff might require a simpler fix than the real issue.

yes, just add the sourcecode and not the binaries wink . You can also mail to support AT llblgen DOT com if you want.

Frans Bouma | Lead developer LLBLGen Pro
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 18:01:37   

I'm putting together a new zip now. While re-reading your messages though, one difference jumped out at me - you're projecting into anonymous classes whereas I'm mapping into objects where the property names may or may not exactly match the names of the fields from the data source.

Example:

select new { o.OrderId, o.CustomerId, o.EmployeeId, o.OrderDate }; versus select new Category{ Id = c.CategoryId, Name = c.Name };

joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 18:20:12   

Ok, attached to this message is a zip file.

In order to run it, you'll have to fix the broken references to the LLBLGen dlls and edit the connection string in the UnitTests.config file to point at a valid database. I've also included a backup of my database just to be thorough.

I haven't gotten to the point of putting any meaningful asserts in the tests, and I'm not expecting any specific data back yet from the database, right now it's just operating on the basis that exception = bad, no exception = good.

The five test cases of interest are in ProductCatalogServiceTestFixture, I commented out the ones that have nothing to do with the error.

All of the functionality those tests exercise live in ProductCatalogService and ProductCatalogRepository.

While I was building this, I had an idea for a workaround that I went ahead and added a test case for - unfortunately it didn't work and actually fails in a completely different way from all of the others.

public IList<ProductFamily> GetProductFamiliesInACategory5(Guid categoryId) { var allfamilies = repo.GetProductFamilies(); var allmaps = repo.GetProductFamilyCategoryMaps().Where(x => x.CategoryId == categoryId); var families = from m in allmaps from f in allfamilies select f; return families.ToList(); }

If you have any questions or there's any way I can help, let me know.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 18:23:40   

joliver wrote:

Ok, attached to this message is a zip file.

Thanks, will check it out in a minute. simple_smile

In order to run it, you'll have to fix the broken references to the LLBLGen dlls and edit the connection string in the UnitTests.config file to point at a valid database. I've also included a backup of my database just to be thorough.

I haven't gotten to the point of putting any meaningful asserts in the tests, and I'm not expecting any specific data back yet from the database, right now it's just operating on the basis that exception = bad, no exception = good.

hehe, that's always a good starter simple_smile

The five test cases of interest are in ProductCatalogServiceTestFixture, I commented out the ones that have nothing to do with the error.

All of the functionality those tests exercise live in ProductCatalogService and ProductCatalogRepository.

While I was building this, I had an idea for a workaround that I went ahead and added a test case for - unfortunately it didn't work and actually fails in a completely different way from all of the others.

public IList<ProductFamily> GetProductFamiliesInACategory5(Guid categoryId) { var allfamilies = repo.GetProductFamilies(); var allmaps = repo.GetProductFamilyCategoryMaps().Where(x => x.CategoryId == categoryId); var families = from m in allmaps from f in allfamilies select f; return families.ToList(); }

Nope, that works too:


var cq = from c in metaData.Customer
         select new { Cid= c.CustomerId, CName = c.CompanyName };

var oq = from o in metaData.Order
         select new { Oid = o.OrderId, OCId = o.CustomerId, OEid = o.EmployeeId, ODate = o.OrderDate };

var q = (from c in cq
         from o in oq 
         where c.Cid == o.OCId
         &&  o.OEid == 3
         select c).Distinct();

The only difference is that you use concrete types and I use anonymous types.

Will look into your reprocase.

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 18:35:06   

1 succeeds, 4 fail.

Looking into it.

(edit) indeed seems to be related to the real types you're using. The provider makes a mistake handling them somewhere.

Frans Bouma | Lead developer LLBLGen Pro
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 18:39:14   

Otis wrote:

1 succeeds, 4 fail.

Looking into it.

At least it's reproducible, that's a better alternative than simply being crazy simple_smile

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 19:00:51   

joliver wrote:

Otis wrote:

1 succeeds, 4 fail.

Looking into it.

At least it's reproducible, that's a better alternative than simply being crazy simple_smile

I wish that was an option for me so I could call it a day wink

4 succeed, 1 fails... almost there simple_smile

The thing is a bit tricky though.

It has to resolve this: [setreference].f.id

the setreference is a reference to the join. the 'f' is a product, so not a field or single value returned from a query. If you'd have used an anonymous type, this was detectable, however now it's just a type like any other. By doing a low-level type check to see if the type of 'f' is a single value or not, it can decide if this is a reference to a field in another query or just an object. It looks simple though, one could say: "hey, you know 'id' is also to be handled", but that's unknown when f is handled because setreference.f.id is first encountered, which then tries to handle setreference.f. This then returns the object 'id' is a property of. This is all recursively handled using a visitor so it's not as if it knows what will come when it handles 'f' in setreference.f.

The query which fails now has no references to elements in the query: SELECT [LPA_L2].[Id], [LPA_L2].[Name] FROM ( (SELECT [].[CategoryId], [].[ProductFamilyId] FROM (SELECT [LPLA_1].[CategoryId], [LPLA_1].[ProductFamilyId] FROM [TestProject].[dbo].[ProductCategoryMap] [LPLA_1] ) [LPA_L4] WHERE ( ( [LPA_L4].[CategoryId] = @CategoryId1))) [LPA_L1] CROSS JOIN (SELECT [LPLA_3].[ProductFamilyId] AS [Id], [LPLA_3].[Name] FROM [TestProject].[dbo].[ProductFamily] [LPLA_3] ) [LPA_L2] )

Wonder what that is, but it's again alias related... (as always hehe wink )

(it's test 5 btw)

Frans Bouma | Lead developer LLBLGen Pro
Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 03-Jun-2008 19:24:52   

5 passed, 0 failed, 0 skipped, took 1,77 seconds.

simple_smile

The final error was a somewhat sloppy one from my part. I'll attach a new build in a sec.

(edit) new build attached.

I'll release a new build with source etc. tomorrow.

Frans Bouma | Lead developer LLBLGen Pro
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 03-Jun-2008 21:21:44   

Otis wrote:

5 passed, 0 failed, 0 skipped, took 1,77 seconds.

simple_smile

The final error was a somewhat sloppy one from my part. I'll attach a new build in a sec.

(edit) new build attached.

I'll release a new build with source etc. tomorrow.

Thanks for the quick response as always. I'll download these dlls now and put them through the paces, and see if I can break it in any new and interesting ways, but hopefully I'll fail simple_smile

Hopefully you've called it a day by now and will shortly be getting some rest. Thanks again Frans, really appreciate it.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 04-Jun-2008 19:24:48   

I've posted the RC.

Frans Bouma | Lead developer LLBLGen Pro
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 05-Jun-2008 19:04:55   

Ouch! I hate to be the bearer of bad news, but I downloaded the RC labeled Jun 4 and tests 2-5 fail again.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 05-Jun-2008 19:08:35   

joliver wrote:

Ouch! I hate to be the bearer of bad news, but I downloaded the RC labeled Jun 4 and tests 2-5 fail again.

That's weird as they succeed here ? (the RC is the same code as I posted above in binary form)...

I'll re-test.

Frans Bouma | Lead developer LLBLGen Pro
joliver
User
Posts: 39
Joined: 15-Feb-2007
# Posted on: 05-Jun-2008 19:11:53   

Otis wrote:

weird as they succeed here ? (the RC is the same code as I posted above in binary form)...

I'll re-test.

Maybe somehow the files got mixed up? I'm testing with the files from LLBLGenPro_v2.6_RC_06042008.zip inside the RuntimeLibraries\DotNET20 and DotNET35 directories.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39872
Joined: 17-Aug-2003
# Posted on: 05-Jun-2008 19:16:03   

Indeed... flushed rage

The files are old (06032008 of the linq provider). I get 5 passing tests with my code here.

I'll upload a new RC. (will take 10 minutes or so, you can try the build attached to a previous post, these should work).

(edit) The development was done on another machine than the build stuff. So I forgot to update the svn folders before the build. Luckily the RTM installer build stuff is automated in finalbuilder, this beta/RC is a manual process...

I've updated the RC zip.

Frans Bouma | Lead developer LLBLGen Pro
1  /  2