Do I have to have redundant lists of fields for my FetchEntity calls and my linq queries, or is there a way to use an IncludeFieldsList with IncludeFields()?

Also, how do I manage fields for a query that I've done a "group by" on?

tmatelich wrote:

Do I have to have redundant lists of fields for my FetchEntity calls and my linq queries, or is there a way to use an IncludeFieldsList with IncludeFields()?

AFAIK, you only can use lambda expressions

tmatelich wrote:

Also, how do I manage fields for a query that I've done a "group by" on?

Could you please illustrate that with an example?

var query = from foo in metaData.Foo where foo.bar == 9 group foo by foo.baz into g select g;

putting a .IncludeFields results in a compiler error due to trying to apply the lambda to an IGrouping.

Mostly, this is moot, unless someone else comes along with a similar question. I've determined that the number of fields I'm including along with the number of reports I expect to get makes field manipulation performance-detrimental.

IncludeFields() is used when fetching entities/collection. When Grouping you actually specify the list of fields to return, so using IncludeFields() has no meaning in this case.

That is not accurate. The grouping contains the actual entity, as written.

foreach(var g in query) { Console.WriteLine(g.Key); foreach(FooEntity f in g) { } }

Selecting specific fields would either result in a local data type, or the creation of a new entity internally.

The linq query which returns 'g' (thus the grouping result) returns the result of a merge of two queries, the grouped data (the keys) and the entities related to the grouped keys. So indeed, 'g' is an enumerable over entities, but to get that, the runtime fetches two queries and merges them (as it's a hierarchical set).

The excluded fields are specified with an extension method which expects the fields to be excluded to be part of the TSource type of the query. Your IQueryable<TSource> has as TSource an IGrouping<T, U>, where U is the entity type and T is the keytype. So this clashes. This is caused by the fact that IGrouping is a hierarchical set, not a flat enumerable: per key you have an enumerable, not just 1 enumerable.

That's why you can't specify the excludingfields on these queries, even though it might be ok to do so in theory. There's no code in the runtime at this point to support this.

I just tried to add an extension method which uses an IGrouping<T, U> but that failed deep inside the core where it couldn't create a correlation filter between key query and entity query. Avoiding to waste time on figuring out why that was, I tried the obvious: you can add the excluding fields call to the original source ->

[Test]
public void GetAllEmployeesWithoutPhotoNorNoteFields2()
{
    using(DataAccessAdapter adapter = new DataAccessAdapter())
    {
        LinqMetaData metaData = new LinqMetaData(adapter);
        var q = (from e in metaData.Employee.ExcludeFields(x => x.Photo, x => x.Notes)
                 group e by e.ReportsTo into g
                 select g);

        int count = 0;
        foreach(var g in q)
        {
            count++;
            foreach(var v in g)
            {
                Assert.IsTrue(v.Photo.Length <= 0);
                Assert.IsTrue(v.Notes.Length <= 0);
            }
        }
        Assert.AreEqual(6, count);
    }
}

here I group on reports to and exclude photo and notes from the entities.

Clever

Thanks.

One other question before I delete my include fields list. Are there any heuristics available to determine when Include/Exclude Fields will yield a performance gain, or is it just measuring?

tmatelich wrote:

Clever Thanks. One other question before I delete my include fields list. Are there any heuristics available to determine when Include/Exclude Fields will yield a performance gain, or is it just measuring?

It's about the limitation of the amount of data transported over the wire. So excluding a couple of int fields in 10 rows won't make a difference. However excluding a field which contains large datablocks (e.g. 10MB files) will make a difference as per row you will transport much less data over the wire.

So if you fetch a lot of rows (thousands) and you exclude half of the fields, you will get higher performance, but it depends on how big the data is that you don't fetch how big the advantage will be.