LLBLGEN v4.2.16.311 , Adapter

Hi, I found strange behavior when using DefaultView and changing entity in its collection. DefaultView.Count shows +1 until I set filter again.

I checked DOC http://www.llblgen.com/documentation/4.2/LLBLGen%20Pro%20RTF/Using%20the%20generated%20code/Adapter/gencode_usingentityview_adapter.htm

There you write that when using DefaultView.Filter then "View behavior on collection changes" is automaticaly set as DataChangeAction = ReapplyFilterAndSorter and this way DefaultView is always updated when collection change

It seems this doesn't work in my case.

Let's look at simple example:

Dim ecTemp As New HelperClasses.EntityCollection(Of myEntity)()
adapter.FetchEntityCollection(ecTemp, Nothing)

Now I set some filter to DefaultView:

ecTemp.DefaultView.Filter = HelperClasses.LineConfig2DeviceTypeFields.LineConfigId = 23

Here say that ecTemp.DefaultView.Count is =A I checked and DefaultView.DataChangeAction=ReapplyFilterAndSorter <- thats fine

Now I update current entity in collection with the same (just fresh copy from DB) anyway in this example I use exactly same entity to make it simple Replacing original entity in collection:

'...
ecTemp(ecTemp.IndexOf(CurrentEntity)) = CurrentEntity

Here ecTemp.DefaultView.Count=A+1 ! ecTemp.DefaultView seems cointaining the same record 2x while collection ecTemp is fine with just 1 copy of that entity and ecTemp.Count now is same as in the beginning

In some cases DataBinding is fired and as Databinding is via .DefaultView sometimes bound GridView crash on out of index in CollectionCore.cs "Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index"

Why filter is not refreshed automatically now ?

Well I have to set filter again manually this strange way

ecTemp.DefaultView.Filter = ecTemp.DefaultView.Filter

and it works and ecTemp.DefaultView.Count=A

I hope you can replicate this issue

Hi Rosacek,

I can't reproduce it using RTL 4.2.16.0311.

[TestMethod]
public void ReaplyFilterOnView()
{
    int expectedTotalCont = 842;
    int expectedTotalWithCustomerAlfki = 16;

    var orders = new EntityCollection<OrderEntity>();
    using (var adapter = new DataAccessAdapter())
    {
        adapter.FetchEntityCollection(orders, null);
    }

    // inital total count test
    Assert.AreEqual(expectedTotalCont, orders.Count);

    // set the filter and test the initial count. 
    orders.DefaultView.DataChangeAction = PostCollectionChangeAction.ReapplyFilterAndSorter;
    orders.DefaultView.Filter = OrderFields.CustomerId == "ALFKI";
    Assert.AreEqual(expectedTotalWithCustomerAlfki, orders.DefaultView.Count);

    // force a change on a entity that triggers a change on the filter. 
    orders[0].CustomerId = "ALFKI";
    Assert.AreEqual(expectedTotalWithCustomerAlfki + 1, orders.DefaultView.Count);

    // back as it was
    orders[0].CustomerId = "XYZ";
    Assert.AreEqual(expectedTotalWithCustomerAlfki, orders.DefaultView.Count);
}

Do you have a reproducible code or steps we can replicate over here?

Hi, I am fighting with this another day but still cannot understand what is wrong. I tried in other apps, other collections, same problem.

' fetching collection
Dim ecTemp As New HelperClasses.EntityCollection(Of EntityClasses.CustomerEntity)()
adapter.FetchEntityCollection(ecTemp, Nothing)

' set some simple filter, here based on first entity in collection
ecTemp.DefaultView.Filter = HelperClasses.CustomerFields.RowVersion = ecTemp(0).RowVersion

' reload very first entity from DB (as filter is set based on this entity then this entity is for sure included in .DefaultView
Dim freshEntity As New EntityClasses.CustomerEntity(ecTemp(0).CustomerId)
adapter.FetchEntity(freshEntity)

' after next row I got DefaultView.Count = +1
ecTemp(0) = freshEntity

' after next row DefaultView.Count is again OK
ecTemp.DefaultView.Filter = ecTemp.DefaultView.Filter

See http://screencast.com/t/73T297Mxu

When I set ecTemp.DefaultView.Filter then bunch of code is fired: EntityViewBase.cs / protected virtual void SetFilter( IPredicate filter) from there CollectionCore.cs / public virtual List<int> FindMatches( IPredicate filter ) and there is build List of index for filtered entities

Maybe I am wrong, but the same should be fired right after:

ecTemp(0) = freshEntity

because DefaultView should reapply filter.

But after

ecTemp(0) = freshEntity

I get into EntityViewBase.cs / private void _relatedCollectionListChanged( object sender, ListChangedEventArgs e )

Where e.ListChangedType is ItemAdded (in fact I did not add anything, rather replacing and ecTemp.Count keeps same figure!

From there EntityViewBase.cs / private void HandleRelatedCollectionItemAdded isInsert value =True isNewIndex value =True .. case PostCollectionChangeAction.ReapplyFilterAndSorter

EDIT, this screencast may help: http://screencast.com/t/wXkVbBz2rveA

Reproduced using:

private void TestDefaultView()
{
    var customers = new EntityCollection<CustomerEntity>();
    var customers2 = new EntityCollection<CustomerEntity>();

    using (var adapter = new DataAccessAdapter())
    {
        adapter.FetchEntityCollection(customers, new RelationPredicateBucket(CustomerFields.Country == "UK"), 7, new SortExpression(CustomerFields.CustomerId | SortOperator.Ascending));

        customers.DefaultView.Filter = new PredicateExpression(CustomerFields.City == "London");
        var x = customers.DefaultView.Count;
        customers.DefaultView.RaisesItemChangedEvents = true;

        adapter.FetchEntityCollection(customers2, new RelationPredicateBucket(CustomerFields.Country == "UK"), 1, new SortExpression(CustomerFields.CustomerId | SortOperator.Ascending));

        if (customers[0].CustomerId == customers2[0].CustomerId)
        {                   
            customers[0] = customers2[0];

            x = customers.DefaultView.Count;
        }
    }
}

Only when using a filter, the DefaultView.Count is incremented, otherwise it stays the same.

Hmm... The root cause is in the collection.

When you do collection[index] = someNewEntity, it issues a ListChanged event with argument AddItem. This is the only option, as ListChanged doesn't have a value which signals the item was 'replaced'.

It then ends up in the view, which checks whether the # of indices it has is lower than the # of items in the collection. This is the check which fails (as the numbers stayed the same), and therefore nothing is reapplied.

But the real issue is that if you do: collection[index] = someEntity, the entity which was already at that spot isn't properly removed first. This is done to avoid a huge performance penalty as we know we already will insert a new element at the same spot (as RemoveAt(x) moves elements in an O(n) fashion), however we do have to signal the old element was removed. This isn't done.

If it would be done, the view would first remove the old element, reapply the filter, then the new element is added, and it would reapply the filter again. Also, the entity which was replaced should be added to a removal tracking collection which isn't done now either.

Strange that no-one ever ran into this (as it's there for over a decade!). I'll check what bugfix to apply so it gets properly solved and as a bonus your issue is then fixed too.

Hi Otis,

the view would first remove the old element, reapply the filter, then the new element is added

First, collection.count keeps the same value, just DefaultView.Count is incremented and call to FindMatches may fix that. Don't get why you need RemoveAt.

I think would be nice to not fire ListChanged twice (both for remove and add) as that is just 1 operation (like transaction) in fact

Also, the entity which was replaced should be added to a removal tracking collection which isn't done now either

If PK for old and new entity is same, then removal tracking is not necessary, better to say it is bad idea. Because if someone uses removal tracking can blindly DELETE from DB all "removed" entities, INCLUDING those entities which were just replaced ( e.g. by fresh entity reload from DB) in collection and which were not intented to be deleted from DB!!!

In fact I still doubt if this case is "pure" Remove+Add. I could get the same resultř if I clone just fields form new entity into old entity, no Remove+no Add, but result at the end is same.

Btw I hope when fields are changed by code , then DefaultView.Filter is reaplied correctly, no bug there, or?

Strange that no-one ever ran into this (as it's there for over a decade!)

Was strange to me also. Took almost three days to isolate the problem and found that is not my bug and really looks like ancient bug in LLBL, nobody found till now.

In my app I created Master/Child. When master row changes then I set DefaultView.Filter to child collection based on foreign key (PK in master collection)... I did some quick UI tests in my app, everything was fine. But two days after I could see on some records exception "index out of range..". That came from Devex GridView, looked crazy. I checked changes in code, database inconsistency etc.

Then I found bindingsource CurrentChanged event fires GridView internal code like sorting etc ... and it crashed inside Devex code. Finally I found that sometimes .Count is +1, which was not possible... and finally found the problem when replacing entity in collection while having Filter on DefaultView

Workaround was simple, use GridView.BeginDataUpdate to freeze grid refresh code. This way even DefaultView.Count shows incorrect value for +1, grid is not refreshed, then reapply filter to DefaultView manually. And finally unlock GridView.EndDataUpdate. At this moment DefaultView was ok and no exception was raised.

Anyway I tried to get the true reason, so I spent huge time to isolate code to avoid my custom code, side effects etc.

This year third bug in LLBL found, if I keep this speed, you will add my name to LLBL testers Don't know why but I found already this year several serious bugs in other tools as well. Unlike the others, you fix any bug within hours or days, great!

Hmm reading your posts again, it seems you see the entity being added, while it shouldn't be. This is very odd though, as the code won't see the index as a new index so nothing really should happen...

The issue we identified in my previous post isn't directly related.

Looking into it.

Rosacek wrote:

Hi Otis,

the view would first remove the old element, reapply the filter, then the new element is added

First, collection.count keeps the same value, just DefaultView.Count is incremented and call to FindMatches may fix that. Don't get why you need RemoveAt.

I think would be nice to not fire ListChanged twice (both for remove and add) as that is just 1 operation (like transaction) in fact

Also, the entity which was replaced should be added to a removal tracking collection which isn't done now either

If PK for old and new entity is same, then removal tracking is not necessary, better to say it is bad idea. Because if someone uses removal tracking can blindly DELETE from DB all "removed" entities, INCLUDING those entities which were just replaced ( e.g. by fresh entity reload from DB) in collection and which were not intented to be deleted from DB!!!

In fact I still doubt if this case is "pure" Remove+Add. I could get the same resultř if I clone just fields form new entity into old entity, no Remove+no Add, but result at the end is same.

Hmm, that's a good point. We'll roll back the change. It's something that's rather obscure but indeed replacing the object at an index removes indirectly the element already there but it could be that was just to add it somewhere else later on. We'll revert the changes.

Btw I hope when fields are changed by code , then DefaultView.Filter is reaplied correctly, no bug there, or?

Yes that's still working and not related to this issue. The Item(i) property of the collection issues an ListChanged.ItemAdded and the view anticipates on this poorly, as the existing element which was replaced was in the indices.

We'll look into issuing an ListChanged.ItemChanged event from the collection (to the view), so the view will anticipate on it differently. This is actually the best option I think as it comes down to replacing all fields in the entity object at index I to all values of a different entity object.

Strange that no-one ever ran into this (as it's there for over a decade!)

Was strange to me also. Took almost three days to isolate the problem and found that is not my bug and really looks like ancient bug in LLBL, nobody found till now.

Sorry about that

In my app I created Master/Child. When master row changes then I set DefaultView.Filter to child collection based on foreign key (PK in master collection)... I did some quick UI tests in my app, everything was fine. But two days after I could see on some records exception "index out of range..". That came from Devex GridView, looked crazy. I checked changes in code, database inconsistency etc.

Then I found bindingsource CurrentChanged event fires GridView internal code like sorting etc ... and it crashed inside Devex code. Finally I found that sometimes .Count is +1, which was not possible... and finally found the problem when replacing entity in collection while having Filter on DefaultView

Workaround was simple, use GridView.BeginDataUpdate to freeze grid refresh code. This way even DefaultView.Count shows incorrect value for +1, grid is not refreshed, then reapply filter to DefaultView manually. And finally unlock GridView.EndDataUpdate. At this moment DefaultView was ok and no exception was raised.

Anyway I tried to get the true reason, so I spent huge time to isolate code to avoid my custom code, side effects etc.

This year third bug in LLBL found, if I keep this speed, you will add my name to LLBL testers Don't know why but I found already this year several serious bugs in other tools as well. Unlike the others, you fix any bug within hours or days, great!

Sorry for the issues, I really hate errors in my code so I try to fix them a.s.a.p. as that's what I want the software vendors I rely on to do as well (but sadly no-one does indeed ).

These UI binding issues are indeed hard to track down, .NET UI frameworks also don't help as WPF/Winforms swallow a lot of exceptions as well so they go unnoticed.

I have a clear overview of what to do now so it should be solved soon.

It appears to me that LLBL thinks ItemAdded Then "added" entity is tested if matches filter - YES - then it is added to DefaultView List(of index). There for Colection.Count is corect (keeps same) But DefaultView.Count changes +1 )its LIST (of indexes containf duplicated index for replaced entity)

After manual reapply myColection.DefaultView.Filter=myColection.DefaultView.Filter then Setfilter is reaised it calls to FindMatches, and DefaultView List (of indexes) is rebuild from scratch) and after that DefaultView List(of indexes) is correct, and therefore DefaultView.Count show correct fugure. That is seen in screencast I attached in previou posts

Anyway I did no study your code deeply, maybe I am wrong, just hint from me

It indeed issues an ItemAdded, and the view checks whether this is an insert or an append. It decides it's an insert and happily inserts the index in the list of indices.

I lean to change it to an ItemChanged, as that's better covering what happened. With an ItemChanged issued, everything works as expected. If we keep the ItemAdded we need to add code to the ItemAdded handler, which basically comes down to verifying whether it's actually an itemchanged

Working on fix.

I went thru this post again.

If in ItemChange PKold=PKnew then it is just entity update... and looks you know what to do to fix it

If PKold<>PKnew then it is RemoveAt +InsertAt and then removal tracking etc should be used as you pointed... also I think you know what to check/fix

Thx

Rosacek wrote:

I went thru this post again.

If in ItemChange PKold=PKnew then it is just entity update... and looks you know what to do to fix it

Yes, we'll change it to an ItemChanged, as in teh ItemAdded handler in the view, we can't determine whether it's an insert at index I or a replace on index I (they'll look the same). So with ItemChanged issued from the this[index] setter your issue is fixed.

If PKold<>PKnew then it is RemoveAt +InsertAt and then removal tracking etc should be used as you pointed... also I think you know what to check/fix

Thx

Yes correct. Though we decided not to correct this in v4.2 as it's a breaking change. Technically changing the event is also a breaking change but it's between view and collection and that's not a problem.

I have a debug build attached to this post which should fix your issue with count / indices.

(edit) we'll think about changing it for v5.0, but that's not certain. We won't change the removal tracker issue for v4.2. Your issue is fixed though.

Do I need to upgrade first to the latest build or replace attached DLL in LLBLGEN v4.2.16.311 is ok ?

Rosacek wrote:

Do I need to upgrade first to the latest build or replace attached DLL in LLBLGEN v4.2.16.311 is ok ?

Replacing the dll is OK, it should work immediately.

We merged the other change into the v5 codebase (the breaking change regarding removal tracker): if the entity is equal (pk equal) then the removal tracker won't track it, otherwise it will.

Works, fixed in the speed of light

If in ItemChange PKold=PKnew then it is just entity update... and looks you know what to do to fix it

I think of the case when I replace .IsNew entity by another .IsNew entity In this edge case, both PK=0, but hard to say if that is EntityUpdate or EntityRemoveAt+EntityInsertAt

I would vote for EntityRemoveAt+EntityInsertAt for new entity being replace by another new entity, even PKs are equal

That's covered It will perform a Contains(e) anyway, which uses .Equals(), and which checks the PK if the entity isn't new, and otherwise it will do a field value compare. So that should be OK

Glad your problem is solved!