Fields.Clone/ShallowClone

Posts   
 
    
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 24-Feb-2013 10:43:45   

There doesn't seem to be anything in the documentation about exactly what Fields.Clone/ShallowClone do.

The intellisense for both is also not very explanatory but I think that Clone() creates a completely independent Fields/set of Field instances whereas ShallowClone creates a new Fields instance but it still points to the original set of Field instances.

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 25-Feb-2013 03:34:18   

Fields.ShallowClone performs simply this:

(EntityFieldsCore<TField>)this.MemberwiseClone()...

... so it will create a new instance of each field without keep reference to the referenced object (if any). It also will copy the value of all non-static members. Ref...

Fields.Clone is more complete because it does a ShallowClone plus: - Clone the internals, which will pass all the dynamic info of the field. Useful if you have a field with complex expressions, dyn queries, etc. - It will correct the supertype/subtype linked fields.

David Elizondo | LLBLGen Support Team
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 25-Feb-2013 14:54:56   
... so it will create a new instance of each field without keep reference to the referenced object (if any). It also will copy the value of all non-static members.

Not sure what this means.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39910
Joined: 17-Aug-2003
# Posted on: 25-Feb-2013 15:44:20   

Fields never point to the original fields object (at least not in v3). The shallow/normal clone difference is in the re-use of bookkeeping data: shallowclone shares bookkeeping data (indexing data) and clone() does't. This is not really an issue with fields objects in entities, as they all share the same bookkeeping data instance till you add a field manually: then it creates a copy locally to store the additional data. Clone() always creates a clone of that, shallowcopy doesn't.

In v4, we made the distinction between 'entity fields object in entity' vs. 'entity fields object not associated with an entity instance'. Cloning the former comes down to cloning the value storage, and the field info storage. Cloning the latter comes down to cloning each field as well, though expressions are shared.

In general I'd advice against cloning entity fields objects, unless you have to in an edge case. It's far easier to simply re-create an instance of the entity fields object through the factory. Fields objects (so EntityFields2) have a couple of purposes: 1) data containers for entities 2) projection definition for queries 3) low-level dto for insert/update queries.

1 and 3 are related, and not a concern for the user, as the user works with the entity and its properties, not entityfields objects themselves. 2 is for query definition purpose, but then it doesn't contain data.

What is the purpose of the clone, exactly?

Frans Bouma | Lead developer LLBLGen Pro
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 27-Feb-2013 16:40:27   

The Clone is for an IncomeEntity (income as in salary etc.) There is an intermediate m:n table called IncomeToOwner with the other side being Clients.

I need to be able to Edit an IncomeEntity plus its IncomeToOwners collection in a single dialog form.

If the user clicks cancel, then I just throw everything away. If they Accept, then I replace the Fields in the original entity with that of the cloned entity and 'merge' the IncomeToOwners back reusing existing IncomeToOwners where possible.

I am sure I could use SaveFields etc. but the problem is with the intermediate table and trying to restore that to its original state if the user cancels. It seemed easier to just Clone Income plus any existing IncomeToOwners so they can be thrown away or integrated.

This is complicated further by the fact that the oroginal IncomeEntity is also displayed in another grid and I didn't want changes to be seen there until finalised.

Also there are currently four different entities following this pattern so I use generics where I can.

Walaa avatar
Walaa
Support Team
Posts: 14995
Joined: 21-Aug-2005
# Posted on: 27-Feb-2013 18:40:01   

Personally I'd recommend of fethcing a new instance of the entity when going into the details form, where you edit it. This will keep it separate from the Main/original form. When commiting the changes you can refetch the Main Page again, or pass the modified entity back to the main page to replace the original one.

simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 15-Mar-2013 11:20:52   

What I described above worked on the relative simple level of one root entity plus one intermediate collection plus another collection from that.

However, we are now implementing screens which go a bit deeper than that so I would to reopen this thread on a more general level.

Using in-memory serialization, it is easy to create a clone of a whole entity graph. That clone graph is then passed to a dialogue form and the user can edit as required. If the user clicks Cancel, then the whole clone graph is thrown away which is great.

When the user accepts the change, the problem is how to merge that cloned graph back into the original graph in the most efficient way.

What me and my business partner need to write is a simple EntityGraphMerge class. I am pretty sure there is nothing in the LLBLGen framework currently that does exactly what we need. We've also looked in the forums and whilst there was talk of implementing something like this back in 2009, there is nothing concrete.

We are prepared to write, document and post the result up here for anyone who would find it useful but we would like your (urgent) help to identify the metadata and other items to make it a generic, standalone, works with everything, one-off, comprehensive, safe and efficient utility class.

Things to bear in mind:- - Entity Field versioning is not really of use here since it doesn't take into account child collections and the like - much easier to replace the original entity fields with that of its clone and fire a PropertyChanged(null). - Versioning in general is not required, the user will either accept all the changes they made to the clone or none of them. So no need to keep track of incremental changes - Removal of entities from collections. I described the EntityGraphAuditor I use in another thread. It will maintain a central list of entities which were removed from collections for the purposes of deletion, so we shouldn't have a problem identifying removed entities. - The original graph is involved in complex databinding (over many controls and binding sources), therefore the changes made to make it look like the accepted clone must be absolutely minimal. (We had thought about using the cloned graph as a direct replacement of the original but it is not as simple as just setting the root object as the datasource on a single BindingSource in our case) - Refetching entities is not an option. - I'll be honest, I don't understand AdjacencyLists and Topology sorts flushed , so I've no idea whether ObjectGraphUtils will help or not. Since these are not general graphs but structurally related, I think it should be straightforward to use the metadata to go traverse down from the root entity: entities get their fields merged/replaced, child owned collections are compared and adjusted as appropriate - then repeat for their contents. - No need for checking code in the merger, other than maybe checking that the root entities are the same type and ID.

I see it working as simply as this:-

var merger = new EntityGraphMerger();
merger.Merge(originalRootItem, clonedRootItem);

where clonedRootItem was created from originalRootItem and the root item is either a single entity or an EntityCollection<T>. We need the single entity root item as an entity initially but, like the serialization stuff, I think the internal code will not change much to accommodate an entity collection as a root item.

Can you help us (me and Ken Hayden that is) to write this class?

Cheers Simon

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 18-Mar-2013 05:27:22   

Hi Simon,

IMHO, it looks like it would be easier to just take the cloned graph and replace the original one. You said you considered and it looks like a lot of effort, but honestly I think the merger utility class wont be that trivial either.

But sure, we could make suggestions and help you on the process. What do you have so far?

BTW, Had you see this thread?: http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=16462&StartAtMessage=0 as I think there is a lot of discussion/suggestions about the topic there.

David Elizondo | LLBLGen Support Team
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 18-Mar-2013 11:26:59   

Just spent half an hour replying then accidentally hit Ctrl-W and lost the lot! disappointed

Replacing the original cloned graph is a possibility but the screen I have in mind is the most complicated I have every written (16 subscreens all combined on the fly depending on the contents of the entity graph).

I did read the thread you mentioned. What rdhatch described is more or less what I want to achieve (although I ain't paying for it smile ).

My colleague Ken, who is using LLBLGen for the first time, asked me how people work without being able to compare/merge graphs for a parent/child scenario and I didn't really have an answer.

That thread sounded really positive - lots of idea, Frans very interested etc. but then it seemed to fizzle out to nothing. As far as I am aware Context does not do anything new for merging and Algorithmia is an Undo/Redo command-based tool but is incompatible with the reference-based graphs used in the runtime. Also, the thread here http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=16460 seemed to indicate this feature was coming but nothing mentioned since.

I don't think a diffing/merging engine will be trivial to write but it should be doable and once done, will work for all LLBLGEN entity graphs and become a very useful feature.

I haven't done anything as yet but I see two ways of doing it. 1) Since the Clone graph is derived from the Original graph which has a single root entity in common, it should be possible to traverse from this point down to find 'equivalent' entities/collections to compare. 2) Another way is to use a ReferenceEntityMap on each graph to find all entities in each and find their 'equivalents' that way. Each graph would need an associated DeletedEntityTracker collection and comparing the differences between those would reveal what was deleted.

I like rdhatches ldea of using two separate steps to 'identify differences' and then 'apply the differences' back. This is preferable to just 'merging' in one step since that way would require some additional comparer code to test that the merger code actually worked. using two steps means a second pass should give an empty list of changes.

I think that the thread you mentioned overcomplicated things a little with regard to branching and merging and n+2 versions etc. I don't think there is a real need for opening two clones at the same time. Just have one clone at a time and either discard it or merge it.

Other thoughts:

Field differences. I have mixed thoughts on this. On one hand, the clone could be treated as disposable once the decision is made to merge it back into the original and the entire Fields object in the Original entity could be replaced with its equivalent from the Cloned equivalent (if dirty). On the other hand, we have some entities with virtual properties that use PropertyChanged to update themselves when the property/properties they depend on change. Maybe a field-by field change is best.

How equal is equal? If a child entity collection has x entities, is it enough to ensure that the collections are equal when all entities are present in any order or do they also need to be in exactly the same order? It shouldn't matter if an EntityView is being used to apply some order especially if bound to a grid. On the other hand, I have disagreed in the past with Fran's definitive of EntityCollection. Sure it is called a Collection and in the database the entities have no specific order but the fact is, on the .NET side, it really is a List not a Collection and that implies an Order. I think there should at least be an option to be able to adjust the Original-side collection's content to match that of the Clone even if it is off by default.

Object OIDs. I can't remember what these are used for? Are they relevant for non-remoting use?

Change notification. Whatever changes are made, ListChanged/PropertyChanged notification should be minimized as far as possible. A List.Reset/PropertyChanged(null) for each changed item is preferable wherever possible.

Context. How should this be used, if at all?

Cheers Simon

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39910
Joined: 17-Aug-2003
# Posted on: 18-Mar-2013 13:57:36   

Did you look at RDHatch's code? https://github.com/GeniusCode Not sure if it's sufficient or not, but graph versioning is not a small thing. To enable it inside the runtime, all field values + collections + 1:1 referenced entities have to be made versioning aware (so have to store the 'old' version somewhere). I wouldn't go that route, it's not worth the time.

I'm not sure whether you try to do a lot of work in-memory which you could avoid by simply persisting to the DB whatever you have in-memory. If another version comes along, so be it, alter what's in the DB based on that version. After all, that's in essence what you want to do in memory as well.

If cascading deletes/updates need to be performed, you could look into v4's datascopes, which track all changes (additions, changes in entity, removals) and allow you with simple code to build cascade deletes as well (simply supply the relationship for data not in-scope, that's it), it builds the UoW for you which you can commit in 1 go. Don't over-engineer these kind of things, it's not worth it: the more you try to do in-memory, the more you'll be writing a full in-memory object database, as you have to solve the same problems, however you already use a db. Data outside the DB is stale, having 2 versions of it, is always going to become a concurrency problem, and as there's no 'best way' to deal with those simply pick a strategy for that, and update the data in the DB.

Frans Bouma | Lead developer LLBLGen Pro
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 18-Mar-2013 15:45:07   

Just had a look at RDHatch's github source - can't see anything there that was mentioned in the other thread unfortunately.

Graph versioning is not what I need - graph diffing is. There should no need to change anything in the runtime, no need to listen to changes as they happen etc.

I've read the rest of your reply and it doesn't seem to relate to what I am looking to do at all. Persisting to the database is not an option. This is purely an in-memory thing. v4 datascopes seem to be what you were advising against namely tracking all changes.

The concept is very simple.

1) I have a single root entity, PrivateClientFile in my case, which the user is going to edit in a (gui) View. With the prefetches, there are 114 entities in the graph. At any time, the user can choose Cancel, in which all changes are discarded and the PrivateClientFile data is reloaded, or they can choose Save and, assuming everything passes validation, it is persisted using UnitOfWork. This all works absolutely fine.

Because this data is large, it is split over 16 subViews, some of which are grids. If the user wants to edit something in one of the Grids, they double-click on the row and a popup dialog appears and allows the user to edit a part of the entity graph. It may be a single entity or a single entity plus child data. Either way it shouldn't matter, it is still a subset of the original graph.

2) When the user has this dialogue form open, they may choose OK or Cancel. Cancel will ignore all the changes made in that dialogue. OK needs to accept those changes back into the original data graph.

Using the original graph on the dialog can cause a lot of problems. If a user clicks Cancel on the Edit dialog, rereading from the database is not correct because it will also remove any changes they made elsewhere in the graph. Therefore the data on the dialog edit form must be kept 100% separate from that on the main View until it is accepted. We want to clone the original graph to make this possible. Even though just a subset of the original graph will be modified, we are happy to clone the whole graph because it is only a few lines of code to do so. Having a clone graph makes Cancelling easy, the clone is just thrown away. Merging the data back is the issue.

How the changes on the dialog edit form were made and in what order is not relevant so there is no need to track changes. At the point the user clicks OK, the original graph needs to be modified to reflect those changes and that is all. Once this is done, the original graph and the clone graph should be identical. The clone is then thrown away.

Whilst it is possible to use the clone graph as a 'replacement' version for the original, this is not ideal since the original graph is bound over many different views - much better to make the fewest modifications directly within the original graph so only the one view whose data has been modified will update.

I've made a start on this, here is a sample of the test code where originalEntity is the PrivateClientFileEntity at the root:

            Debug.Assert(originalEntity.AccountReviewDate == null);
            var clonedEntity = GraphMerger.CloneGraph(originalEntity);
            Debug.Assert(clonedEntity.AccountReviewDate == null);

            clonedEntity.AccountReviewDate = DateTime.Now;

            var changes1 = GraphMerger.GetChangeSet(originalEntity, clonedEntity);
            Debug.Assert(changes1.FieldChanges.Count == 1);

            Debug.Assert(originalEntity.AccountReviewDate == null);
            GraphMerger.ApplyChangeSet(originalEntity, changes1);
            Debug.Assert(originalEntity.AccountReviewDate != null);

            clonedEntity = GraphMerger.CloneGraph(originalEntity);
            var changes2 = GraphMerger.GetChangeSet(originalEntity, clonedEntity);
            Debug.Assert(changes2.FieldChanges.Count == 0);

All it does is call GraphMerger.CloneGraph() to get a cloned graph. A field is then changed on the clone. Calling GraphMerger.GetChangeSet() will then compare the two graphs and make a list of changes that need to be applied to the OriginalGraph so it will be the same as the ClonedGraph. I then call GraphMerger.ApplyChangeSet() to make those changes. Cloning and then getting the changes again should find no changes, thus they are now the same.

I haven't got much done yet except to identify FieldChanges on an entity pair and be able to apply that change back onto the original graph.

I have used some reflection to call the GetRelatedData() method on each of the entity pairs

What I need help on is how to use that GetRelatedData data to iterate the related collections from the root entity pair downwards until all entity/collection pairs have been visited and recursively compared to produce some lists of changes. I currently have three lists currently to populate, though maybe more will be required later:

        readonly List<FieldChange> fieldChanges = new List<FieldChange>();
        readonly List<ChildEntityAddedChange> childEntitiesAdded = new List<ChildEntityAddedChange>(); 
        readonly List<ChildEntityRemovedChanged> childEntitiesRemoved = new List<ChildEntityRemovedChanged>();

where the contents of each list is a memento of what to change and how. For example FieldChange looks like this for now

        public class FieldChange
        {
            public EntityBase2 Entity;
            public IEntityField2 Field;
            public object NewValue;
        }

Since I know the cloned graph will be thrown away, Entity and Field are taken directly from the Cloned graph for now. It could be that they are replaced by an EntityType/PK values/FieldName in future but it makes it easy to find the matching Entity in the Original graph because of the way entities compare themselves.

I am not asking you write any code for this but if you could point out how to make use of the GetRelatedData for traversal and give me a heads up on any potential gotchas you can see immediately (and that I would find eventually), I would be grateful.

Walaa avatar
Walaa
Support Team
Posts: 14995
Joined: 21-Aug-2005
# Posted on: 18-Mar-2013 21:32:51   

So I think you are looking for a way to traverse the entire graph, correct. And for that you should check the ObjectGraphUtils class.

simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 19-Mar-2013 08:24:18   

Walaa wrote:

So I think you are looking for a way to traverse the entire graph, correct. And for that you should check the ObjectGraphUtils class.

Well maybe.

But as soon as I see phrases like "TopologySort" and "AdjacencyLists" my eyes just glaze over.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39910
Joined: 17-Aug-2003
# Posted on: 19-Mar-2013 09:47:12   

simmotech wrote:

Walaa wrote:

So I think you are looking for a way to traverse the entire graph, correct. And for that you should check the ObjectGraphUtils class.

Well maybe.

But as soon as I see phrases like "TopologySort" and "AdjacencyLists" my eyes just glaze over.

'X<-Y' means Y depends on X.

Graph: Customer<-Order<-OrderDetails

I save: an instance of Order, which references 10 OrderDetails and 1 Customer.

Order depends on Customer, so I can't simply save Order first. Topological sorting simply sorts dependencies in a graph, so when I save the Order instance and I topological sort the graph it's part of, I get: Customer, Order, 10xOrderDetails.

It does that by simply doing a depth-first-search, which means it traverses the whole graph, following the edges (the connections between the nodes), by using recursion and first going for the deepest node in the graph and then work its way upwards. The funny thing is that if you collect the nodes you run into along the way and reverse that list, you get the dependency order simple_smile . Voila, topological sorting.

Adjacency lists are lists of connections, 'edges'. you can store a graph in a couple of ways: simply let each node reference the nodes it is connect with (like our Customer, Order and the 10 OrderDetail entities do), or you can create a list of connections between the nodes (vertices). This list is called an adjacency list. The advantage of the adjacency list is that you can look at a graph which vertex/node is connected with which other vertices/nodes without traversing the graph to that node to look at its connections.

To be able to do a depth-first-search, so traverse all edges of a graph, you first have to know which edges there are. As the entities references each other, the objectgraphutils first collect this information and create the adjacency lists, so which entities are connected with which other entities. In our example above, this means: ('Order' means the order instance in the in-memory graph, 'Customer' means the customer instance etc.)

Order - Customer Order - OrderDetail1 Order - OrderDetail2 etc.

We now feed this list of 'edges' to the topological sort algorithm, which simply traverses these edges, using depth first search. So it will start with the order instance, will look into the adjacency list to which vertices it is connected, and then traverse these one by one, using recursion, so it will 'visit' the associated entity of the first edge it finds in the list, and then look again in the adjacency list to which nodes that node is connected to etc.

So collecting the adjacency lists is enough to get all information about which node is connected to which other node as the adjacency list is simply a set of connections, 'references' so you will, but just in 1 place.

Frans Bouma | Lead developer LLBLGen Pro
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 19-Mar-2013 12:13:02   

Thanks Frans, that's a useful description.

I think the info I need is more akin to the fast serialization stuff though (and I can't remember very much about how that works LLBLGen-wise). Its not about the dependency order for saving but more about using the metadata stored within the entities.

Using your example and starting with OrderA and OrderB (original and cloned respectively):

1) I can compare the Fields of OrderA and OrderB and note the differences. 2) I need to then find the 'owned' member collections. OrderDetailsEntityCollectionA and OrderDetailsEntityCollectionB in this case but I need to do it generically using the metadata. 3) I can then compare two entity collections. The B side is the master in this case. I work out which entities are in A but not in B - a note is made that these are Removals. I work out which entities are in B but not A - a note is made that these are Additions. I then find the entities that are in both collections. For each of these I perform step 1 recursively.

There will be more to it than this of course: - 1:1 relationship entity compares etc.etc. - keeping a dictionary of visited entities etc.

but you get the idea - once the process finishes, the 'notes' stored can be applied to OrderA and once that is finished, the graphs created by OrderA and OrderB will be identical.

I have some metadata as described in this method here but what else do I need?

        void ProcessEntity(EntityBase2 original, EntityBase2 clone)
        {
            FindFieldChanges(original, clone);

            var originalRelatedData = ((IEntityCore) original).GetRelatedData();
            var cloneRelatedData = ((IEntityCore) original).GetRelatedData();

            // What should I do with 2 x Dictionary<string, object> here

            // What other metadata should I be collecting and processing
        }

Walaa avatar
Walaa
Support Team
Posts: 14995
Joined: 21-Aug-2005
# Posted on: 19-Mar-2013 19:49:13   

2) I need to then find the 'owned' member collections. OrderDetailsEntityCollectionA and OrderDetailsEntityCollectionB in this case but I need to do it generically using the metadata.

Why do you need that?

Aren't you after traversing the graph and its clone, so you can compare corresponding entities and modify one to match the other?

simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 20-Mar-2013 08:03:07   

Walaa wrote:

2) I need to then find the 'owned' member collections. OrderDetailsEntityCollectionA and OrderDetailsEntityCollectionB in this case but I need to do it generically using the metadata.

Why do you need that?

Aren't you after traversing the graph and its clone, so you can compare corresponding entities and modify one to match the other?

Matching corresponding Field values is only part of the job. Their owned collections have to be matched also.

So OrderA which owned a collection of 5 OrderItemsA might have to be adjusted to match the modified OrderB which had 1 of its OrderItemsB removed and 2 added, say. That can only be done by comparing EntityCollections.

Walaa avatar
Walaa
Support Team
Posts: 14995
Joined: 21-Aug-2005
# Posted on: 20-Mar-2013 18:32:36   

Please use EntityBase.GetMemberEntityCollections() to get the related collections. Also you might find GetDependentRelatedEntities() & GetDependingRelatedEntities() useful.

Please check the reference manual for these methods.

simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 21-Mar-2013 11:23:39   

Was hoping for a bit more help than RTFM to be honest.

Otis avatar
Otis
LLBLGen Pro Team
Posts: 39910
Joined: 17-Aug-2003
# Posted on: 22-Mar-2013 11:03:06   

In all honesty, what do you want us to do? We can only give you pointers, hints where to look for the building blocks for what you need to build in your own code; we can't write your code for you.

comparing collections isn't that hard, Linq will help you there, or use BCL Extensions' SetEqual method to compare two IEnumerable's whether they're equal. BCL Extensions is OSS we ship on github: https://github.com/SolutionsDesign/BCLExtensions

You for yourself have to determine when an entity object A is different from entity object B: be it PK differences, or if they have the same PK value (and thus same HashCode), what to do if B is then different from A.

Frans Bouma | Lead developer LLBLGen Pro
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 22-Mar-2013 14:37:00   

Hi Frans

I have been thinking about this feature a little more:-

I believe it is a very useful feature and it is a curious omission from an otherwise fine set of features that is LLBLGen. You may disagree as to its usefulness of course, but if it could be robustly implemented, I am sure you would be happy to include it in a future release as another feature.

Back in 2009, there were a couple of threads discussing similar (though not the same) features. In fact, you sounded keen to implement this and indicated that it was planned for v3. Since then nothing has happened.

I think it is important to differentiate the feature I am proposing and those in the other threads. They seemed a lot more complicated than necessary being based on Undo/Redo. I am suggesting a simple Clone, Diff and Merge of 'entity graphs'. The Clone is trivial, the Diff and Merge less so.

I believe that with your intimate knowledge of the LLBLGen runtime framework, you could probably spec. up how to do the Diff between two 'entity graphs' in about half an hour (I inverted 'entity graphs' because I believe it isn't a graph problem as such - more how to iterate over pairs of entities/collections).

For me to do the same would take much longer, basically because I don't have anything like the knowledge you have and the lower-level workings I used to know for the serialization stuff is just a hazy memory now. Plus things have changed in v3 since then. The other problem is time, I just don't have the time any more to trace through low-level code working out what is happening and how to make use of metadata etc. I wish I did because I love to do that but it just isn't possible.

I am not asking you to write any code for this, I am more than prepared to do that and I also have some real usages for testing.

What I was hoping for is for you to give some guidance/discussion on which way to go to get that robust implementation:-

Something along the lines of :- - Right, we have two paired entities at the top. To compare those, we can go through the Fields and note what is different. We can skip XYZ because they are not data fields [IDs for example] and will be picked up later during ABC anyway (decide whether to compare related IDs or related entities). - Once the Fields are diffed, we then go through the related stuff. - We can use the metadata from XYZ()/ABC() or DEF() to find Depending entities and Dependent entites [I can't even remember which is which!] and QRS() to find any 1:1 relationships which we deal with by doing XXXX. - The collections can be compared like this...(blah, blah) that will give us Additions and Removals. We can use Reference comparisons or PK comparisons for .... but we have to check <Some odd condition check and way around it here> - We may need to compare orders within the collection to since they are lists but that may be an option - We need a simple HashSet to check whether we have visited an entity before - just stop if we have - We only need to continue comparing entities pairs that still remain on the clone side. (Should we recurse or just append to a list???) - Once all entities have been processed, we will have some collections of Diff notes - FieldChanges, CollectionRemovals, CollectionAdditions etc (what should we store to identify the entity they are to be applied on - Type/PKs or just store the reference to the Clone anyway since it will be discarded)

For the Merge, we need to - Decide whether to push updates directly into the Fields and call OnPropertyChanged(null) later or whether everything should be via SetValue (or an option to do that) in case the entities have code that 'adjusts' values. - How to 'finger poke' entities into/from the List<T> in CollectionCore so that attached Views all see the changes at once. - etc. etc. For testing, we Merge the Diffs back into the original and ReDiff the two and expect get Diffs back.

I realise that this is a little beyond a normal support request, so if you don't want to help me 'fill in the blanks' then I promise I won't hassle you any more (well not on this subject anyways sunglasses ).

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 23-Mar-2013 06:54:33   

From the top of my head, I think your process should consider this:

1) Establish what is the concept of 'equality' for you: - ID's (c1.CustomerId == c2.CustomerId, means that they are equal? - CurrentValue, DBValue, of both? (what if some entity is fetched and the other is new?) - If you have c1 and c2, both with values in all fields, which one would you peek up for the merge?

2) One you sorted out (1), you can peak an entity and which fields should go onto the merged result. Now is time to work with related data, which could be divided in cases: a) c1.Orders[0] is not present at c2.Orders. The question here is: Was it removed from c2.Orders or added to c1.Orders?

3) Traverse...

I think that, in your casa you have an OriginalEntity and a ClonedEntity. So it's not a full merge but simply cloning the changes from ClonedEntity to OriginalEntity. So that would give us the criteria for (1) and (2): If there is a difference between c1 and c2, the cloned one has precedence on the changes.

I will see if I can cook a very basic example that you can start with, and work on top of that. I hope to do that at this weekend.

David Elizondo | LLBLGen Support Team
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 19-Apr-2013 15:17:01   

This all went a bit quiet. simple_smile

daelmo avatar
daelmo
Support Team
Posts: 8245
Joined: 28-Nov-2005
# Posted on: 20-Apr-2013 08:00:14   

simmotech wrote:

This all went a bit quiet. simple_smile

Yes simple_smile sorry. I wrote some code, but it seems to simply and I failed to overcome nicely the point 1). I working on it. I will share it soon, hope it will help you somehow.

BTW, Do you have something so far?

David Elizondo | LLBLGen Support Team
simmotech
User
Posts: 1024
Joined: 01-Feb-2006
# Posted on: 24-Apr-2013 16:04:40   

daelmo wrote:

simmotech wrote:

This all went a bit quiet. simple_smile

Yes simple_smile sorry. I wrote some code, but it seems to simply and I failed to overcome nicely the point 1). I working on it. I will share it soon, hope it will help you somehow.

BTW, Do you have something so far?

Not really, we are doing the full graph clone and replacing the original with the clone when user the accepts the changes. It's not efficient though and that pains me but it will have to do for now.

OK, the concept of 'equality': Since we are only considering OriginalGraph and CloneOfOriginalGraph, we can assume they will broadly be similar, especially if the root is a single Entity (a Root of an EntityCollection has a different starting point but is just a case of matching by ID from one list to the other)

We therefore know the two Entity types are the same and the PKs are the same. So comparing the entity (but not its related entities) should be a case of comparing the Fields only. The DBValue will be the same but the Clone may have a different CurrentValue. So that is the change we need to record. We store a EntityFieldValueChange 'record' noting OriginalEntity, the Field that changed and its new value.

Then that entity's Related entities are traversed via its collections and considered in the same manner. This is bit I don't know how to do. The metadata is there somewhere to find DependingRelatedEntities and DependentRelatedEntities but I look at the FastSerializer stuff and my eyes glaze over. Thats where I was hoping Frans would say "Oh, you just need to loop through this collection, process those entities, store this or that, then at the end of the loop do the other" or something like that. I guess the result will more 'records' to add to the diff list: EntityAddedChange which notes the collection and the New entity (remember the clone will no longer need it so it can be taken directly from there) and EntityRemovedChange which notes the collection and the minimum necessary information to identify what to remove (maybe just a PK?)

Eventually we have a list of 'records' that describe all the changes that need to be made to OriginalEntityGraph to make it look like ClonedEntityGraph. If that list is empty then nothing changed. You can use SaveButton.Enabled = diffResultList.Count != 0; and be confident that is correct.

If changes are found and the use clicks Save, then the list is used by the Merge to apply those changes - changing field values and adding/removing entities from collections.

Since the ClonedEntityGraph will not be required after the Merge, there may be opportunities to cannibalise it such as transplanting the Fields from Clone Entity to its Partner OriginalEntity etc.

Cheers Simon