I am trying out this code to clone an entity tree structure

        public static T CloneEntityAsNew<T>(T entity, List<Type> excludedTypes, List<EntityBase2> excludeEntityInstances, bool generateNewObjectIDs = true) where T: EntityBase2
        {
            var clone = CloneEntity(entity);

            var referencedEntityMap = new ReferencedEntityMap(clone);

            foreach (var e in referencedEntityMap.GetSeenEntities())
            {
                if ((excludedTypes != null && excludedTypes.Contains(e.GetType())) || (excludeEntityInstances != null && excludeEntityInstances.Contains(e))) continue;

                e.IsNew = true;
                e.IsDirty = true;

                if (generateNewObjectIDs)
                {
                    ((IEntityCore) e).ObjectID = Guid.NewGuid();
                }

                e.Fields.IsDirty = true;

                for (var i = 0; i < e.Fields.Count; i++)
                {
                    if (e.Fields[i].IsPrimaryKey)
                    {
                        //e.Fields[i].ForcedCurrentValueWrite(null);
                    }

                    e.Fields[i].IsChanged = true;
                }
            }

            return clone;
        }

I have a single top-level ReportEntity with a 1:n with ReportSectionEntity (the latter also have a self-referencing relations but I don't think that affects anything). The primary key on ReportEntity is "ID", an int, and is a SCOPE_IDENTITY() sequence.

The ForcedCurrentValueWrite(null) does indeed reset the PK to null but the child ReportSectionEntities don't receive notification so their ReportID FK stays at the PK value. I can't seem to find a way to 'poke' the entity to get it to update its children's FK to match.

I tried higher level calls without success:- "e.SetNewFieldValue(i, null);" won't reset the PK field at all, presumably because it isn't nullable.

"e.SetNewFieldValue(i, newPK--);" does change the PK but still doesn't update child FKs.

I have a CommonEntityBase class and so I can add code in there in a "ResetPrimaryKey()" method if there are any protected methods that would help.

What can I do?

PS ReportSection has references to other entities which aren't cloned as new. These are filtered out in the excludedTypes parameter. This makes a solution from just the PK end preferable.

LLBLGen Runtime library version/build no.? (check forum guidelines to know how to get the correct build no.)

(Edit) Also please check the following thread for deep cloning, I think it will be helpful. https://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=19363

v4.2.15.0121 (latest), Adapter

That deep cloning thread doesn't help really - that involves two entities representing the same thing and needing a context. Not the case here.

The point is that resetting a (sequence) PK on an entity doesn't automatically update the child entities in its in-memory graph. I believe that is by design (Frans in another post: "Changing PK's is a thing we consider a bad practise: changing the PK is actually the same as removing the entity and adding a new one with the new PK. ")

But in this case, it is a valid use-case because it is being reset, not changed (to another ID). Whilst I can hack it for now because I happen to know the types involved, I would prefer to find a way to do it generically once and for all.

The metadata is probably all there, I just don't know which bits to use!

I can't reproduced using the CloneHelper posted by Walaa and Northwind DB:

[TestMethod]
public void CloneAndReplicatePKFK()
{
    var order = new OrderEntity(10249);
    var path = new PrefetchPath2(EntityType.OrderEntity);
    path.Add(OrderEntity.PrefetchPathOrderDetails);

    using (var adapter = new DataAccessAdapter())
    {
        adapter.FetchEntity(order, path);
    }

    OrderEntity clonedOrder = (OrderEntity) CloneHelper.CloneEntity(order);

    Assert.AreEqual(clonedOrder.OrderId , 0);
    Assert.AreEqual(clonedOrder.OrderId, clonedOrder.OrderDetails[0].OrderId);          
}

In this case, the CloneEntity routine actually perform a deep clone and reset the PK-FK. Did you try it?

I tried your CloneHelper code (mine is almost identical anyway) with this unit test and the last test fails. It still returns 35.

        [Test]
        public void TestCloneReportWithCloneHelper()
        {
            var order = new ReportEntity(35);
            var path = new PrefetchPath2(EntityType.ReportEntity);
            path.Add(ReportEntity.PrefetchPathReportSections);

            using (var adapter = new DataAccessAdapter())
            {
                adapter.FetchEntity(order, path);
            }

            ReportEntity clonedOrder = (ReportEntity) CloneHelper.CloneEntity(order);

            Assert.AreEqual(clonedOrder.ID , 0);
            Assert.AreEqual(clonedOrder.ID, clonedOrder.ReportSections[0].ReportID);
        }

Its just dawned on me that our schema are not quite comparable. Mine has ReportSection.ReportID as nullable (0:n). Maybe that makes a difference?

Here is my schema for reference:-

USE [TIPS]
GO

/****** Object:  Table [dbo].[Report]   Script Date: 07/02/2015 13:59:10 ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

CREATE TABLE [dbo].[Report](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [AdditionalDataStore] [nvarchar](max) NULL,
    [Title] [nvarchar](max) NOT NULL,
    [Name] [nvarchar](50) NOT NULL,
    [FrontPageTemplateID] [int] NULL,
    [SigningPageTemplateID] [int] NULL,
    [IncludeFrontPage] [bit] NOT NULL,
    [IncludeSigningPage] [bit] NOT NULL,
    [IncludeEnclosuresList] [bit] NOT NULL,
    [EnclosureListTitles] [nvarchar](max) NULL,
    [IncludeMainBody] [bit] NOT NULL,
    [IncludeMainBodyHeader] [bit] NOT NULL,
    [IncludeAppendix] [bit] NOT NULL,
    [IncludeAppendixHeader] [bit] NOT NULL,
    [IncludeMainBodyTableOfContents] [bit] NOT NULL,
    [IncludeAppendixTableOfContents] [bit] NOT NULL,
    [MainBodyTableOfContentsTitle] [nvarchar](50) NULL,
    [ReportType] [tinyint] NOT NULL,
    [ContextType] [tinyint] NOT NULL,
    [IsHidden] [bit] NOT NULL,
    [Description] [nvarchar](max) NULL,
 CONSTRAINT [PK_f075e264907a442154058d0006f] PRIMARY KEY CLUSTERED 
(
    [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]

GO

ALTER TABLE [dbo].[Report]  WITH CHECK ADD  CONSTRAINT [FK_65ea6294f64b65ab75e4e878f29] FOREIGN KEY([FrontPageTemplateID])
REFERENCES [dbo].[Template] ([ID])
GO

ALTER TABLE [dbo].[Report] CHECK CONSTRAINT [FK_65ea6294f64b65ab75e4e878f29]
GO

ALTER TABLE [dbo].[Report]  WITH CHECK ADD  CONSTRAINT [FK_95297404c09b37e47dfbf38a933] FOREIGN KEY([SigningPageTemplateID])
REFERENCES [dbo].[Template] ([ID])
GO

ALTER TABLE [dbo].[Report] CHECK CONSTRAINT [FK_95297404c09b37e47dfbf38a933]
GO

USE [TIPS]
GO

/****** Object:  Table [dbo].[ReportSection]    Script Date: 07/02/2015 13:59:23 ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

CREATE TABLE [dbo].[ReportSection](
    [ID] [int] IDENTITY(1,1) NOT NULL,
    [ParentSectionID] [int] NULL,
    [Include] [bit] NOT NULL,
    [Title] [nvarchar](max) NULL,
    [TitleVisible] [bit] NOT NULL,
    [Text] [nvarchar](max) NULL,
    [BottomText] [nvarchar](max) NULL,
    [PageBreakBefore] [bit] NOT NULL,
    [PageBreakAfter] [bit] NOT NULL,
    [AdditionalDataStore] [nvarchar](max) NULL,
    [TextVisible] [bit] NOT NULL,
    [BottomTextVisible] [bit] NOT NULL,
    [DisplayOrder] [smallint] NULL,
    [ReportID] [int] NULL,
    [TextTemplateID] [int] NULL,
    [BottomTextTemplateID] [int] NULL,
    [SectionType] [tinyint] NOT NULL,
    [Location] [tinyint] NOT NULL,
 CONSTRAINT [PK_8b07d874cc587b32e69be157a27] PRIMARY KEY CLUSTERED 
(
    [ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]

GO

ALTER TABLE [dbo].[ReportSection]  WITH CHECK ADD  CONSTRAINT [FK_4b2183c4765a324dffad2eb9826] FOREIGN KEY([TextTemplateID])
REFERENCES [dbo].[Template] ([ID])
GO

ALTER TABLE [dbo].[ReportSection] CHECK CONSTRAINT [FK_4b2183c4765a324dffad2eb9826]
GO

ALTER TABLE [dbo].[ReportSection]  WITH CHECK ADD  CONSTRAINT [FK_a86686847d29646358ece433c8b] FOREIGN KEY([BottomTextTemplateID])
REFERENCES [dbo].[Template] ([ID])
GO

ALTER TABLE [dbo].[ReportSection] CHECK CONSTRAINT [FK_a86686847d29646358ece433c8b]
GO

ALTER TABLE [dbo].[ReportSection]  WITH CHECK ADD  CONSTRAINT [FK_d0692e8479e9077b85a88f0fab3] FOREIGN KEY([ParentSectionID])
REFERENCES [dbo].[ReportSection] ([ID])
GO

ALTER TABLE [dbo].[ReportSection] CHECK CONSTRAINT [FK_d0692e8479e9077b85a88f0fab3]
GO

• In your test, the assert that fails is the second one, Right?
• Also, in your schema, I don't see the FK between ReportSection -> Report. Is that a model-only relationship? If so, what are the fields that are involved in the relationship?

Yes, the second one fails.

Sorry, must have mis-pasted. The FK relationship looks like this:-

ALTER TABLE [dbo].[ReportSection] ADD CONSTRAINT [FK_7c530164201bd83c5e850b29c67] FOREIGN KEY ( [ReportID] ) REFERENCES [dbo].[Report] ( [ID] ) ON DELETE NO ACTION ON UPDATE NO ACTION GO

I did a test with your schema. I figured it out that in your case the FK isn't reset as in my Northwind example, because the FK is not part of the PK, so this code is never called:

if (field.IsPrimaryKey) field.ForcedCurrentValueWrite(null);

Also, as the entity is new, and it's ID is identity, it doesn't have sense to replicate the new value (null) across the graph.

The actual PK-FK sync in this scenario happens when you save the state to the DB. for instance, the following code would succeed:

// fetch
var report = new ReportEntity(_reportId);
var reportPath = new PrefetchPath2(EntityType.ReportEntity);
reportPath.Add(ReportEntity.PrefetchPathReportSections);
using (var adapter = new DataAccessAdapter())
{
    adapter.FetchEntity(report, reportPath);
}

// clone and save
ReportEntity clonedReport = (ReportEntity)CloneHelper.CloneEntity(report);
using (var adapter = new DataAccessAdapter())
{
    adapter.SaveEntity(clonedReport, true, true);
}

Assert.AreEqual(clonedReport.Id, _reportId + 1);
Assert.AreEqual(clonedReport.Id, clonedReport.ReportSections[0].ReportId);

I hope that makes sense

--EDIT-- Read next message first!!

(I hope that makes sense ) Nope.

because the FK is not part of the PK

Not sure what you mean here.

As far as I am aware IsPrimaryKey literally means IsPrimaryKey and not 'PartOfPrimaryKey' so

if (field.IsPrimaryKey) field.ForcedCurrentValueWrite(null);

is called on both ReportEntity and all the ReportSectionEntities. (it does - I put a breakpoint in to double-check)

Also, as the entity is new, and it's ID is identity, it doesn't have sense to replicate the new value (null) across the graph.

It does make sense to replicate it otherwise the child entities have a mismatch: the parent ReportEntity they hold says its PK is null (or 0 via the ID property) but the ReportID fields says it is <oldPKID>. However the NorthWind Order/OrderDetail structure is similar: The entity is new and its ID is identity, yet the unit test proves it is changing the FK field does it not.

This issue has resulted in corruption for me hence this support call. Maybe its because I happened to ultimately use adapter.SaveEntity(clonedReport, false, true); However, this graph is not saved immediately (and may not be saved to the database) so I can't rely on that 'fixing' up the sync mappings.

Also, your northwind unit test doesn't do any saving. It does the clone and the OrderDetail.OrderId is updated as expected.

In an attempt to prove that it is a nullable FK that is the issue, I've just tried another couple of similar parent/child entity clones where the FK is non-nullable in a unit test. Guess what - none of them reset the FK field. So my entities are apparently not doing the same as yours. I do have extensions to Entity and EntityCollection so I will review those in case they are having an effect.

But if your Northwind Order/OrderDetails is working correctly then it is expected that an FK syncs with its PK (and doesn't require saving). So why would a nullable FK be treated differently?

Does your unit test with my Report/ReportSection schema work without the save? If it does then it looks like my entity extensions are causing the problem. If it doesn't then either nullable-FKs work differently to non-nullable FKs (or your Northwind unit test isn't working??)

Just to check my sanity, I managed to cobble together a working northwind database. And then I noticed that the PK of OrderDetail includes OrderId! So now I understand what you meant by "because the FK is not part of the PK".

So my previous message is mainly redundant because the FK is updated in neither of my scenarios. The nullable FK was a red-herring. It only happened to work in Northwind because OrderDetail.OrderId was a PK too.

The issue still remains that I need a generic way to reset FK fields in a graph when its parent entity gets its PK reset. I need to do that without having to save the entity.

How can I do this using entity internal data?

Well the following is the code that sync the PK-FK in the runtimes, after the save action

protected void OnEntityAfterSave(object sender, EventArgs e)
{
    IEntityCore persistedEntity = (IEntityCore)sender;
    List<EntitySyncInfo<IEntityCore>> entitySyncInfos = new List<EntitySyncInfo<IEntityCore>>(GetEntitySyncInformation(persistedEntity).Values);
    for(int i = 0; i < entitySyncInfos.Count; i++)
    {
        SyncFKFields(entitySyncInfos[i]);
    }
}

So I suggest adding a method in CommonEntityBase that has similar code, to signal the synching, and be called whenever you want.

At the point of the clone, I would be accessing each entity once. This event handler code however needs an entity passing in so, unless I am mistaken, I would need to call this code on every other entity in the graph apart from the current one. Really?

This code tries SyncFKFields() for all EntitySyncInfos but still doesn't change the PK

        static readonly FieldInfo SyncDictionaryAccessor = typeof(EntityCore<IEntityFields2>).GetField("_relatedEntitySyncInfos", BindingFlags.NonPublic | BindingFlags.Instance);

        public void ResyncFKs()
        {
            var syncDictionary = SyncDictionaryAccessor.GetValue(this) as FastDictionary<Guid, FastDictionary<string, EntitySyncInfo<IEntityCore>>>;
            if (syncDictionary == null) return;

            foreach (var x in syncDictionary.Values)
            {
                foreach (var y in x.Values)
                {
                    SyncFKFields(y);
                }
            }
        }

What else can I try?

It looks like SyncFKFields specifically excludes resetting for null PK fields fields - no wonder it didn't work! (just downloaded the source code...)

                    if(dataSupplier.IsNew)
                    {
                        // only sync new PK sides with FKs if the PK field is changed and not NULL.
                        setValue &= (dataSupplier.Fields.GetIsChanged(pkFieldIndex) && pkFieldCurrentValue != null);
                    }

I suppose I could try to set the IsNew property in a separate loop after the main loop. Or I could try to copy the relevant bits from SyncFKFields but then I don't know the significance of whether setting the Used property on the EntitySyncInfo<> is relevant of not.

Could someone please come up with an official and reliable way of achieving this?

The cloned graph needs only new PK's? Set all entities' IsNew flag to true, set the IsChanged flag for the fields which have to look as if they got a new value (except the PK fields). For the PK and FK fields, set the IsChanged flag to false, and forcewrite the field values to null, on the EntityFields2 object. This should give a graph with entities which are all new and have no PK field values set and the IsChanged flag is not set. Saving the graph will then sync up the FKs during the save, as the PK's are identity fields: the data in the entity class instances in memory are supposed to become NEW entity instances in the DB, after all. The FKs of identity fields aren't synced in memory, you have to clear them manually during the traversal of the graph before persistence. They're synced when the graph is persisted. It's key the FK field values are set to null and their IsChanged flags are set to false.

Forget the Sync info, it's for internal housekeeping, and not meant for user usage.

The code I am trying to write should be generic and work in all scenarios. Crucially, it also needs to allow for some Types that are not to be made new. The code I showed in the first message allows me to pass in a list of Entity types and/or Entity instances that are left as references to the originals (I have removed the generateNewObjectIDs parameter and code BTW- not needed and screws up the SyncInfos). In my current scenario, both ReportEntity and ReportSectionEntity have multiple references to TemplateEntity which aren't to be cloned as IsNew.

Also, I don't necessarily want to save them to the database. It could be that I choose to clone a graph and then save it via DTOs or, as in my current case, I want to present the graph in the GUI, let the user make modifications and then Save or Cancel. In the latter case, the graph is only persisted if a user clicks save.

Set all entities' IsNew flag to true

OK

set the IsChanged flag for the fields which have to look as if they got a new value (except the PK fields and *some* FKs)

Not sure quite what to do here. I can exclude PK fields no problem of course but wouldn't all other fields (including some FKs) need IsChanged=true so they are written to the database?

For the PK and FK fields, set the IsChanged flag to false, and forcewrite the field values to null, on the EntityFields2 object. This should give a graph with entities which are all new and have no PK field values set and the IsChanged flag is not set.

This is where the Exclude Entity Types/Instances lists come in and complicates things. I only want to reset the FK fields to null where the PK belongs to an entity that is not in the exclude lists. Therefore it is at the point I reset the PK on an entity that I need to reset the FK on its related entities. Doing it from the entity with the FK side is more troublesome. In theory, I could test the related entity to see if it is outside the exclude list but since I am dealing with Fields in a loop, it is difficult to find the related entity.

Saving the graph will then sync up the FKs during the save, as the PK's are identity fields: the data in the entity class instances in memory are supposed to become NEW entity instances in the DB, after all. The FKs of identity fields aren't synced in memory, you have to clear them manually during the traversal of the graph before persistence. They're synced when the graph is persisted. It's key the FK field values are set to null and their IsChanged flags are set to false. 

Need to do this without persisting as mentioned and so I am attempting to set to the FK field values to null (and IsChanged=false) but how to do it? If attempted from the PK side, I need to detect the related FKs and reset them. For a generic method, I believe only the SyncInfos can tell me this but you said its best not to use them. If attempted from the FK side, I start with just an FK field. From that I would need to find the entity in the graph to which it is referring and then see if it is passes the exclude filters before I can reset it. I don't know how I could do that.

If you don't want to persist them, why bother with the FK/PK fields at all? And why not fetch the data again? Keep in mind this is not a feature we support out of the box so it might be we don't have a sufficient answer for you.

If you don't want to persist them, why bother with the FK/PK fields at all? And why not fetch the data again?

I have detailed this twice in the thread. DTOs - no point saving a random number in the FK field; saving - user may start to create a clone then change their and not save it.

Keep in mind this is not a feature we support out of the box so it might be we don't have a sufficient answer for you.

I see. Well thanks for your time. I know it can be done and it is useful, not an edge case, so I'll hack away until I get a solution.

This seems to work fine.

        public static T CloneEntityAsNew<T>(T entity, List<Type> excludedTypes, List<EntityBase2> excludeEntityInstances) where T: EntityBase2
        {
            var clone = CloneEntity(entity);

            var entitiesToMarkNew = new ReferencedEntityMap(clone)
                .GetSeenEntities()
                .Where(e => ((excludedTypes == null || !excludedTypes.Contains(e.GetType())) && (excludeEntityInstances == null || !excludeEntityInstances.Contains(e))))
                .ToList();

            foreach (var e in entitiesToMarkNew)
            {
                e.IsNew = true;
                e.IsDirty = true;
                e.Fields.IsDirty = true;

                var allSyncInfos = GetSyncInfos(e);

                var syncInfosWithResetPK = allSyncInfos
                    .Where(si => entitiesToMarkNew.Contains(si.DataSupplyingEntity))
                    .ToArray();

                var fkIndexesToReset = syncInfosWithResetPK
                    .Select(si => si.Relation.GetFKEntityFieldCore(0).FieldIndex) // We don't support multi-field FKs - work of the devil
                    .ToArray();

                for (var i = 0; i < e.Fields.Count; i++)
                {
                    var field = e.Fields[i];

                    if (field.IsPrimaryKey || (field.IsForeignKey && fkIndexesToReset.Contains(i)))
                    {
                        field.ForcedCurrentValueWrite(null);
                    }
                    else
                    {
                        field.IsChanged = true;
                    }
                }
            }

            return clone;
        }

        static IEnumerable<EntitySyncInfo<IEntityCore>> GetSyncInfos(EntityBase2 entity)
        {
            var syncDictionary = SyncDictionaryAccessor.GetValue(entity) as FastDictionary<Guid, FastDictionary<string, EntitySyncInfo<IEntityCore>>>;

            return syncDictionary == null
                ? new EntitySyncInfo<IEntityCore>[0]
                : syncDictionary.Values.SelectMany(x => x.Values);
        }

        static readonly FieldInfo SyncDictionaryAccessor = typeof(EntityCore<IEntityFields2>).GetField("_relatedEntitySyncInfos", BindingFlags.NonPublic | BindingFlags.Instance);

Thanks for the share.