• Runtime version 5.9.4
• Adapter
• Postgres 15
• Npgsql 4.0.12
• .NET Standard 4.8.0

The amount of work being done on background threads went way up in the version of this fairly old product I'm trying to release and now I'm getting some unexpected behavior. Former released version was 4.2 with Npgsql 2.0.14. I'm getting exceptions that seem to be jumping threads when doing inserts/updates simultaneously, and an update turned into an insert in this example.

In the logs below, you'll see that I am doing a transaction on thread [13] - the second line was generated by this log statement:

 s_ILog.Debug($"{Model.DataFileName} {kind} [{string.Join("] [", changed
             .Select(ch => $"{(ch.IsNew?"new":"")} 
             {
                       string.Join(",", ch.Fields.Where(f=>f.IsChanged)
                            .Select(f=>$"{f.Name} {f.DbValue}->{f.CurrentValue}"))}"))}]");

So, I have IsNew is false with 4 changed fields. However, in the middle of the log block, you'll see an INSERT INTO "public"."tube_board" command, and it's being run on thread 9, where a different INSERT was being executed. This tube_board insert fails because there are FK's that are not populated in this INSERT statement, and the real UPDATE is below that on it's real thread.

Each function has it's own Adapter.

2023-06-07 14:36:15,609 DEBUG ReportEditor.ViewModel.TubeViewModel [13] - start transaction 258
2023-06-07 14:36:15,611 DEBUG ReportEditor.ViewModel.TubeViewModel [13] - DCR086T126I258 NumReports [ NumReports ->0,AppendReady ->False,OverrodeValidation ->False,ResolutionSatisfied ->False]
2023-06-07 14:36:15,606 DEBUG System.Diagnostics.Redirection [9] - Method Enter: CreateInsertDQ
2023-06-07 14:36:15,634 DEBUG System.Diagnostics.Redirection [9] - Method Enter: CreateSingleTargetInsertDQ
2023-06-07 14:36:15,636 DEBUG System.Diagnostics.Redirection [9] - Generated Sql query:
        Query: INSERT INTO "public"."dispositions" ("disposition_id", "timestamp", "discrepancy_id", "report_entry_id", "analyst_user_name", "reso_pass", "keep", "mechanism", "cdg_id", "run_id") VALUES (nextval('"public"."dispositions_disposition_id_seq"'), :p2, :p3, :p4, :p5, :p6, :p7, :p8, :p9, :p10) RETURNING "disposition_id"
        Parameter: :p2 : DateTime. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: 2023-06-07T14:36:15.5975122-07:00.
        Parameter: :p3 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 125726.
        Parameter: :p4 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 974810.
        Parameter: :p5 : String. Length: 1073741824. Precision: 0. Scale: 0. Direction: Input. Value: "B5678".
        Parameter: :p6 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 0.
        Parameter: :p7 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: True.
        Parameter: :p8 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 3.
        Parameter: :p9 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 87277.
        Parameter: :p10 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 1.


2023-06-07 14:36:15,638 DEBUG System.Diagnostics.Redirection [9] - Method Exit: CreateSingleTargetInsertDQ
2023-06-07 14:36:15,640 DEBUG System.Diagnostics.Redirection [9] - Method Exit: CreateInsertDQ
2023-06-07 14:36:15,642 DEBUG System.Diagnostics.Redirection [9] - Method Enter: CreateInsertDQ
2023-06-07 14:36:15,643 DEBUG System.Diagnostics.Redirection [9] - Method Enter: CreateSingleTargetInsertDQ
2023-06-07 14:36:15,645 DEBUG System.Diagnostics.Redirection [9] - Generated Sql query:
        Query: INSERT INTO "public"."tube_board" ("tb_id", "num_reports", "append_ready", "overrode_validation", "resolution_satisfied") VALUES (nextval('"public"."tube_board_tb_id_seq"'), :p2, :p3, :p4, :p5) RETURNING "tb_id"
        Parameter: :p2 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 0.
        Parameter: :p3 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.
        Parameter: :p4 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.
        Parameter: :p5 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.


2023-06-07 14:36:15,647 DEBUG System.Diagnostics.Redirection [9] - Method Exit: CreateSingleTargetInsertDQ
2023-06-07 14:36:15,650 DEBUG System.Diagnostics.Redirection [9] - Method Exit: CreateInsertDQ
2023-06-07 14:36:15,654 DEBUG System.Diagnostics.Redirection [13] - Method Enter: CreateUpdateDQ(4)
2023-06-07 14:36:15,717 DEBUG System.Diagnostics.Redirection [13] - Method Enter: CreateSingleTargetUpdateDQ(4)
2023-06-07 14:36:15,805 DEBUG System.Diagnostics.Redirection [13] - Generated Sql query:
        Query: UPDATE "public"."tube_board" SET "num_reports"=:p1, "append_ready"=:p2, "overrode_validation"=:p3, "resolution_satisfied"=:p4 WHERE ( "public"."tube_board"."tb_id" = :p5)
        Parameter: :p1 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 0.
        Parameter: :p2 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.
        Parameter: :p3 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.
        Parameter: :p4 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.
        Parameter: :p5 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 14405.


2023-06-07 14:36:15,818 DEBUG System.Diagnostics.Redirection [13] - Method Exit: CreateSingleTargetUpdateDQ(4)

disposition exception [9]

• ex {"An exception was caught during the execution of an action query: 23502: null value in column \"df_id\" of relation \"tube_board\" violates not-null constraint. Check InnerException, QueryExecuted and Parameters of this exception to examine the cause of this exception."} System.Exception {SD.LLBLGen.Pro.ORMSupportClasses.ORMQueryExecutionException}
• Data {System.Collections.ListDictionaryInternal} System.Collections.IDictionary {System.Collections.ListDictionaryInternal}
• DbSpecificExceptionInfo Count = 4 System.Collections.Generic.Dictionary<SD.LLBLGen.Pro.ORMSupportClasses.ExceptionInfoElement, object> HResult -2146233088 int HelpLink null string IPForWatsonBuckets 0x00007ffdf20fbdb7 System.UIntPtr
• InnerException {"23502: null value in column \"df_id\" of relation \"tube_board\" violates not-null constraint"} System.Exception {Npgsql.PostgresException}
• InvolvedEntity {Zetec.Analysis.EntityClasses.TubeBoardEntity} SD.LLBLGen.Pro.ORMSupportClasses.IEntityCore {Zetec.Analysis.EntityClasses.TubeBoardEntity} IsTransient false bool Message "An exception was caught during the execution of an action query: 23502: null value in column \"df_id\" of relation \"tube_board\" violates not-null constraint. Check InnerException, QueryExecuted and Parameters of this exception to examine the cause of this exception." string
• Parameters {Npgsql.NpgsqlParameterCollection} System.Collections.IList {Npgsql.NpgsqlParameterCollection} QueryExecuted "\r\n\tQuery: INSERT INTO \"public\".\"tube_board\" (\"tb_id\", \"num_reports\", \"append_ready\", \"overrode_validation\", \"resolution_satisfied\") VALUES (nextval('\"public\".\"tube_board_tb_id_seq\"'), :p2, :p3, :p4, :p5) RETURNING \"tb_id\"\r\n\tParameter: :p2 : Int32. Length: 0. Precision: 10. Scale: 0. Direction: Input. Value: 0.\r\n\tParameter: :p3 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.\r\n\tParameter: :p4 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.\r\n\tParameter: :p5 : Boolean. Length: 0. Precision: 0. Scale: 0. Direction: Input. Value: False.\r\n" string RemoteStackTrace null string RuntimeBuild "5.9.4_NetFull" string RuntimeVersion "5.9" string Source "SD.LLBLGen.Pro.ORMSupportClasses" string StackTrace " at SD.LLBLGen.Pro.ORMSupportClasses.ActionQuery.Execute() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.9\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQuery.cs:line 297\r\n at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.ExecuteActionQuery(IActionQuery queryToExecute) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.9\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 577\r\n at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.ExecuteElements(List1 elementsToRun) in C:\\Myprojects\\VS.NET Projects\\LLBLGen Pro v5.9\\Frameworks\\LLBLGen Pro\\RuntimeLibraries\\ORMSupportClasses\\Query\\ActionQueryController.cs:line 177\r\n at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.PersistQueue(List1 queueToPersist, Boolean insertActions) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.9\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 1038\r\n at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntity(IEntity2 entityToSave, Boolean refetchAfterSave, IPredicateExpression updateRestriction, Boolean recurse) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.9\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 954\r\n at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.<>c__DisplayClass19_0.<SaveEntity>b__0() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.9\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 413\r\n at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntity(IEntity2 entityToSave) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.9\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 843\r\n at ReportEditor.ViewModel.DispositionsViewModel.SetStatus(Boolean keep, DispositionMechanism mech) in C:\\dev\\eddynet\\projects\\ReportEditor\\ViewModel\\EresoViewModels.cs:line 573" string
• TargetSite {Int32 Execute()} System.Reflection.MethodBase {System.Reflection.RuntimeMethodInfo}

tube_board stack trace [13]

System.dll!System.Diagnostics.DefaultTraceListener.Write(string message, bool useLogFile) Line 178  C#
System.dll!System.Diagnostics.DefaultTraceListener.WriteLine(string message, bool useLogFile) Line 209  C#
System.dll!System.Diagnostics.TraceInternal.WriteLine(string message, string category) Line 512 C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.DynamicQueryEngineBase.CreateUpdateDQ(SD.LLBLGen.Pro.ORMSupportClasses.IEntityFieldCore[] fields, SD.LLBLGen.Pro.ORMSupportClasses.IFieldPersistenceInfo[] fieldsPersistenceInfo, System.Data.Common.DbConnection connectionToUse, System.Collections.Generic.List<SD.LLBLGen.Pro.ORMSupportClasses.IPredicate> pkFilters) Line 722   C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.Adapter.QueryCreationManager.CreateUpdateDQ(SD.LLBLGen.Pro.ORMSupportClasses.IEntity2 entityToSave, SD.LLBLGen.Pro.ORMSupportClasses.IFieldPersistenceInfo[] persistenceInfoObjects, System.Collections.Generic.List<SD.LLBLGen.Pro.ORMSupportClasses.IPredicate> pkFilters) Line 312 C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.Adapter.QueryCreationManager.CreateQueryForEntityToSave(bool insertActions, SD.LLBLGen.Pro.ORMSupportClasses.IEntity2 entityToSave, SD.LLBLGen.Pro.ORMSupportClasses.IPredicateExpression updateRestriction, SD.LLBLGen.Pro.ORMSupportClasses.InheritanceHierarchyType typeOfHierarchy, SD.LLBLGen.Pro.ORMSupportClasses.IFieldPersistenceInfo[] persistenceInfoObjects) Line 1255    C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.Adapter.QueryCreationManager.CreateSaveQueryForEntity(bool insertActions, SD.LLBLGen.Pro.ORMSupportClasses.ActionQueueElement<SD.LLBLGen.Pro.ORMSupportClasses.IEntity2> element, SD.LLBLGen.Pro.ORMSupportClasses.EntityBase2 entityToSave) Line 523 C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.PersistQueue(System.Collections.Generic.List<SD.LLBLGen.Pro.ORMSupportClasses.ActionQueueElement<SD.LLBLGen.Pro.ORMSupportClasses.IEntity2>> queueToPersist, bool insertActions) Line 1010  C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.CommitImpl(SD.LLBLGen.Pro.ORMSupportClasses.IDataAccessAdapter adapterToUse, bool autoCommit) Line 743    C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntityCollection(SD.LLBLGen.Pro.ORMSupportClasses.IEntityCollection2 collectionToSave, bool refetchSavedEntitiesAfterSave, bool recurse) Line 1971  C#
SD.LLBLGen.Pro.ORMSupportClasses.dll!SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.SaveEntityCollection.AnonymousMethod__0() Line 429  C#
ReportEditor.exe!ReportEditor.ViewModel.TubeViewModel.UpdateTubeBoard(ReportEditor.ViewModel.TubeBoardUpdate kind, bool changed_reports, bool retrying_from_exception) Line 1544    C#
ReportEditor.exe!ReportEditor.ViewModel.ResoTubeViewModel.HandleValidationChange(bool badness_changed) Line 1700    C#
ReportEditor.exe!ReportEditor.ViewModel.TubeViewModel.AdjustState(ReportEditor.ViewModel.ValidationViewModel.FileAnalyzedState s, string analysis_group) Line 738   C#
ReportEditor.exe!ReportEditor.ViewModel.ValidationViewModel.Validator_DoWork(object sender, System.ComponentModel.DoWorkEventArgs e) Line 663   C#
System.dll!System.ComponentModel.BackgroundWorker.OnDoWork(System.ComponentModel.DoWorkEventArgs e) Line 107    C#
System.dll!System.ComponentModel.BackgroundWorker.WorkerThreadStart(object argument) Line 245   C#
[Native to Managed Transition]  
[Managed to Native Transition]  
mscorlib.dll!System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage msg, System.Runtime.Remoting.Messaging.IMessageSink replySink) Line 298  C#
mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 980  C#
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx) Line 928  C#
mscorlib.dll!System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() Line 1252    C#
mscorlib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Line 820   C#

Could you please provide a small repro?

Also, for our understanding, you run an insert on a different thread (9) than an update? or the insert on thread 9 should have been an update? It's a bit confusing. With multi-threaded work, make 100% sure you use isolated objects, adapters, connections, everything. It's required to verify this at runtime, to avoid wrong assumptions.

Additionally, which change made it suddenly fail?

I'll try to work on a repro, but that's not going to be particularly easy (is it ever?). I was really hoping you'd just say "Obviously you haven't read XYZ in the docs."

In case it matters, I have this on my connection string, the last 3 are new to this version:

         conn += ";CommandTimeout=200";
         conn += ";No Reset On Close=true"; //https://github.com/npgsql/npgsql/issues/2920 - the DISCARD ALL was making AdvisoryLock not work
         conn += ";Enlist=false"; //https://www.npgsql.org/doc/performance.html
         conn += ";Include Error Detail=true"; //we don't have sensitive data, and I can't see why we wouldn't want all the info we can get

One thread is doing an insert via SaveEntity, the other is doing an update via SaveEntityCollection. It seems like the entity being updated is jumping into the wrong thread/adapter and getting additionally inserted there.

As far as what is new, the entire tube_board table is new and is adjusted in various threads. It took me a good bit of time to confirm that I wasn't interacting with the tube_board table on the insert thread, because I do often insert and update the tube_board in various paths.

Given the frequent exhortations that adapters are not expensive and their lifetimes should be limited to the logical region they are used, I do not hoard them.

I'm updating to LLBL 5.10.1 and Npgsql 4.1.12 to see if that helps.

Updated to LLBL 5.10.1 and Npgsql 4.0.13 (had issues with 4.1.x that I didn't want to tackle).

I've now seen these misplaced UPDATEs to TubeBoardEntities from two different functions. In the second case, the TubeBoardEntity is being updated with other items being updated. It. In both cases, there is a transaction happening while the other thread is updating the TubeBoardEntity. Additionally, there is an external process which is occasionally deleting the TubeBoardEntities, which is why the update fails in situation #2.

One spot where this happens was in a nested adapter, so I un-nested it. I've seen problems with multiple active adapters on one thread. I tried to add a ThreadLocal<bool> to my Create.Adapter function (set false in Adapter's Dispose function) to ensure there was never nesting, but async means at times I have a different thread a creation vs dispose.

I guess I could block other queries during a transaction?

I'm really not sure what to think here. The most common place where I get the ORMConcurrencyException looks like

            Log.Debug('f');
            using (DataAccessAdapter adapter = Create.Adapter(null, ViewModelLocator.MainStatic.ProcedureVessel))
            {
               adapter.SaveEntity(CurrentDisposition);
            }
            Log.Debug('g');

This last time, I see the 'f', then the Insert of CurrentDisposition, followed by two other updates, one of which I believe would have happened previously and the other concurrently. I don't know why they are jumping threads. The Insert succeeds.

I turned up the tracing and repeated the issue again, happened in a different spot this time. This time I have this code

            adapter.StartTransaction(IsolationLevel.ReadCommitted, "Creating Discrepancies");
            num_changed_entities += adapter.SaveEntityCollection(disc);
            num_changed_entities += adapter.SaveEntityCollection(disp);
            num_changed_entities += adapter.SaveEntityCollection(existing_disc);
            if (!append || num_changed_entities > 0)
               adapter.SaveEntity(_run);
            adapter.Commit();

Looking at my log (attached), I see the transaction and various things that make sense, but when it gets to 'Entity added to update queue:' there are a bunch of TubeBoardEntity interlopers. Is this normal? Shouldn't each adapter have it's own update queue? Perhaps its tied to Npgsql's connection pooling?

If this code is all new, then it might be a small mistake somewhere. I don't have overview of how you split your work between threads so I can't comment on that. All work that's passed to an adapter via an action, e.g. SaveEntity() or SaveEntityCollection, is processed into queues: all inserts are added to an insert queue and all updates are added to an update queue. Then first all inserts are handled and then all updates. This is per adapter. If you pass an entity to multiple adapters, it might end up in multiple queues. Be aware that adapters work on graphs, so if entities have relationships with one another, the adapter will traverse them all. So if entity X has a reference to Y, and you pass X to adapter A1 and Y to adapter A2, both X and Y are reachable by both adapters!

Also, logs aren't sequential in a multithreaded environment: it might be a line logged by thread A is after a line logged by thread B even tho it took place earlier in time, just because B was allowed to log a line while A was waiting for B to complete its logging work. I don't think it's npgsql related, as you get multithreading issues at the orm level.

I was working on a repro, but it's a very complex interaction with many processes all inserting and updating related bits of information. Re-reading about the graphs, I cut an FK from the entity that was causing trouble, which made it so that it wouldn't have to get deleted as often and would be less likely to get pulled in a bad direction. So far, so good.

It seems as though the graph went down to a low-level identification entity then back up to a mostly-unrelated other entity. Is there a way to force them not to be connected? Like turn off the 1-many relationship where it's not used?

If I understand you correctly, you can do that at design time from the Designer, you can remove any unwanted relations.

I guess my question is whether that would impact the graph logic to prevent bleeding between threads, and what negative impacts it might have.

tmatelich wrote:

I guess my question is whether that would impact the graph logic to prevent bleeding between threads, and what negative impacts it might have.

You can 'hide' a relationship from either side by deleting the navigator mapped onto the relationship. If the FK -> PK navigator is gone, PK fields won't be synced with FK fields when you insert new entities. If the PK -> FK navigator is gone, things work fine.

So say we have Customer.Orders 1:n Order.Customer. If I remove the 'Order.Customer' navigator, I can't save an Order unless I explicitly set the FK field in Order as the Customer.CustomerId isn't synced (as there's no Customer referenced from Order). If I remove Customer.Orders, this will work fine, I can set Order.Customer to a Customer instance.

There's also a method, DetachFromGraph you can call on an entity instance which detaches that entity from the graph it's in. This function isn't slow but still has to do some work, which might mean things run slower than expected if you call this method 1000s of times in a tight loop.

That said, you might want to reconsider where you want to place the boundaries for multi-threaded persistence. E.g. batching work is way more efficient than executing multiple threads with single queries, simply because locks on table pages won't be set that long. So persisting a bigger graph with batching switched on might be more efficient than your current setup: persist the graph on 1 thread which will collect all the work in the queues and with batching on will batch all inserts and updates on similar entities together and execute the batch as 1 query. the less amount of roundtrips to the DB, the lower amount of locks, it might surprise you how fast this is.

One key bug in my code was I had functions for cloning entities based on some old code I'd found in the forums years ago, my comment on the code was

   /// <summary>
   /// For cloning an Entity and all related entities, ie the whole graph
   /// actually, it doesn't seem to get all the related entities, not sure if it ever did
   /// http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=7568
   /// http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=20620
   /// http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=20818
   /// </summary>

When I upgraded, whatever may have been broken to make it not update the whole graph started working, or I previously didn't have any problematic entities in my graph. So, inadvertently, I was cloning entities which had unique constraints. I adjusted my use of this clone function to not traverse the whole graph.

And, I did have one place where I still ended up needing to DetachFromGraph.

Anyway, this release is finally out the door. Thanks for the help.