- Home
- LLBLGen Pro
- LLBLGen Pro Runtime Framework
CreatePackedQueryFromCollectedElements throws System.IndexOutOfRangeException
Joined: 05-Oct-2017
Build Version: 5.5 (5.5.0) RTM Build Date: 02-Nov-2018 Project: Adapter project targetting .NET 4.5.2 Database: MS SQL 2019
We have an integration service that creates or updates entities in our system after receiving data from external sources. Sometimes when we call Save Entity Collection with refetch and recurse set to true we get the following error. Any thoughts on what might be causing this?
System.Exception: Error executing {Our Method Here} ---> System.IndexOutOfRangeException: Index was outside the bounds of the array. at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.CreatePackedQueryFromCollectedElements() at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.ProduceElementsToRun(ActionQueueElement
1 actionQueueElement, IActionQuery query, Type typeOfNextElement) at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.Execute(ActionQueueElement
1 actionQueueElement, IActionQuery query, Type typeOfNextElement) at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.PersistQueue(List`1 queueToPersist, Boolean insertActions) at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.CommitImpl(IDataAccessAdapter adapterToUse, Boolean autoCommit) at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntityCollection(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse)
Joined: 21-Aug-2005
Could you please build your code against the ormsupport classes dll in the designer installation folder: Frameworks\LLBLGen Pro\RuntimeLibraries\CompiledRuntimeLibraries?
This one contains the .pdb file which should give us a line number, to start looking at.
Joined: 17-Aug-2003
Additionally, try at least to use the latest 5.5.x build to see if that solves your problem.
The .pdb files are also located on disk (for the release build assemblies), in \Frameworks\LLBLGen Pro\RuntimeLibraries\CompiledRuntimeLibraries
inside the installation folder.
Joined: 05-Oct-2017
Thanks for your replies! We updated this particular bit of code to save the entities individually and the errors seem to have gone away, but I will give this a try if they come back in other parts of our application. I'll also push for using the newest version if it comes back, but that call isn't something I can just make unilaterally at my organization.
Last question, will the included PDB files just work with the DLL we get out of NuGet (assuming we drop the PDB in the bin directory), or do I specifically need to build using the DLL from that location as well. We reference your NuGet package in our CI/CD process so it's a little complicated to add in a DLL that's not coming out of Nuget.
Joined: 28-Nov-2005
Thanks for your replies! We updated this particular bit of code to save the entities individually and the errors seem to have gone away, but I will give this a try if they come back in other parts of our application. I'll also push for using the newest version if it comes back, but that call isn't something I can just make unilaterally at my organization.
Ok. Please let us know so we can reproduce it in our end as well.
Last question, will the included PDB files just work with the DLL we get out of NuGet (assuming we drop the PDB in the bin directory), or do I specifically need to build using the DLL from that location as well. We reference your NuGet package in our CI/CD process so it's a little complicated to add in a DLL that's not coming out of Nuget.
Yes, you can put the .pdb where the nuget dll is. Just make sure the .pdb is from the same .dll build.
Joined: 05-Oct-2017
Hey, so it looks like we were using version 5.5.1 and not 5.5.0 so my apologies for that.
We just got what we think is the same error in another part of our application. Here is the stack trace:
System.IndexOutOfRangeException: Index was outside the bounds of the array. at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.CreatePackedQueryFromCollectedElements() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 366 at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.ExecutePending() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 121 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.PersistQueue(List
1 queueToPersist, Boolean insertActions) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 1283 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.CommitImpl(IDataAccessAdapter adapterToUse, Boolean autoCommit) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 784 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntityCollection(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 2439 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.<>c__DisplayClass20_0.<SaveEntityCollection>b__0() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 429 at Gravitate.Domain.Adapter.DAL.Extensions.AdapterExtensions.SaveEntityCollection[T](IDataAccessAdapter adapter, IEnumerable
1 entities, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\TeamCity\buildAgent\work\561ec3cdcea2c3bf
Joined: 05-Oct-2017
Sure!
I'm trying to set up a local environment that mirrored our server, but in the meantime, I've noticed the following similarities between this and the case from my message above:
- This code and the other instance both involved calling save entity collection with refetch set to false and recurse set to true.
- The entities here and in the other instance we were getting errors on both involved a parent/child structure (many children point to one parent)
- In both locations in our source there were lines of code that set the dirty flag on the parent to true if the parent was not new, and one of the child entities was new/dirty (I'm going to take a stab and guess this is bad practice)
This doesn't happen with each item, so maybe setting the dirty flag when an entity isn't actually dirty might cause issues when saving a collection.
Joined: 17-Aug-2003
mprothme wrote:
Hey, so it looks like we were using version 5.5.1 and not 5.5.0 so my apologies for that.
We just got what we think is the same error in another part of our application. Here is the stack trace:
System.IndexOutOfRangeException: Index was outside the bounds of the array. at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.CreatePackedQueryFromCollectedElements() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 366 at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.ExecutePending() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 121 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.PersistQueue(List
1 queueToPersist, Boolean insertActions) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 1283 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.CommitImpl(IDataAccessAdapter adapterToUse, Boolean autoCommit) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 784 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntityCollection(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 2439 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.<>c__DisplayClass20_0.<SaveEntityCollection>b__0() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 429 at Gravitate.Domain.Adapter.DAL.Extensions.AdapterExtensions.SaveEntityCollection[T](IDataAccessAdapter adapter, IEnumerable
1 entities, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\TeamCity\buildAgent\work\561ec3cdcea2c3bf
The weird thing is... this is ActionQueryController line 366 for v5.5.1
if(_actionQueueElementsWithQuery.Count <=0) // Line 366
{
return null;
}
So the pdb file you've stored in the app's executable folder might not be the one for 5.5.1 ?
Setting a dirty flag shouldn't make a difference btw. I still think it's multi-threading related, like you use the DataAccessAdapter instance in multiple requests. Be absolutely sure you recreate the DataAccessAdapter with each query, don't share that instance among requests: it's not thread safe plus it takes almost no time to recreate one so it's not useful.
Joined: 05-Oct-2017
So I pulled back the database and I was actually able to execute the job and reproduce this in a unit test. As far as I can tell nothing here is running multithreaded (though I have seen multithreading issues as you described before so I know what you're talking about). We also create the adapter in the job and since nothing runs concurrently I don't think that there's an opportunity for 2 save queries to hit it at once.
Just to double-check to make sure I set this up correctly, the version I see in the designer is as follows:
- Version: 5.5 (5.5.1) RTM
- Build Date: 17-Jan-2019
I opened the installation folder for the designer (LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\CompiledRuntimeLibraries) and just copied the following files into the bin folder of my unit test project (there aren't any other LLBLGen Pro DLL/pdb files there). Both DLL files have 5.5.1.0 on the details tab if you right-click and select properties.
- SD.LLBLGen.Pro.DQE.SqlServer.dll
- SD.LLBLGen.Pro.DQE.SqlServer.pdb
- SD.LLBLGen.Pro.ORMSupportClasses.dll
- SD.LLBLGen.Pro.ORMSupportClasses.pdb
When I execute the job I get the same track trace as before
Index was outside the bounds of the array. ---> System.IndexOutOfRangeException: Index was outside the bounds of the array. at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.CreatePackedQueryFromCollectedElements() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 366 at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.ExecutePending() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 121 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.PersistQueue(List
1 queueToPersist, Boolean insertActions) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 1283 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.HandleUpdates(DataAccessAdapterBase adapterToUseAsBase) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 1781 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.CommitImpl(IDataAccessAdapter adapterToUse, Boolean autoCommit) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 779 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.<>c__DisplayClass47_0.<Commit>b__0() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 696 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.Commit(IDataAccessAdapter adapterToUse, Boolean autoCommit) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 699 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntityCollection(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 2439 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.<>c__DisplayClass20_0.<SaveEntityCollection>b__0() in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 429 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.ExecuteWithActiveRecoveryStrategy[T](Func
1 toExecute) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 919 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.SaveEntityCollection(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\Myprojects\VS.NET Projects\LLBLGen Pro v5.5\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 429
Is there any other way I could check if my DLL/PDB Files aren't matching up?
Joined: 17-Aug-2003
Hmm. then the included pdb file is off. I've attached a debug build of the 5.5.1 sourcecode archive, with pdb. Could you try this one? It's signed so it should be a drop-in replacement. Please make sure the pdb file is overwritten with the one in the zip, also in the executing process (so don't copy it only in the generated code's bin folders)
Filename | File size | Added on | Approval |
---|---|---|---|
SD.LLBLGen.Pro.ORMSupportClasses.551.Debug.zip | 1,967,265 | 07-Apr-2021 10:01.41 | Approved |
Joined: 05-Oct-2017
So good news, I ran the tests with the DLL/PDB file and got the following different stack trace:
Index was outside the bounds of the array. ---> System.IndexOutOfRangeException: Index was outside the bounds of the array. at System.String.get_Chars(Int32 index) at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.CreatePackedQueryFromCollectedElements() in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 404 at SD.LLBLGen.Pro.ORMSupportClasses.ActionQueryController.ExecutePending() in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\Query\ActionQueryController.cs:line 121 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.PersistQueue(List
1 queueToPersist, Boolean insertActions) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 1325 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.HandleUpdates(DataAccessAdapterBase adapterToUseAsBase) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 1781 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.CommitImpl(IDataAccessAdapter adapterToUse, Boolean autoCommit) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 779 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.<>c__DisplayClass47_0.<Commit>b__0() in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 696 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.Commit(IDataAccessAdapter adapterToUse, Boolean autoCommit) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 699 at SD.LLBLGen.Pro.ORMSupportClasses.UnitOfWork2.Commit(IDataAccessAdapter adapterToUse) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\UnitOfWork\UnitOfWork2.cs:line 676 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterCore.SaveEntityCollection(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterCore.cs:line 2440 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.<>n__19(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.<>c__DisplayClass20_0.<SaveEntityCollection>b__0() in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 429 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.ExecuteWithActiveRecoveryStrategy[T](Func
1 toExecute) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 919 at SD.LLBLGen.Pro.ORMSupportClasses.DataAccessAdapterBase.SaveEntityCollection(IEntityCollection2 collectionToSave, Boolean refetchSavedEntitiesAfterSave, Boolean recurse) in C:\Temp\LLBLGenPro_v5.5.1_SourceCode\Frameworks\LLBLGen Pro\RuntimeLibraries\ORMSupportClasses\AdapterSpecific\DataAccessAdapterBase.cs:line 429
Joined: 17-Aug-2003
That's indeed more informative It dies here:
if(i < _actionQueueElementsWithQuery.Count - 1)
{
appendDelimiter = commandToAppend[commandToAppend.Length-1] != ';'; // <<<<<<< line 404
}
fragments.Append(commandToAppend);
if(appendDelimiter)
{
fragments.Append(";");
}
We'll see if we can reason about what might cause this. However if you have more info what data you feed the adapter it would be great. I.e. if you save 1000s of entities and only then it goes wrong it'll be hard to track down, if it also fails with a specific graph it's good that we have that so we can build a local test.
Looking at the code, I think it only fails if commandToAppend is an empty string (as length is then 0 so length-1 results in -1 which is of course out of bounds. This might occur when the DQE produces no query. We'll see when that happens.
(edit) I think I know when this happens: you said you marked entities as 'dirty'. So these end up in the query generator to produce a query for. However if they're not new, and no field has been changed, nothing has to be done (as no field has changed!) so no query is produced. Normally, the action query with an empty query string simply doesn't do anything, but in the case of a batch, it's 'packed' into the batch query with other queries. We don't have a test for that in the batcher, so we'll add one.
You can work around this by only marking fields dirty if you need to. Why do you mark entities as 'dirty' at all btw? If no field changes, you don't have to (they're marked as dirty when a field is changed automatically).
(edit) we reproduced it indeed with this assumption and have implemented a fix in the supported versions v5.7 and 5.8, which will be up shortly
Joined: 05-Oct-2017
Awesome, I'm glad you were able to reproduce it on your end!
As for why the code was that way, I'm honestly not 100% sure. I think at one point we ran into an issue where, for some reason, the recursive save on a parent entity seemed to skip over a dirty entity further down in the graph, and setting the parent entity's dirty flag fixed the issue.
That said, I think several changes were made at once to try to resolve the issue, so I'm not actually sure if setting the dirty flag was what fixed the bug, but since the app worked correctly afterward we left it alone.
From now on though we'll just avoid doing that, and hopefully I can use this as an additional reason to justify an upgrade on our end.
Thanks for your help!