LLBLGen version 2.0.0.0 December 6th, 2006 Runtime version 2.0.0.61205 Self-servicing template, .NET 2.0 SQL Server 2005

We have code deployed at many sites wihout issue. At one site we continually get various runtime exceptions when fetching, or even creating entities. The exceptions appear to be random and cannot be reliably replicated.

For example, when creating a new entity:

System.IndexOutOfRangeException: Index was outside the bounds of the array. at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ReadRowIntoFields(Object[] values, IEntityFields rowDestination, Dictionary`2 fieldIndexToOrdinal) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchOneRow(IDataReader dataSource, IEntityFields rowDestination) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ExecuteSingleRowRetrievalQuery(IRetrievalQuery queryToExecute, ITransaction containingTransaction, IEntityFields fieldsToFill) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.PerformFetchEntityAction(IEntity entityToFetch, ITransaction containingTransaction, IPredicateExpression selectFilter, IPrefetchPath prefetchPathToUse, Context contextToUse) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchExisting(IEntity entityToFetch, ITransaction containingTransaction, IPrefetchPath prefetchPathToUse, Context contextToUse) at Commissure.Data.EntityClasses.OrderEntityBase.Fetch(Int32 orderID, IPrefetchPath prefetchPathToUse, Context contextToUse) at Commissure.Data.EntityClasses.OrderEntityBase.InitClassFetch(Int32 orderID, IValidator validator, IPrefetchPath prefetchPathToUse) at Commissure.Data.EntityClasses.OrderEntity..ctor(Int32 orderID) at Commissure.Render.Report.Create(Int32 siteID, ArrayList orderIDs, Int32 signerAcctID, Boolean doLock, Boolean blockScheduled)

When retrieving an existing entity:

System.IndexOutOfRangeException: Index was outside the bounds of the array. at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ReadRowIntoFields(Object[] values, IEntityFields rowDestination, Dictionary`2 fieldIndexToOrdinal) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchOneRow(IDataReader dataSource, IEntityFields rowDestination) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ExecuteSingleRowRetrievalQuery(IRetrievalQuery queryToExecute, ITransaction containingTransaction, IEntityFields fieldsToFill) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.PerformFetchEntityAction(IEntity entityToFetch, ITransaction containingTransaction, IPredicateExpression selectFilter, IPrefetchPath prefetchPathToUse, Context contextToUse) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchExisting(IEntity entityToFetch, ITransaction containingTransaction, IPrefetchPath prefetchPathToUse, Context contextToUse) at Commissure.Data.EntityClasses.OrderEntityBase.Fetch(Int32 orderID, IPrefetchPath prefetchPathToUse, Context contextToUse) at Commissure.Data.EntityClasses.OrderEntityBase.InitClassFetch(Int32 orderID, IValidator validator, IPrefetchPath prefetchPathToUse) at Commissure.Data.EntityClasses.OrderEntity..ctor(Int32 orderID)

Do you have the stack trace which point to the class/function/line that throws the error. Without that it's it's just a guess.

Guess #1: this may be the problem Dictionary`2 fieldIndexToOrdinal. I have seen this with predicates. valid

List<int> ids = new List<int>();
ids.Add(1);
ids.Add(2);
EntityFields.MyField == ids;

invalid

IList<int> ids = new List<int>();
ids.Add(1);
ids.Add(2);
EntityFields.MyField == ids;

Guess #2: Is the site using the correct ORM version?

This error is thrown directly from a standard LLBLGen method call, in various scenarios, for example,

OrderEntity oe = new OrderEntity(1234);

where 1234 is a valid PK for this entity.

This same code runs successfully most of the time, and only fails intermittently, and at this one site. So I do not think we are looking at a coding error here.

Our client deployment ensures that the correct version of the ORM library is installed.

Both errors you have posted are related to the OrderEntity. I guess at this site the Order table definition doesn't accuratly match the generated code.

Would you please make sure that the Order Table DDL (schema definition) in this site's database , matches those at other sites (which has no errors).

If the Database schema are 100% identical, maybe the generated code is out dated.

This error actually occurs for other entities as well; the OrderEntity is just an example.

We also get another type of exception on a different type of entity:

System.InvalidCastException: Unable to cast object of type 'System.Int32' to type 'System.String'. at Commissure.Data.EntityClasses.PersonalInfoEntityBase.get_LastName()

As well as the same IndexOutOfRangeException on this same entity when accessed via a related entity:

System.IndexOutOfRangeException: Index was outside the bounds of the array. at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ReadRowIntoFields(Object[] values, IEntityFields rowDestination, Dictionary`2 fieldIndexToOrdinal) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchOneRow(IDataReader dataSource, IEntityFields rowDestination) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ExecuteSingleRowRetrievalQuery(IRetrievalQuery queryToExecute, ITransaction containingTransaction, IEntityFields fieldsToFill) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.PerformFetchEntityAction(IEntity entityToFetch, ITransaction containingTransaction, IPredicateExpression selectFilter, IPrefetchPath prefetchPathToUse, Context contextToUse) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchExisting(IEntity entityToFetch, ITransaction containingTransaction, IPrefetchPath prefetchPathToUse, Context contextToUse) at Commissure.Data.EntityClasses.PersonalInfoEntityBase.Fetch(Int32 personalInfoID, IPrefetchPath prefetchPathToUse, Context contextToUse) at Commissure.Data.EntityClasses.PatientEntityBase.GetSinglePersonalInfo(Boolean forceFetch) at Commissure.Data.EntityClasses.PatientEntityBase.get_PersonalInfo()

The schemas are identical and the generated code is also identical to that which is running successfully at other sites.

Also, these errors are intermittent. We get these exceptions only about 10 times per day, when the same exact code must run successfully hundreds if not thousands of times.

The best advice I can give you here is to refresh the lgp file an do a complete regeneration, later on replace the new code and refresh all references in your solution, rebuild and see if it works. I know this may sound tedious but seems to me that this an issue of a mismatch between the database schema and the existing code as Walaa pointed in the previous message.

I understand what you are saying, but if it is a mismatch, how can you explain that the exact same code runs fine once, and then fails the next?

I understand what you are saying, but if it is a mismatch, how can you explain that the exact same code runs fine once, and then fails the next?

It's hard to trace or explain an undeterministic issue. Needless to say that we are on the blind side here. In these cases, there must be other factors affecting the behaviour, but they are still hidden from you and from us.

So what we have guessed is based on the few inputs or factors that are known.

We can digg deep if you can supply us with a repro solution, which is a hard thing to ask, I know. Or you may try Gus(goose) solution.

Your query returns an int for a name field, so clearly the query generated returns wrong data for the entity to load. As queries use named fields, it's odd this happens.

One thing you could do is at least try to use the latest v2.0 build for the designer, runtime and templates (so the latest v2.0 installation).

Also, if you're using name overwriting in the config file, be absolutely sure you're connecting to the right catalog and overwriting it with the correct name. As you're using selfservicing, be absolutely sure that you're not setting the connection string at will, as it's global.

We are thinking that, since this happens only at one site, we are looking at some kind of database issue. The server is heavily loaded which could possibly cause the engine to return bad data, or maybe expose bugs in SQL Server or ADO.NET.

We took a backup of the same database and installed it in our lab and are not able to replicate the problem at all.

Hmm, could be indeed, as I have no other explanation why this happens. I'll close the thread for now, if you have new info, just post a new message in this thread and it'll automatically re-open and be added to our support queues

We finally created a special LLBL 2.5 build with debugging code inside ReadRowIntoFields, as Frans suggested, and shipped it to some customers. What happens is that columnOrdinal==values.Length. So this is the offending code:

value = values[columnOrdinal];

As DaveR said, this is intermittent and it has nothing to do with outdated schema or generated code. The app works fine 99.9% of the time, but occassional crashes like these happen from our WinForms application that connects to SQL Server 2005.

Would could cause columnOrdinal to go out of bounds? Shouldn't your code be more defensive and skip that ordinal instead of blindly referencing the values array?

obzekt wrote:

We finally created a special LLBL 2.5 build with debugging code inside ReadRowIntoFields, as Frans suggested, and shipped it to some customers. What happens is that columnOrdinal==values.Length. So this is the offending code:

value = values[columnOrdinal];

As DaveR said, this is intermittent and it has nothing to do with outdated schema or generated code. The app works fine 99.9% of the time, but occassional crashes like these happen from our WinForms application that connects to SQL Server 2005.

Would could cause columnOrdinal to go out of bounds?

the ordinal is calculated from meta-data so it can't go wrong, unless the calculation or the metadata is wrong.

Shouldn't your code be more defensive and skip that ordinal instead of blindly referencing the values array?

Why would it do that if this routine works with precalculated data? Mind you, this method is ran every time a row is read from the db. If it has to do a lot of checks which are redundant as it already knows the outcome, it is slower. The problem is also that if the ordinal is wrong, it is very likely values are fetched into the wrong slots.

As this thread is 2 years old, and you don't give any info, do you use adapter or selfservicing or do you work with DaveR on the same system? Let's assume the latter, so you should look at the code in DaoBase, method ExecuteMultiRowRetrievalQuery, line 1560 (for no inheritance), 1580 for inheritance entities and 2418 if you fetch a single row. The fieldindexToOrdinal collection which contains the ordinals, is created in line 1469-1528. We refactored this method a lot in 2.6, so the methods are much cleaner there.

As you can see, it DOES matter if there's corrupted ordinal data / field indexes, as these are used for the ordinals in the query: for normal entities, it's straight forward, but for inheritance entities, the fields for each entity are after eachother in the resultset. For each entity type in the inheritance chain, the field indexes to ordinal in the resultset are calculated and these indexes are then used to move a value from the row in the resultset to a field in the fields object. As these are pre-calculated, a check is redundant and won't matter: if something is off, more fields are off and values are fetched into the wrong fields.

So it's not something that's received from input, it's all based on meta-data that is correct (as it's in the generated code), and the data used for the fetch is calculated.

I hope this helps you track down the origin of the problem: The stacktrace should suggest a location in YOUR code which triggered this call to ReadRowIntoFields. That then should be used to verify whether you either do something like adding custom fields to the query, or do something very simple and we can look from there. Just adding a check isn't going to cut it, because for example if the method fetches 10 fields and from field 4 and up everything is 1 index off, 6 fields will get the wrong value, an if won't prevent that .

Thanks for the quick reply. We use SelfServicing and the code is very straightforward. A single entity is retrieved and no multi-threading is involved. We have an Order table with 30 plain fields (int, datetime, varchar). That table has not changed in a long time and it is being queried constantly, so the metadata you refer to should be fine, otherwise the system would not function at all. It also happens for a couple of other popular tables. Here's the code we added to DaoBase.cs to debug it:

if(columnOrdinal >= 0)
{
     //*** Custom code
     // Track source of IndexOutOfRangeException
     if (columnOrdinal >= values.Length)
    {
    string vals = "";
    for (int vv = 0; vv < values.Length; vv++)
    {
        string str = values[vv].ToString();
        if (str.Length > 30)
            str = str.Substring(0, 30) + "...";
        vals += str + ",";
    }
    vals = vals.Substring(0, vals.Length - 1);
    throw new InvalidOperationException(string.Format("Error: columnordinal={0}; rowordinal={1}; rowcount={2}; valuecount={3}; values: {4}", columnOrdinal, pair.Value1, rowDestination.Count, values.Length, vals));
    }
    //*** End of custom code
    
    value = values[columnOrdinal];

....

And here's the new stack trace when the ordinal goes out of bounds:

System.InvalidOperationException: Error: columnordinal=29; rowordinal=29; rowcount=30; valuecount=29; values: 1965488,1,7,53,22,,,,{\rtf1\ansi\ansicpg1252\deff0...,CLINICAL HISTORY: Knee injury....,,2,False,False,False,True,False,7/5/2009 12:07:53 AM,7/5/2009 12:30:56 AM,,False,1965488,114,91,139,0,,74,False at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ReadRowIntoFields(Object[] values, IEntityFields rowDestination, List`1 fieldIndexToOrdinal) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchOneRow(IDataReader dataSource, IEntityFields rowDestination, IFieldPersistenceInfo[] fieldPersistenceInfos) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.ExecuteSingleRowRetrievalQuery(IRetrievalQuery queryToExecute, ITransaction containingTransaction, IEntityFields fieldsToFill, IFieldPersistenceInfo[] fieldPersistenceInfos) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.PerformFetchEntityAction(IEntity entityToFetch, ITransaction containingTransaction, IPredicateExpression selectFilter, IPrefetchPath prefetchPathToUse, Context contextToUse, ExcludeIncludeFieldsList excludedIncludedFields) at SD.LLBLGen.Pro.ORMSupportClasses.DaoBase.FetchExisting(IEntity entityToFetch, ITransaction containingTransaction, IPrefetchPath prefetchPathToUse, Context contextToUse, ExcludeIncludeFieldsList excludedIncludedFields) at Commissure.Data.EntityClasses.OrderEntityBase.Fetch(Int32 orderID, IPrefetchPath prefetchPathToUse, Context contextToUse, ExcludeIncludeFieldsList excludedIncludedFields) at Commissure.Data.EntityClasses.OrderEntityBase.InitClassFetch(Int32 orderID, IValidator validator, IPrefetchPath prefetchPathToUse) at Commissure.Data.EntityClasses.OrderEntity..ctor(Int32 orderID)

As it happens when fetching a single entity (hence the call to FetchOneRow()), can you reproduce this every time? If so, could you create a reprocase for us? As always, please use the latest runtime libs for 2.5, if you don't do that already

I think we mentioned this already.. This problem is not reproducible. It happens only on production environments intermittently, like once a day or once a week, while that code is executed hundreds or thousands times per day.

obzekt wrote:

I think we mentioned this already.. This problem is not reproducible.

I don't know who 'we' is, obzekt. You re-open a 2-year old thread and don't explain much in the new post, like if you work on the same application as topicstarter or not. Sorry if I sound a bit blunt, but we don't know what you know about your code, all we know is that you're the only person with this error.

It happens only on production environments intermittently, like once a day or once a week, while that code is executed hundreds or thousands times per day.

Hmm. that suggests that it's not a deterministic problem, e.g. not related to corrupt input data, as that would otherwise result in a problem that's reproducable, but seems more related to threading, even though you say you don't use threading. Do you use this in a website scenario? If so, do you share any objects among threads (like in the application object). I know this sound like a guessing game, but you're the only one who reports this problem and we can't otherwise think of any cause of this: it either works always as you already guessed, or never.

If you look at the code in FetchOneRow, you'll see that the Value2 value of the pair (which becomes the columnOrdinal value) is always smaller than the count of rowDestination, the list of fields to fill. Those same fields were used to produce the query.

It looks like as if the query was created with 30 fields to fill, but the resultset that comes back from the DB has 29 columns.

Exactly, the resultset is one field smaller for some unexplained reason. It's not a web application, it happens on a WinForms program that connects to SQL. And it's not related to threading since that OrderEntity that failed is a new object created by the main UI thread.

I opened this forum thread again because we (DaveR and I) only recently had the opportunity to upgrade some customers to the new version of our application that contains the debugging code showing exactly where the crash occurs.

Another problem we have (also intermittent) is InvalidCastExceptions, typically converting datetime fields. For example, we have this very basic SP:

SELECT GETDATE() AS CurrentDateTime

Then in code we do this:

object o = RetrievalProcedures.GetCurrentDateTime().Rows[0][0];
if (o is DateTime)
    return (DateTime)o;

throw new InvalidOperationException(string.Format("sp_GetCurrentDateTime returned {0}", (null == o) ? "null" : o.ToString()));

And here's what we get:

System.InvalidOperationException: sp_GetCurrentDateTime returned 1360

It makes no sense to get that number from GETDATE(). Sometimes it doesn't even get to our code and it crashes with this stack trace:

System.IndexOutOfRangeException: There is no row at position 0. at System.Data.RBTree1.GetNodeByIndex(Int32 userIndex) at System.Data.RBTree1.get_Item(Int32 index) at System.Data.DataRowCollection.get_Item(Int32 index) at Commissure.Data.Database.get_CurrentDateTime()

I'm not an expert on this, but I suspect that some network problems corrupt the data that comes from SQL (or there is a bug in some driver) and they manifest themselves as errors originating from LLBL. I'm not saying LLBL has a bug, and as you wrote, you can't cover up the problem of indices going out of bounds. But it is a problem that affects our customers, and we are unable to replicate it in our lab. Our plan is to drop LLBL usage from the client and use a WCF business layer.

Note that apart from that WinForms app connecting to SQL over the LAN, we have other server applications (a website and an NT Service) that use the same exactly LLBL assemblies but run side-by-side SQL server (same box). These applications have NEVER crashed with these errors. That's why the suspicion goes on the network.

Indeed very strange problems... I agree that it could only be some kind of network error or a low-level driver problem. All data reads on .NET are always done over datareaders, which are basicly cursors on the server. It might be that under stress (it sounds that it occurs only under stress conditions), the tempdb or the database server makes mistakes in these situations. Do you have all sqlserver service packs installed?

Network issues are however not that likely if you think about the transport: it's TCP/IP, which is connection oriented so no corrupt data. or do you use named pipes across the network?

If I have to look into something for you, please let me know.