Hey all,

We finally switched from LLBL 1.0.2003.3 to LLBL 1.0.2005.1 on our server... Using SQLServer 2000, .NET 2.0, IIS 6.0

The code has not changed, but now we are having some serious performance issues.

The app involved uses concurrent threads and no transactions.

Current code built with LLBL 1.0.2003 does not show this performance problem.

Thanks,

Frannie

'performance issues' is too general. Please check what takes the resources on the server: did you switch to .NET 2.0 as well? Have you checked the windows performance counters to see which part takes a lot of CPU/Memory?

1.0.2005.1 supports tracing, please enable tracing on the DQE and check whether the app performs a lot of queries or not. You likely use selfservicing, which might be a reason, though I find it a bit strange the performance problems weren't there in 1.0.2003.3

We've been using .NET 2.0 for a while... and yes, we are also using self-servicing...

Here's some test code we came up with to show some performance differences between 2003 and 2005... this test code doesn't hit the db at all... There's nothing special about our PendingUpdates table... 10 columns, no blobs, etc.

int numEnts = 10000; IEntity [] entities = new PendingUpdatesEntity[numEnts];

Console.WriteLine("Creating " + numEnts + " entities"); long startTicks = DateTime.Now.Ticks;

for(int i=0; i< numEnts; i++) { entities[i] = new PendingUpdatesEntity(); }

long _ticCreateEntities = DateTime.Now.Ticks - startTicks; Console.WriteLine("CreateEntities ticks = " + _ticCreateEntities);

running test app with 2003 dac (1 instance)... Creating 10000 entities CreateEntities ticks = 2031237

running test app with 2005 dac (1 instance)... Creating 10000 entities CreateEntities ticks = 3281229

when running 4 instances of the test app... 2003 CreateEntities ticks = 3281229, 3437478, 3437378, 3281229 2005 CreateEntities ticks = 5312466, 5312466, 5156217, 5156217

Does LLBL 2005 just take longer to create entities than 2003? Is there a better way to do something like this?

Thanks,

Frannie

(edited, I removed the GC.Collect, as it seems to randomly slow down the code).

Interesting...

One thing I can think of is that in 1.0.2005.1 inheritance has been implemented which has extra overhead, but it shouldn't be that noticable. In v2.0 we made a lot of changes to the field construction which was the bottleneck in creating entities. I'll see if I can reproduce your numbers with code from 1.0.2003.3, 1.0.2005.1 and 2.0

(edit): Testcode:

class Class1
{
    /// <summary>
    /// The main entry point for the application.
    /// </summary>
    [STAThread]
    static void Main(string[] args)
    {
        for(int i = 0; i < 5; i++)
        {
            DoTest();
        }
    }

    private static void DoTest()
    {
        int numEnts = 10000;
        IEntity[] entities = new CustomerEntity[numEnts];

        Console.WriteLine("Creating " + numEnts + " entities");
        Stopwatch sw = new Stopwatch();
        sw.Start();
        for(int i = 0; i < numEnts; i++)
        {
            entities[i] = new CustomerEntity();
        }
        sw.Stop();
        Console.WriteLine("CreateEntities ticks = {0}, which is {1}ms", sw.ElapsedTicks, sw.ElapsedMilliseconds);

        OrderDetailsCollection orderDetails = new OrderDetailsCollection();
        sw.Reset();
        sw.Start();
        orderDetails.GetMulti(null);
        sw.Stop();
        Console.WriteLine("Fetching {0} orderdetails took: {1}ms", orderDetails.Count, sw.ElapsedMilliseconds);
    }
}

1.0.2003.3 Creating 10000 entities CreateEntities ticks = 806525670, which is 269ms Fetching 2015 orderdetails took: 173ms Creating 10000 entities CreateEntities ticks = 945143932, which is 315ms Fetching 2015 orderdetails took: 36ms Creating 10000 entities CreateEntities ticks = 956866327, which is 319ms Fetching 2015 orderdetails took: 140ms Creating 10000 entities CreateEntities ticks = 867899085, which is 290ms Fetching 2015 orderdetails took: 36ms Creating 10000 entities CreateEntities ticks = 722720257, which is 241ms Fetching 2015 orderdetails took: 195ms

1.0.2005.1 Creating 10000 entities CreateEntities ticks = 1334332035, which is 445ms Fetching 2015 orderdetails took: 333ms Creating 10000 entities CreateEntities ticks = 1447236570, which is 483ms Fetching 2015 orderdetails took: 326ms Creating 10000 entities CreateEntities ticks = 1221540143, which is 408ms Fetching 2015 orderdetails took: 275ms Creating 10000 entities CreateEntities ticks = 1256965732, which is 420ms Fetching 2015 orderdetails took: 311ms Creating 10000 entities CreateEntities ticks = 1180931325, which is 394ms Fetching 2015 orderdetails took: 297ms

2.0 Creating 10000 entities CreateEntities ticks = 1264394182, which is 422ms Fetching 2015 orderdetails took: 352ms Creating 10000 entities CreateEntities ticks = 1207632593, which is 403ms Fetching 2015 orderdetails took: 73ms Creating 10000 entities CreateEntities ticks = 1239495562, which is 414ms Fetching 2015 orderdetails took: 73ms Creating 10000 entities CreateEntities ticks = 1345158007, which is 449ms Fetching 2015 orderdetails took: 74ms Creating 10000 entities CreateEntities ticks = 995612235, which is 332ms Fetching 2015 orderdetails took: 244ms

It's still an odd result. I'll see if I can dig up why the 1.0.2003.3 fetch /create (which is running into the same roadblocks) is that much faster, as it apparently doesn't run into some code the later versions do.

I do know that with inheritance support we had to introduce a couple of routines internally for support for this in the fields collection which does increase the execution time abit.

One thing I want to rule out is that the 1.0.2003.3 runtime I'm using here is build on .net 1.x which might have introduced faster IL as it's a debug build after all.

Also weird is that when I test it with Stopwatch, the v2.0 code is slower in creating the entities, while it is the same code!

HOWEVER, what I wonder is, this can't be a bottleneck in your system, as saving a lot of data is much slower than creating a couple of entities. So what exactly is your system doing?

When I exclude the fetch:

1.0.2003.3 Creating 10000 entities CreateEntities ticks = 895479420, which is 299ms Creating 10000 entities CreateEntities ticks = 1053839812, which is 352ms Creating 10000 entities CreateEntities ticks = 1122706987, which is 375ms Creating 10000 entities CreateEntities ticks = 931370730, which is 311ms Creating 10000 entities CreateEntities ticks = 1085763780, which is 362ms

1.0.2005.1 Creating 10000 entities CreateEntities ticks = 1305067290, which is 436ms Creating 10000 entities CreateEntities ticks = 1517936925, which is 507ms Creating 10000 entities CreateEntities ticks = 1800407205, which is 601ms Creating 10000 entities CreateEntities ticks = 1542786405, which is 515ms Creating 10000 entities CreateEntities ticks = 1608310980, which is 537ms

2.0 Creating 10000 entities CreateEntities ticks = 1329900525, which is 444ms Creating 10000 entities CreateEntities ticks = 1081409715, which is 361ms Creating 10000 entities CreateEntities ticks = 1123246200, which is 375ms Creating 10000 entities CreateEntities ticks = 1140581730, which is 381ms Creating 10000 entities CreateEntities ticks = 1190299327, which is 397ms

so indeed 1.0.2005.1 is too slow. V2.0 has a different architecture than v1.0.200x.y, so it has a much lower memory footprint for entities and also is therefore faster than its predecessor. v1.0.2003.3 doesn't have to take care of inheritance, so it doesn't have to check if a field added to a fieldscollection is perhaps overloaded by a subtype. This tiny check, which does use a dictionary/hashtable, is still a noticable call when creating a lot of entities.

(edit). Profiling 1.0.2003.3 and 1.0.2005.1 and v2.0, indeed reveals where the bottleneck is: for every field added to EntityFields, 1.0.2003.3 doesn't have to store the name of the entity the field belongs to (this is for inheritance query building), which is done by 1.0.2005.1. v2.0 uses a cached set so this isn't a problem there. 1.0.2005.1 and 2.0 both still have to correct the alias if the field is overloaded (so inheritance hierarchy, both fields are called 'ID', the supertype then gets as alias <entityname>_ID). This routine is pretty fast though with a lot of entities you notice it's there. In 1.0.2003.3 it's not there as 1.0.2003.3 doesn't support inheritance.

Thanks for the details and confirmation that what we're seeing is expected. We should be able to refactor our application to avoid the problem or simply accelerate the upgrade to 2.0.

To answer your question - this particular application does almost no saves to the db - it is mainly fetching subsets of data then saving the row data in an XML form that we distribute to other client applications. We haven't seen any issues in the applications which do a mix of reading and writing to the database.

The impact is not measurable on our test system - it was only once we were attempting to serve several hundred requests at once that the problem was visible (which makes sense given deadlocks on the dictionary/hashtable occuring between the threads).

Thanks again for the confirmation.

Frannie