We just deployed our first application using LLBL as the DAL. Everything was fine until we went to production and CPU started going through the roof. The code is on its way to profiling but wanted to see if anyone has any ideas as to why this may happen. \ LLBL v 2.6 final Oct 6, 2008 ORMSupportClasses 2.6.08.1013 LinqSupportClasses 2.6.08.1001 SqlServewrDQE 2.6.08.1006

Adapter Model.

We are using LINQ all over the place. Could it be that the LINQ portion is CPU intensive? We are using TypeConverters as well. Can these be the culprits? We set all the diagnostic settings to 0. Also, we have only one generic get function which wraps around the LinqMetaData wrapper. This returns an IQueryable to be used with other extension methods.

 public IQueryable<T> Get<T>() where T : Data.Document.EntityClasses.CommonEntityBase
        {
            
            return from data in DocumentLinqProxyRead.GetQueryableForEntity(
                       (int)Enum.Parse(typeof(Data.Document.EntityType), typeof(T).Name)
                       )as DataSource2<T>
                   select data;
        }

public IQueryable<T> Get<T>(Expression<Func<T, bool>> expression) where T:Data.Document.EntityClasses.CommonEntityBase
        {
            return from data in (DocumentLinqProxyRead.GetQueryableForEntity(
                      (int)Enum.Parse(typeof(Data.Document.EntityType), typeof(T).Name)
                      ) as DataSource2<T>)
                      .Where(expression)
                    select data
                
                   ;
        }

Once I get more details I will post but for now this is all I have. Any help is appreciated.

The linq provider is very quick (<1ms per query), so that's not it. Typeconverters are also very fast (unless you do massive complex things in them), so that's also not it. Unless profiling is done, i think it's not really possible to say what's wrong. What you could check if you know where linq queries are slow is whether the query is ran in the db or that it fetches all data to the client. Also if you use a lot of nested queries in the projection, this might cause some CPU load, as the merging of the queries has to be done on the client side.

Be sure to switch off ANY tracing in the config file, and try to identify places in your application where performance is slow, e.g. screen X gives problems, then start from there at the dev side, try to see if youre fetching a lot of data to the client. Also run a profiler to see whether you focus on the right code (e.g. it might be some processing code at the BL level is really the slow part).

Although you use an older runtime lib build, it shouldn't matter that much, only at the DB side, we tuned some DISTINCT emitting for some queries, which could make the execution of the queries faster.

(I assume your CPU usage is all eaten up by your application code, not by a shared db instance?)

(edit) also keep in mind that such a wrapper could mean that many extension methods are appended to a query, which might make it run in memory perhaps. If you could explain a bit more about your application (I assume a website?) it could also help. e.g. if you have a webservice, and it serializes a lot of data (e.g. you're pulling hundreds of thousands of entities over the wire, it will be slow obviously)

As far as queries running on db vs client, we paid extra attention to make sure we dont bring back the whole table and filter the data in memory, so, i am 99% sure this is not happening. BACKGROUND: We have several mvc.net and asp.net web applications. The apps are consuming a repository which wraps around the llbl db specific and returns llbl db generic entities. As I stated above, the repo has two generic get methods which return IQueriables. These Get<T>() methods are used in conjunction with extension methods. So a typical call will look like this (some have more extensions but i dont think we ever used over 10:

using (Data.DocumentRepository.Repos.DocumentRepository repo
    = new Data.DocumentRepository.Repos.DocumentRepository())
                    {
                        _allcats = repo.Get<hbec.SiteIndexCategoryEntity>()
                            .OrderBySequence()
                            .WithSubcategory()
                            .ToList();

                    }

public static IOrderedQueryable<ENTITIES.SiteIndexCategoryEntity> OrderBySequence(this IQueryable<ENTITIES.SiteIndexCategoryEntity> query)
        {
            return from sic in query
                   orderby sic.Sequence
                   select sic;
        }

 public static IQueryable<ENTITIES.SiteIndexCategoryEntity> WithSubcategory(this IQueryable<ENTITIES.SiteIndexCategoryEntity> query)
        {
            return query.WithPath(c => c.Prefetch(cc => cc.SiteIndexSubcategory));
        }

The DBA profiled the query execution and things look fine from there. Also, the actual performance of the web site is fine. The problem is that the production web servers have massive CPU spikes. We went from 10%-15% CPU utilization to 10%-100%. CPU is all over the place.

EDIT: The above query is cached so we are not fetching the whole table and its children every request.

The queries look fine.

Ok, as you didn't see massive spikes during development but do during production, this either is due to 1) the data processed by the application is much bigger (see below) or 2) you now have more connections hammering the website and therefore critical code paths which were not noticeable during development are now visible.

ad. 1: if you have much bigger sets of data to work with, please pay attention to what you're doing with web forms and databinding. If you're using llblgenprodatasource2 controls in your webforms, you bind data to the grids / controls, and the data bound is cached in for example the viewstate. If you use the viewstate setting, the data is serialized to the viewstate, which can be big and slow (cpu wise). this can be helped by switching ON fast serialization. Please test this in dev first. Fast serialization is a global setting, so it's 1 line and everything uses fast serialization. This is recommended for adapter. you should use this first in dev as you use an older runtime build and we had a tiny bugfix in that area, so it might be you run into this problem. (I've to add that the first concern is to keep the set bound to grids as small as possible of course)

It's also important that you identify slow webforms in production, so open forms (e.g. do what a normal web visitor would do) and check CPU utilization. If a webform takes 2-4 seconds with 100% cpu usage for example, you have a good candidate for investigation. Use Windows performance counters as well to verify which areas are doing a lot of work.

I'd also suggest using a performance profiler on your .net application to see where are those spikes comming from (if they are comming from .net at all). ANTS Performance Profiler or AQTime might be good options. There is also a performance profiler built in some VSTS SKUs.

Looks like it was nothing with LLBL . This release was supposed to be for switching from dataset DAL to LLBL/LINQ/REPOSITORY. It was some multi threading code that they had snuck into this release that caused the spikes. Sorry for false alarm. In the end, the CPU load did become 3-4% heavier vs the build with the old datasets dal, but the difference is nothing significant to us. The test was done on mediocre servers. In the end, the maintainability of the linq DAL is worth the slight overhead. BTW, I am not certain if the slight increase and how much of it is caused by the new DAL

The extra CPU is likely caused by the fact that more objects are created, query is generated, objects have to be filled etc. With datatables, they're very easy to fill them from a datareader (reader.GetValues() -> use the received array as a datarow in the datatable, that's pretty much it). So they have a slight advantage in that area (also the reason why they're a little bit quicker during fetching), however all the work to manage them is on the developer's shoulders, so it's a tradeoff

Glad it's solved! (as these things, especially on a friday, are never things you want to see happening when deploying an application )