Hi,

I have a query which results in a farily normal object graph, with quite some relations, many-to-many constructs, and a prefetch path of about 4 levels deep, but the actual data is quite limited in size.

It takes LLBLGen 13 queries which results in 27 fetched rows (total rows of all queries). So fare nothing new, and no problems.

However, when I send this over a WCF Soap channel (which works) the whole object graph explodes into an amazing 14000 lines, 1.6MB soap message, with more than 30 nested levels of XML elements, and loads of duplicate object references:

<AircraftEntity Ref="68b3b988-a12d-4e16-81dd-9e171bd5daa6"></AircraftEntity>
<AircraftEntity Ref="22f34798-e82d-474e-93f1-50b38c095b4f"></AircraftEntity>
<AircraftEntity ObjectID="bebf8bca-376f-4530-834f-e18dfd40b592">
... 
</AircraftEntity

All object references seem to be in there a couple of hundred times.

Is there an alternative strategy for sending object graphs over WCF Services?

I'm using LLBLGen 2.5, Adapter Templates.

Which LLBLGen Pro runtime library version are you using? (Consult: http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=7722)

Walaa wrote:

Which LLBLGen Pro runtime library version are you using? (Consult: http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=7722)

Hi Walaa,

I'm using: LLBLGen 2.5 Final (August 28th 2007) SD.LLBLGen.Pro.ORMSupportClasses.NET20 v2.0.50727 SD.LLBLGen.Pro.DQE.SqlServer.NET20 v2.0.50727

I'm using Visual Studio 2008, target platform .NET 3.0.

The return type is an inherited type:

LeaseDetailEntity -> ContractEntity

Do you need the entire .lgp file to see all relations etc.?

Thank you,

Best Regards,

1-

LLBLGen 2.5 Final (August 28th 2007)

You are using an early release of the runtime libraries. Would you please download and use the latest release, just to cover this possibility.

2- Please try the following code and check the output xml string, to see what size it is and what's written in there

string xml = string.Empty;
myCollection.WriteXml(XmlFormatAspect.Compact25 | XmlFormatAspect.DatesInXmlDataType | XmlFormatAspect.MLTextInCDataBlocks, out xml);

3- Please post the client code showing the channel creation and the server calls.

Walaa wrote:

1-

LLBLGen 2.5 Final (August 28th 2007)

You are using an early release of the runtime libraries. Would you please download and use the latest release, just to cover this possibility.

2- Please try the following code and check the output xml string, to see what size it is and what's written in there

string xml = string.Empty;
myCollection.WriteXml(XmlFormatAspect.Compact25 | XmlFormatAspect.DatesInXmlDataType | XmlFormatAspect.MLTextInCDataBlocks, out xml);

3- Please post the client code showing the channel creation and the server calls.

I'm sorry to have posted the .NET runtime version there instead of LLBLGen version .

I've downloaded the latest release, regenerated the code, and referenced the new runtime libraries (dec. 16th), this results in a way way smaller request (51Kb), which is about what I expected it to be.

I didn't think it would make that much of a difference...

Thanks a lot Walaa

Hi Walaa,

Appeareantly I wasn't completely right.

When sending an IList<LeaseDetailEntity> over WCF with 10 records + prefetch paths, total graph is about 187 database records) the message becomes about 1.3MB (details_wcf.xml).

However serialized as you suggested, it is about 164kb (details_serialized.xml).

I've attached the resulting files, it seems that as the entities all reference eachother, they get duplicated a lot, meaning that each added entity exponentionally increases the size of the WCF message.

this is the actual code calling the service:

            using (ContractsServiceClient _service = new ContractsServiceClient())
            {
                LeaseDetailEntity[] contractsByMsn = _service.GetContractsByMsn("0", new ListFilter());
                RenderView("List", contractsByMsn);
            }

client configuration:

<system.serviceModel>
    <diagnostics wmiProviderEnabled="false" performanceCounters="All">
      <messageLogging logMalformedMessages="false" logMessagesAtServiceLevel="false" logMessagesAtTransportLevel="false"/>
    </diagnostics>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>
    <bindings>
      <wsHttpBinding>
        <binding name="WSHttpBinding_IContractsService" closeTimeout="00:01:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00" bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard" maxBufferPoolSize="4194304" maxReceivedMessageSize="1524288" messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true" allowCookies="false">
          <readerQuotas maxDepth="128" maxStringContentLength="8192" maxArrayLength="16384" maxBytesPerRead="4096" maxNameTableCharCount="16384"/>
          <reliableSession ordered="true" inactivityTimeout="00:10:00" enabled="false"/>
          <security mode="Message">
            <transport clientCredentialType="Windows" proxyCredentialType="None" realm=""/>
            <message clientCredentialType="Windows" negotiateServiceCredential="true" algorithmSuite="Default" establishSecurityContext="true"/>
          </security>
        </binding>
      </wsHttpBinding>
    </bindings>
    <client>
      <endpoint address="http://localhost/ContractServices/ContractsService.svc" binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_IContractsService" contract="ContractsService.IContractsService" name="WSHttpBinding_IContractsService">
        <identity>
          <dns value="localhost"/>
        </identity>
      </endpoint>
    </client>
  </system.serviceModel>

all communication is done over a simple wsHttpBinding.

The ServiceContract is defined as this:

    [ServiceContract()]
    public interface IContractsService
    {
        [OperationContract]
        IList<LeaseDetailEntity> GetContractsByMsn(string msn, ListFilter filter);
    }

The service is defined like this:

    public class ContractsService : IContractsService
    {
        IList<LeaseDetailEntity> IContractsService.GetContractsByMsn(string msn, ListFilter filter)
        {
            if (string.IsNullOrEmpty(msn))
                throw new ArgumentOutOfRangeException("msn", msn);

            using (IDataAccessAdapter adapter = DataAccessAdapter.GetDataAccessAdapter())
            {
                EntityCollection<LeaseDetailEntity> leaseDetailEntities = new EntityCollection<LeaseDetailEntity>();

                RelationPredicateBucket bucket = new RelationPredicateBucket();
                bucket.Relations.Add(LeaseDetailEntity.Relations.ContractItemEntityUsingContractId, JoinHint.Inner);
                bucket.Relations.Add(ContractItemEntity.Relations.AircraftEntityUsingAircraftId, JoinHint.Inner);
                bucket.PredicateExpression.Add(AircraftFields.Msn % (string.Format("{0}%", msn)));

                adapter.FetchEntityCollection(leaseDetailEntities, bucket, filter.MaxNumberOfItems, null, LeaseDetailEntity.PrefetchPathFull, filter.PageNumber, filter.PageSize);

                return leaseDetailEntities;
            }
        }
    }

and this is how it's configured:

<system.serviceModel>
    <diagnostics performanceCounters="All">
      <messageLogging logEntireMessage="true" logMalformedMessages="true"
        logMessagesAtServiceLevel="true" logMessagesAtTransportLevel="true"
        maxSizeOfMessageToLog="5262144" />
    </diagnostics>
    <services>
      <service behaviorConfiguration="CompanyName.CMS.Services.ContractServices.ContractsServiceBehavior" name="CompanyName.CMS.Services.ContractServices.ContractsService">
        <endpoint address="" binding="wsHttpBinding" name="CompanyNameHttp" contract="CompanyName.CMS.Services.ContractServices.IContractsService">
          <identity>
            <dns value="localhost" />
          </identity>
        </endpoint>
        <endpoint address="mex" binding="mexHttpBinding" name="CompanyNameMex" contract="IMetadataExchange" />
      </service>
    </services>
    <behaviors>
      <serviceBehaviors>
        <behavior name="CompanyName.CMS.Services.ContractServices.ContractsServiceBehavior">
          <!-- To avoid disclosing metadata information, set the value below to false and remove the metadata endpoint above before deployment -->
          <serviceMetadata httpGetEnabled="true"/>
          <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
          <serviceDebug includeExceptionDetailInFaults="false"/>
        
        </behavior>
      </serviceBehaviors>
    </behaviors>
  </system.serviceModel>

Thats about all the information I can have, any ideas what I could change to optimize the messages?

Sorry for the delay, we had somewhat of a hard time finding what's wrong. Looking into it.

I think the main issue is that the IList<> is serialized by .NET, and it calls for every entity IXmlSerializable.WriteXml(), but as a separate graph. This means that if an entity is referenced by more than one entity in the IList<> it is serialized more than once.

With sending an entitycollection across the wire, this is in 1 graph and therefore every entity which is referenced by more than 1 entity in the entitycollection is still serialized once, as it's considered the same graph: the list of entities already processed is used for ALL entities in the entity collection.

When sending the IList<>, every entity in it is serialized separately by .NET, and therefore the list of processed entities for the graph of entity [0], isn't shared with the serialization of entity [1] etc.

So you should send an entity collection across the wire and specify that in your interface as well. If you don't do that, .NET will perform its own serialization mechanism (so serialize every entity separately)

Otis wrote:

Sorry for the delay, we had somewhat of a hard time finding what's wrong. Looking into it.

I think the main issue is that the IList<> is serialized by .NET, and it calls for every entity IXmlSerializable.WriteXml(), but as a separate graph. This means that if an entity is referenced by more than one entity in the IList<> it is serialized more than once.

With sending an entitycollection across the wire, this is in 1 graph and therefore every entity which is referenced by more than 1 entity in the entitycollection is still serialized once, as it's considered the same graph: the list of entities already processed is used for ALL entities in the entity collection.

When sending the IList<>, every entity in it is serialized separately by .NET, and therefore the list of processed entities for the graph of entity [0], isn't shared with the serialization of entity [1] etc.

So you should send an entity collection across the wire and specify that in your interface as well. If you don't do that, .NET will perform its own serialization mechanism (so serialize every entity separately)

Hi Frans,

Thank you for the answer and, yes that's what we tried initially, however with this scenario the WCF client code generator doesn't seem to like the generic EntityCollections, and generates weakly typed datasets (they're probably already not strongly typed available in the WSDL?).

Actually, thinking back to the beginning of creating the services, the LLBLGen entities turn into weakly typed datasets as well if we don't reuse the DAL assembly on the client side (bad practice anyway).

Currently we've decided to change our service to not pass entities over the service boundary, but use intermediate objects.

Partially because of these issues, but also to be able to preserve backwards compatibility in case the database (thus the entity model) entity changes, and we do need to stay in control of the datacontracts.

This will generate a little bit more work, especially with updates but it's more compliant to best practices I guess.

Thanks!

Wiebe Tijsma

Generic collections are indeed not supported in XML webservice scenario's, as the type parameter determines what class to instantiate at the other end and this isn't supported by .NET, so it will revert to datasets.

If you send entities over the wire, you indeed have to use the llblgen pro assemblies on the other side as well, and if that's a problem, use DTO's which are filled by projections. You could also generate the XML first on the server and send that over using compact25 XML, which is pretty much as compact as you want it to be

Otis wrote:

Generic collections are indeed not supported in XML webservice scenario's, as the type parameter determines what class to instantiate at the other end and this isn't supported by .NET, so it will revert to datasets.

If you send entities over the wire, you indeed have to use the llblgen pro assemblies on the other side as well, and if that's a problem, use DTO's which are filled by projections. You could also generate the XML first on the server and send that over using compact25 XML, which is pretty much as compact as you want it to be

Ow I didn't know you could project to custom classes, that's quite useful, looks a bit complex (I see a lot of string constants or reflection coming up).

There might be a way with LINQ to simplify Entity<>DTO mapping scenario's (will it be in the next LLBLGen version that supports LINQ?)

Can the DataProjectorToCustomClass also "project" the data back into the EntityView<TEntity> with the same EntityPropertyProjector instances?

Otherwise it seems easier to just transfer the entityvalues to the DTO in code... (dto.FieldValue= entity.FieldValue)

Thanks!

Wiebe

Can the DataProjectorToCustomClass also "project" the data back into the EntityView<TEntity> with the same EntityPropertyProjector instances?

You'd have to create your own adapter to project from a collection of custom class objects to an entity collection.

Walaa wrote:

Can the DataProjectorToCustomClass also "project" the data back into the EntityView<TEntity> with the same EntityPropertyProjector instances?

You'd have to create your own adapter to project from a collection of custom class objects to an entity collection.

Ok thanks...

wtijsma wrote:

Otis wrote:

Generic collections are indeed not supported in XML webservice scenario's, as the type parameter determines what class to instantiate at the other end and this isn't supported by .NET, so it will revert to datasets.

If you send entities over the wire, you indeed have to use the llblgen pro assemblies on the other side as well, and if that's a problem, use DTO's which are filled by projections. You could also generate the XML first on the server and send that over using compact25 XML, which is pretty much as compact as you want it to be

Ow I didn't know you could project to custom classes, that's quite useful, looks a bit complex (I see a lot of string constants or reflection coming up).

It's rather straightforward, you can obtain projection objects from an entity's fields object so if you're projecting to a DTO with similar field names, it's very easy

There might be a way with LINQ to simplify Entity<>DTO mapping scenario's (will it be in the next LLBLGen version that supports LINQ?)

v2.6 will support linq indeed and yes, it's very easy to fill DTO's with data using linq queries

Can the DataProjectorToCustomClass also "project" the data back into the EntityView<TEntity> with the same EntityPropertyProjector instances?

Otherwise it seems easier to just transfer the entityvalues to the DTO in code... (dto.FieldValue= entity.FieldValue)

It is done in general by a bunch of generated classes into which you set a DTO and an entity comes out