I've noticed that serializing 5000+ entities can take a long time (10 seconds or so typically). I've seen in other threads that its "not a good idea" to be remoting these many entities around, but I would like to know if anyone has come up with a EntityFormatter or something like that that can serialize entities better than the standard way. I've looked at the custom serialization of entity, entitycollection, etc and cannot see any obvious solutions, but then again serialization is a new topic for me and I don't know all the tricks out there. Any ideas?

You may try the v2 projection feature to create a projection on new custom classes which are very tiny and use those as DTO's (data transfer objects).

To try this out please download the v.2 beta. and for more information please refer to the LLBLGen Pro v.2 documentation "v.2 beta docs -> EntityViews -> Projections"

Walaa wrote:

You may try the v2 projection feature to create a projection on new custom classes which are very tiny and use those as DTO's (data transfer objects).

To try this out please download the v.2 beta. and for more information please refer to the LLBLGen Pro v.2 documentation "v.2 beta docs -> EntityViews -> Projections"

So the DTOs get serialized instead of the entities? From what I can tell they are strongly typed datacarriers that record the state of an entity's fields object. Is this correct? Isn't this what the custom serialization in the 2005 LLBLGEN does anyways, minus the strongly typing?

I looked throught the custom serialization code of the 2005 version and it looks like it is only the state that is getting serialized...I guess I'm just confused how DTOs can improve this. Is it that using DTOs to record state is faster than the normal custom serialization process?

Binary serialization won't be that much faster. Xml serialization will, as you can use the XmlSerializer serialization which generates special code to emit the xml.

Otis wrote:

Binary serialization won't be that much faster. Xml serialization will, as you can use the XmlSerializer serialization which generates special code to emit the xml.

We use binary serialization

Most of our remoting performance hit comes from the serialization/deserialization process instead of the actual transfer of data over the wire. The processors on both the server and client machines (2 tier architecture) go crazy trying to break down and recreate these entities.

I am honestly at a loss as to why this takes so long though, because the initial creation of the entities by the DataAccessAdapter happens relatively quickly...making me think that the deserialization of the graph should happen at least in the same amount of time. Am I missing something?

Another thing. I've profiled a BinaryFormatter serialization of an entity graph with 7 objects, and EntityBase2.GetObjectData is called 20 times. With a more complex graph (about 5000 entities with 4 levels of depth), it was called 40,000 times! Of note is that the calls to GetObjectData barely used any of the processor time (even though they were called more times than they should be) but the most expensive part was the overhead of calls inside the BinaryFormatter.

I know that the remoting system somehow makes sure instances of the same object come out the other side as the same object (instead of two seperate objects), but could a better way to do this be possibly: 1. Flatten the object graph, create a dictionary of all entities' hashcodes with their serialized instance. (Dictionary of Hashcode->MemoryStream). Call it hashCodeEntityLookup. 2. In GetObjectData of EntityCollection2, instead of

info.AddValue("Entity" + i, List[i])

use

if not hashCodeEntityLookup.ContainsKey(List[i].GetHashCode) then
    hashCodeEntityLookup.Add List[i].GetHashCode, SharedFormatter.Serialize( List[i] )
end if
info.AddValue("Entity" + i, List[i].GetHashCode)

This would not work for empty entities, so for these use something like:

info.AddValue("Entity" + i, "new empty entity")

1. In the deserialization constructor

this.add( SharedFormatter.Deserialize(hashCodeEntityLookup( info.GetValue("Entity" + i)) )

[Edit] Step 3 should probably cache the deserialized instance somewhere, and use that instead of deserializing for every collection a particular entity instance is found in.... [End Edit]

I don't really know the best way to get the hashCodeEntityLookup dictionary into the stream, maybe have it as a shared member variable of EntityCollectionBase2. There are also some more blanks to fill in there, like the deserialization of empty entities...Any thoughts?

Are you using .NET 2.0 by any chance? If not, you should give it a try, binary serialization has been speed up for quite a bit in .net 2.0.

I'm with you that serializing / deserializing data using binary formatters is weird and also not that fast.

Otis wrote:

Are you using .NET 2.0 by any chance? If not, you should give it a try, binary serialization has been speed up for quite a bit in .net 2.0.

I'm with you that serializing / deserializing data using binary formatters is weird and also not that fast.

Yeah, already using 2.0/binary. Its taking about 15 minutes to bring across 20,000 entities, which we need to do for bulk inserts. And its not so much the time, as these bulk inserts are only done once every few months. the major problem is the processors being completely tied up during this period.

I guess we'll just figure out a workaround

Well, I guess I'll try playing around with my own custom serialization. Is there a recommended way to override the EntityCollectionBase2 serialization code? It accesses private member variables that are not available to derived code (like _containingEntity) so I can't completely override it, but I can't have my derived code call base.getobjectdata,or base.new(info, context) because those fully serialize the contents of the EntityCollection which I wanted to do in my own custom way.

How should I do this?

mikeg22 wrote:

Well, I guess I'll try playing around with my own custom serialization. Is there a recommended way to override the EntityCollectionBase2 serialization code? It accesses private member variables that are not available to derived code (like _containingEntity) so I can't completely override it, but I can't have my derived code call base.getobjectdata,or base.new(info, context) because those fully serialize the contents of the EntityCollection which I wanted to do in my own custom way. How should I do this?

It depends on what you want to get on the other end of the pipe: if you just want a flat list of data to insert, it's enough to simply send over a flat list of EntityFields2 objects. _containingEntity etc. are variables which are set during entity object instantiation, so you don't have to send that over. You have to send over 2 things: - the data required to build a new graph of empty objects which is similar to the graph of objects you serialized - the data required to work with the entities on the deserialized side. So if you know the entities are always new, you don't have to send dirty/changed flags for example, just field data.

Adapter already packs data of an entity in tight containers to keep dataoverhead to a minimum, however there's still overhead for the situation when a person wants the exact graph with all the state involved on the deserialized side.

With bulk inserts, it's often more efficient to simply use the bulk import logic of the database used. For example, (silly example, but you get the point), if you want to import data into access, it's way more efficient to simply import a flat file comma separated. The speed of that is incredible.

So you might want to look into this for your situation as well (I'm not assuming you're using access, but every database has a way to import bulk data very efficiently), like exporting all entities to insert in a given file, then compress the file, send it over to the server, unpack it there and feed it directly to the import logic of the database engine.

Otis wrote:

mikeg22 wrote:

Well, I guess I'll try playing around with my own custom serialization. Is there a recommended way to override the EntityCollectionBase2 serialization code? It accesses private member variables that are not available to derived code (like _containingEntity) so I can't completely override it, but I can't have my derived code call base.getobjectdata,or base.new(info, context) because those fully serialize the contents of the EntityCollection which I wanted to do in my own custom way. How should I do this?

It depends on what you want to get on the other end of the pipe: if you just want a flat list of data to insert, it's enough to simply send over a flat list of EntityFields2 objects. _containingEntity etc. are variables which are set during entity object instantiation, so you don't have to send that over. You have to send over 2 things: - the data required to build a new graph of empty objects which is similar to the graph of objects you serialized - the data required to work with the entities on the deserialized side. So if you know the entities are always new, you don't have to send dirty/changed flags for example, just field data.

Adapter already packs data of an entity in tight containers to keep dataoverhead to a minimum, however there's still overhead for the situation when a person wants the exact graph with all the state involved on the deserialized side.

With bulk inserts, it's often more efficient to simply use the bulk import logic of the database used. For example, (silly example, but you get the point), if you want to import data into access, it's way more efficient to simply import a flat file comma separated. The speed of that is incredible.

So you might want to look into this for your situation as well (I'm not assuming you're using access, but every database has a way to import bulk data very efficiently), like exporting all entities to insert in a given file, then compress the file, send it over to the server, unpack it there and feed it directly to the import logic of the database engine.

We have a very fast connection between client and server so data size is an issue, but not the bottleneck. The bottleneck for us is the processing required to break down and rebuild the entity graph. For some reason, it seems to be more processor intensive to create an entity graph from a serialization channel than it is to create the same graph from a database connection

I pretty much just wanted to prove to myself that there is no faster way to do it than is currently done. With the current serialization code, the data transfered is very very small, but each entity gets rebuilt from scratch and initialized field by field with its data. I just wonder if it would be less processor intensive to directly serialize the entity (without events ).

I know you ave much more knowledge of this subject than me so you've probably done it in the most efficient way, but I guess I'd just like to understand why things are going as slow as they are!

mikeg22 wrote:

We have a very fast connection between client and server so data size is an issue, but not the bottleneck. The bottleneck for us is the processing required to break down and rebuild the entity graph. For some reason, it seems to be more processor intensive to create an entity graph from a serialization channel than it is to create the same graph from a database connection

I'm also sometimes puzzled why it's so slow, considering the fact that binary serialization just dumps an object's data 1:1 into the output. What I know which does slow down the serialization logic is that if there are a lot of objects, the process will be significant slower.

I pretty much just wanted to prove to myself that there is no faster way to do it than is currently done. With the current serialization code, the data transfered is very very small, but each entity gets rebuilt from scratch and initialized field by field with its data. I just wonder if it would be less processor intensive to directly serialize the entity (without events ).

I know you ave much more knowledge of this subject than me so you've probably done it in the most efficient way, but I guess I'd just like to understand why things are going as slow as they are!

If you want to have the fastest serialization code, you should cut down the # of objects to serialize. An entity collectio has entities, entities have entityfields objects which have field objects which point to strings etc. the # of objects can be larger than expected, even though the data is pretty small. I already pack the fields into a single object, so that's smaller than it is in memory, though the number of objects might still be massive which can slow down the deserialization code. It might be that you can store the data you want to send (which is all you're interested in, objects are re-created on the other end) in a different, custom structure which has much less objects, so your serialization/deserialization speed is better. Rebuilding the graph might take more effort though, however you can do this in your own formatter, though that's a bit cumbersome to write.

Your profiling shows that most time is spend inside the BinaryFormatter? Or in my code?

Otis wrote:

I'm also sometimes puzzled why it's so slow, considering the fact that binary serialization just dumps an object's data 1:1 into the output. What I know which does slow down the serialization logic is that if there are a lot of objects, the process will be significant slower.

I've seen this also. I have read, but never seen confirmed, that when an object gets serialized/deserialized, the formatter has to make sure it is not the same object as another that has been serialized/deserialized in the same stream. This sounds like an O(N^2) problem if done in an inefficient way. My best guess is that this is where the problem lies.

Otis wrote:

If you want to have the fastest serialization code, you should cut down the # of objects to serialize. An entity collectio has entities, entities have entityfields objects which have field objects which point to strings etc. the # of objects can be larger than expected, even though the data is pretty small. I already pack the fields into a single object, so that's smaller than it is in memory, though the number of objects might still be massive which can slow down the deserialization code. It might be that you can store the data you want to send (which is all you're interested in, objects are re-created on the other end) in a different, custom structure which has much less objects, so your serialization/deserialization speed is better. Rebuilding the graph might take more effort though, however you can do this in your own formatter, though that's a bit cumbersome to write.

Your profiling shows that most time is spend inside the BinaryFormatter? Or in my code?

My profiling shows a big black hole of processor usage, which I guess is the BinaryFormatter code. Your code accounts for almost none of the processor usage.

I tried changing the custom entitycollection serialization code to pack the entities into an arraylist and add that to the SerializationInfo, but when I tried to deserialize, the arraylist had items in it, but the items where blank, making me think that the internal entities hadn't deserialized yet. This made this method impossible unless I implemented the "deferred deserialization" interface (I can't remember the name of it) that makes a call when all the objects have deserialized. But this I know would be a giant pain.

mikeg22 wrote:

Otis wrote:

I'm also sometimes puzzled why it's so slow, considering the fact that binary serialization just dumps an object's data 1:1 into the output. What I know which does slow down the serialization logic is that if there are a lot of objects, the process will be significant slower.

I've seen this also. I have read, but never seen confirmed, that when an object gets serialized/deserialized, the formatter has to make sure it is not the same object as another that has been serialized/deserialized in the same stream. This sounds like an O(N^2) problem if done in an inefficient way. My best guess is that this is where the problem lies.

Oh that's indeed the case, though I don't think they've made it so hard on themselves, afterall, a single hashtable is enough to find out if an instance is already been seen. . But you never know what they've cooked up, that's right

Otis wrote:

If you want to have the fastest serialization code, you should cut down the # of objects to serialize. An entity collectio has entities, entities have entityfields objects which have field objects which point to strings etc. the # of objects can be larger than expected, even though the data is pretty small. I already pack the fields into a single object, so that's smaller than it is in memory, though the number of objects might still be massive which can slow down the deserialization code. It might be that you can store the data you want to send (which is all you're interested in, objects are re-created on the other end) in a different, custom structure which has much less objects, so your serialization/deserialization speed is better. Rebuilding the graph might take more effort though, however you can do this in your own formatter, though that's a bit cumbersome to write.

Your profiling shows that most time is spend inside the BinaryFormatter? Or in my code?

My profiling shows a big black hole of processor usage, which I guess is the BinaryFormatter code. Your code accounts for almost none of the processor usage.

Very strange, it then indeed might be the large number of object instantiations which kills it.

I tried changing the custom entitycollection serialization code to pack the entities into an arraylist and add that to the SerializationInfo, but when I tried to deserialize, the arraylist had items in it, but the items where blank, making me think that the internal entities hadn't deserialized yet. This made this method impossible unless I implemented the "deferred deserialization" interface (I can't remember the name of it) that makes a call when all the objects have deserialized. But this I know would be a giant pain.

Yeah, that's horrible. Hmm...

Perhaps a silly thing, but have you tried Xml serialization? (it's probably even slower). Or fetching the data in a datatable and use binary remoting mode? It packs the data in a very flat structure so it's likely that it's faster than the current object tree setup?