Release 1.0.2005.1 Final

I am still having issues with building a proper proxy that can be consumed by anyone that wishes. The 3rd party web service templates provide a way to build a generic proxy, but do not implement the entity objects interface, which forces the consumer to have the LLBL object on both sides of the firewall.

Issues:

The base class implementation of the “EntityCollection” inherits IXmlSerializable which override how the object serializes. The same go for the Entity classes as well. When ever this is done you must supply the schema information either by the XmlSchemaProvider and to map the EntityCollection xsd type to the Entity[]. Yes, an array of Entities. This is because we need to get EntityCollection class get method to look like this in the proxy. This works if we remove the IXmlSerializable from the collection classes and use the default that is already provided.

Proxy Class Code:

public Entity[] GetCollection() 
{
    object[] results = this.Invoke("GetCollection", new object[0]);
    return ((Entity[])(results[0]));
}

** My question to LLBLGen Pro developers is why they choose not to rely on the existing collection type’s support. By overriding the Entity classes and not the EntityCollection class would provide the same serialization format wouldn't it. You are not serializing any fields in the collection class so why do you change the format in the XmlWriter? **

I am trying to get the WSDL to generate the soap response message to look like this:

<s:element name="GetCollectionResponse"> <s:complexType> <s:sequence> <s:element minOccurs="0" maxOccurs="unbounded" name="Entity" type="s1:Entity" /> </s:sequence> </s:complexType> </s:element>

Next, the Entity classes work fine except the templates do not provide the implementation of the entity classes in the proxy. You only get this code in the proxy class:

Proxy Class Code:

public class CustomerEntity 
{
}

So I must expand the <ComplexType></ComplexType> in the Scheme Provider that is in the web services file in the Helper Classes namespace to provide information about the properties within the class. The thing is if you remove the IXmlSerializable from the collection class this proxy is generated properly in the Entity. Wired! Does any body have code that does this property?

Now, there has been some talk about building objects outside the LLBL architecture and passing these object back over the web service. I just think is is a lot of overhead to do this. If this is done I think it should look something like what Christian Weyer sample looks like:

http://weblogs.asp.net/cweyer/archive/2004/08/02/205798.aspx

But, I still think the LLBL object should support soap messaging without having to override its underlining architecture, do you agree with this?

I just wanted to create a new thread for this topic and maybe we can find the best solution to the problem without having to create our own objects to be serialized or having to have the LLBL object on both sides of the firewall.

Thanks.

sburtonctr wrote:

Release 1.0.2005.1 Final

I am still having issues with building a proper proxy that can be consumed by anyone that wishes. The 3rd party web service templates provide a way to build a generic proxy, but do not implement the entity objects interface, which forces the consumer to have the LLBL object on both sides of the firewall.

Issues:

The base class implementation of the “EntityCollection” inherits IXmlSerializable which override how the object serializes. The same go for the Entity classes as well. When ever this is done you must supply the schema information either by the XmlSchemaProvider and to map the EntityCollection xsd type to the Entity[]. Yes, an array of Entities. This is because we need to get EntityCollection class get method to look like this in the proxy. This works if we remove the IXmlSerializable from the collection classes and use the default that is already provided.

That's great, but how are you going to solve the issue when you return a CustomerEntity from the webmethod and the customerentity has an orders collection which has order entities which contain a collection of order detail entities?

Proxy Class Code:

public Entity[] GetCollection() 
{
    object[] results = this.Invoke("GetCollection", new object[0]);
    return ((Entity[])(results[0]));
}

** My question to LLBLGen Pro developers is why they choose not to rely on the existing collection type’s support. By overriding the Entity classes and not the EntityCollection class would provide the same serialization format wouldn't it. You are not serializing any fields in the collection class so why do you change the format in the XmlWriter? **

well, you're free to return whatever type you like from a webmethod. If you want to return an entitycollection, you want an entitycollection to appear on the client, and thus, you want a factory object be created when it's created on the client, concurrencypredicatefactory if set, entityvalidator if set etc.

There is data serialized into the XML for an entitycollection, and it's required if you want the rich objects on the client. Same goes for the entities: if you want state tracking with the entities etc., the XML will be more verbose than just a bunch of tags and field data. That's the way it is unfortunately.

But, I still think the LLBL object should support soap messaging without having to override its underlining architecture, do you agree with this?

I can't remove IXmlSerializable, as that would make it impossible to serialize the EntityCollection into XML if a user returns an EntityCollection from a webmethod.

If you want to use the data inside, and don't want the objects on the client side, consider using a DTO object, as whatever you do, you do need a container object on the client. Please realize that if you use a generic object like an arraylist or a dumb array, you will lose features which are present in the current entitycollection and teh new features coming in v2.

That's great, but how are you going to solve the issue when you return a CustomerEntity from the webmethod and the customerentity has an orders collection which has order entities which contain a collection of order detail entities?

You could handle this in the XmlWrite method of the entity object. The entity object should know if it has a collection class property.

void IXmlSerializable.WriteXml( XmlWriter writer )
        {

            writer.WriteElementString( "Name", this.Name );
            writer.WriteStartElement( "AddressCollection" );

            //slow solution, but for loop would be better.
            foreach( AddressEntity address in _addresses )
            {

                ( ( IXmlSerializable )address ).WriteXml( writer );

            }
            writer.WriteEndElement( );


        }

I do not know where it is best to put the custom serialization at (Object or Collection), I am not saying this is right, I just need a solution. We need to find a way to define a proper schema for the collections and the objects. Since the base class are overriding the serialization methods shouldn't it also supply the schema.

Do you understand my problem? I don't know if I am getting my point across. Lets say the client machine does not use .Net. How would you call the web service if the proxy is not created correctly. This means the client needs the Entity class in the proxy and it needs to return an array of the Entity classes. you don't need to put the collection class in the proxy unless you have special properties (fields) that the client must retrieve date.

Sorry, for all this, but I really like LLBL's architecture compared to others we have tried, but this is costing a lot of time on being able to build webservice component without having to copy the LLBL object entities into non-LLBL object and collection just to serialize them acrossed a web service that does not have LLBL on the client. This should be possible because the .Net architecture support it.

I only suggested removing the collection class IXmlSerializable stuff because microsoft web service team ask

why am I doing that???

. I don't know enough about your product yet, but still trying different ways.

Is it better to open a help desk ticket or should I keep this on the form. Thanks. Please be patient with me, I am a little slow. lol.

Thanks

Frans,

i think you should remove the IXMLSerializable interface from the entity collection and entitybase2 objects.

then have option to generate it in the entities themselves. Then, on each NON field property of the each entitiy, including relation properties, put xmlignore above each property. This would allow one to use the xml serializer to VERY easily serialize entities, or they could implement IXMLSerializable if they want more custom serialization. (you could add or remove the xml ignore attributes on relations through the designer) This would allow by default, interoperable simple web messages.

I also think you should add a couple things to the designer. Right now, i have your generator generate custom entity classes to pass across web services, and through the use of custom properties i can add attributes above the properties generated for the fields. This is VERY easy to do and adds a ton of flexibility to the generation process...

Now, there has been some talk about building objects outside the LLBL architecture and passing these object back over the web service. I just think is is a lot of overhead to do this. If this is done I think it should look something like what Christian Weyer sample looks like.

the overhead is very small. to convert 1000 llblgen entities over to my custom entities takes ~4.6ms

to convert from my entities, to llblgen entities is ~114ms after i did some optimization.

In reality you should not be converting that many entities, so it will be very quick.

100 entities to custom objects ~560us, custom objects -> llblgen entities = ~4.8 ms.

Answer wrote:

Frans,

i think you should remove the IXMLSerializable interface from the entity collection and entitybase2 objects.

then have option to generate it in the entities themselves.

Why would I make that optional, as it would create only confusion among users who forget to enable it. Also, it's generic code so why should it be in the generated code, as that would only increase the amount of generated code? I fail to see why this is important, so please explain why NOT having IXmlSerializable is an advantage...

The IXmlSerializable interface is there so people can return an entity/entitycollection from a webmethod without a lot of problems. Of course I hate that design, but it's the way MS wants it to be. I rather have an external xml serializer object, but the XmlSerializer build into .net isn't an option, it can't deal with cyclic references nor hashtables nor interface based types.

But perhaps I'm missing something, I really don't see why not having IXmlSerializable on an entity would be better...

Then, on each NON field property of the each entitiy, including relation properties, put xmlignore above each property. This would allow one to use the xml serializer to VERY easily serialize entities, or they could implement IXMLSerializable if they want more custom serialization. (you could add or remove the xml ignore attributes on relations through the designer) This would allow by default, interoperable simple web messages.

You mean, serializing the data into xml? Everything I tried made the .NET xmlserializer hang or crash. The problem is that there are cyclic references. A and B have a 1:1 relation. Serializing an A instance, which refernces a B instance... how would you do that? I would traverse what's referenced by A and serialize that too. I can't direct the Xmlserializer to do that properly, as B also references A. So that's definitely not going to work. The only thing that works in all cases is custom xml serializing code. Which is already in place.

But I'm lost now what people really REALLY want. I don't understand it anymore. At one time, webservices is about messages and you should just send messages. Perfect. At another point, people want to send large object graphs. That doesn't add up. In another situation people want to serialize the object graphs to xml but don't want to use the entity objects on the client. Ok, though why bother with entities in the webmethod then? Just pull the data in a dynamic list and send it over: you will lose change tracking and other data anyway, so why bother with entities?

I also think you should add a couple things to the designer. Right now, i have your generator generate custom entity classes to pass across web services, and through the use of custom properties i can add attributes above the properties generated for the fields. This is VERY easy to do and adds a ton of flexibility to the generation process...

I'm not sure what exactly you're asking what should be added . In v2, the template system will be overhauled, so it will be much easier to add a task to run (couple of clicks), save that setup and re-use it in another cycle.

The main issue I have with all of this is that there isn't a single way how people WANT TO utilize webservices, though there is just one way how you can make .NET return an xml representation of your data from a webmethod.

I'm fine with building in another Xml aspect mode, similar to compact, but for example 'DataOnly'. You'll then get the data and have fun. But will that be usable? Highly unlikely, because the format isn't flexible, and it has a drawback: say I have 20 orders in a collection and these orders are of 4 customers. When I would serialize that into xml, I'd get the data of those 20 orders, but these reference a customer, so inside the order tag, I'd get a customer tag, which has an orders collection and inside that, the 5 orders of the customer..... ok, WHAT should be done there, yet again the order data? Or nothing? If you say, 'the order data', I've to serialize the order again, and why not serializing the customer again as well? So that's not going to work. Leaving it empty? No, that wouldn't be good either, because what if the customer has orders which aren't in the collection? Should I add a special tag there which refers to an order already processed? I do that now, but it's proprietry, and what's worse: I have to identify the entities in the data, with the objectid they have, which already comes close to compact xml serialization.

Serializing a flat entitycollection without hierarchies, that's simple. The problems begin when a hierarchy of objects is about to be serialized.

But reading this thread, I see there's a problem with the IXmlSerializable interface being present on entity and entitycollection? I still miss why that's the case, but if someone can clearly explain why, and taking the above into account, I'm all ears.

sburtonctr wrote:

That's great, but how are you going to solve the issue when you return a CustomerEntity from the webmethod and the customerentity has an orders collection which has order entities which contain a collection of order detail entities?

You could handle this in the XmlWrite method of the entity object. The entity object should know if it has a collection class property.

It does The current Entity2Xml routine produces Xml for the entity itself and delegates the Xml for any containing collection to the collection itself.

I do not know where it is best to put the custom serialization at (Object or Collection), I am not saying this is right, I just need a solution. We need to find a way to define a proper schema for the collections and the objects. Since the base class are overriding the serialization methods shouldn't it also supply the schema.

Do you understand my problem?

Not in full, sorry. . Am I correct in that you want the data inside an object graph (set of entities, which contain entities, which contain entities... ) and just that data in XML, and also the schema for that ?

Currently the schema isn't provided as the Xml producing routines on entity and entitycollection are meant for one purpose only: to be able to re-instantiate an entity and entity collection and all the refering objects on the other side, and thus also vice versa. This thus can help you work with entities on teh client side, have full change tracking, send them back to the service and there you just persist them.

The schema therefore isn't necessary as it's ignored anyway.

So, if you don't need the entity objects on the client, I don't see why you should bother with the entities being returned from the webmethod. It's easier IMHO to fetch the data in a typedlist/dynamic list and serialize that datatable into Xml using a dummy dataset: you need a container which is a generic type and NOT in the llblgen pro type space anyway on the other side and with that rule you automatically abandone changetracking and other features in the entities.

Also, please read my previous post about data-only xml serialization of an object graph and the problems to overcome with the cyclic references. I.o.w.: it's a bit problematic to get it right for all situations.

I don't know if I am getting my point across. Lets say the client machine does not use .Net. How would you call the web service if the proxy is not created correctly. This means the client needs the Entity class in the proxy and it needs to return an array of the Entity classes. you don't need to put the collection class in the proxy unless you have special properties (fields) that the client must retrieve date.

No I don't think you should focus on that. If the client machine doesn't use .NET, you can safely forget returning entity classes from a webmethod, unless you want to use the xml produced to re-instantiate objects in java or vb6 or what have you. As I explained above, the xml produced by the entities and entitycollections is in the form as it is now to be able to re-instantiate the same objects on the other side. If that's not a requirement, please go for a simpler route and simply return a datatable or your own DTO.

Sorry, for all this, but I really like LLBL's architecture compared to others we have tried, but this is costing a lot of time on being able to build webservice component without having to copy the LLBL object entities into non-LLBL object and collection just to serialize them acrossed a web service that does not have LLBL on the client. This should be possible because the .Net architecture support it.

I wished it was that simple. There are two situations: 1) a flat collection of objects, no hierarchy. 2) a hierarchy of objects (graph).

In situation 1, your argument makes sense, though I then would go for a typedlist or dynamic list and use the dataset to produce the xml. This xml is consumable on for example java with a utility library, the schema etc. is defined there. In situation 2, the problems start, because the object graph by itself is complex as I described in my previous post. Also, with graphs, it's not the same as hierarchical data which isn't a graph. Take Order -> Customer -> order collection (.Orders)-> Order for example. How would you describe in the XML that an order is already somewhere else in the Xml document when you run into Customer and have to serialize its Orders into the document? IMHO hard to do without custom tags like I use to signal an already processed element.

I only suggested removing the collection class IXmlSerializable stuff because microsoft web service team ask

why am I doing that???

. I don't know enough about your product yet, but still trying different ways.

Well, if I don't, you can't return an entitycollection from a webmethod. So I have to implement IXmlSerializable. I can't rely on XmlSerializer because that class is pretty dumb: the factory set in an entitycollection is an interface typed object. XmlSerializer quits when it sees an interface. I have to serialize this type into the stream, because how am I suppose to re-instantiate the entitycollection on the client otherwise?

So this comes down to: what do you want to do? Just have the data on the other side, or the data + specifications of its container and the benefits of that container?

If you just want the data, you don't need to return the entitycollection from the webmethod, as you won't be reinstantiating it on the client.

And the webservice team of Microsoft asks why IXmlSerializable is implemented? Well, I think they first have to look at the stuff they provided before asking questions to others. For starters, their IXmlSerializable approach in .NET 1.x and how they 'solved' it in .NET 2.0 to get proper proxys is beyond what I call good design, IF you want to use webservices in the way they were advertised in vs.net 2002/3. More and more people are getting conviced webservices aren't for passing large object graphs back and forth, but for high-level services which produce flat-table data, often in small fragments, based on a command message that's send to them by a client.

Is it better to open a help desk ticket or should I keep this on the form. Thanks. Please be patient with me, I am a little slow. lol.

No you can keep it on the forum, others can then jump in as well .

Why would I make that optional, as it would create only confusion among users who forget to enable it. Also, it's generic code so why should it be in the generated code, as that would only increase the amount of generated code? I fail to see why this is important, so please explain why NOT having IXmlSerializable is an advantage...

The IXmlSerializable interface is there so people can return an entity/entitycollection from a webmethod without a lot of problems. Of course I hate that design, but it's the way MS wants it to be. I rather have an external xml serializer object, but the XmlSerializer build into .net isn't an option, it can't deal with cyclic references nor hashtables nor interface based types.

But perhaps I'm missing something, I really don't see why not having IXmlSerializable on an entity would be better...

Quote:

Then, on each NON field property of the each entitiy, including relation properties, put xmlignore above each property. This would allow one to use the xml serializer to VERY easily serialize entities, or they could implement IXMLSerializable if they want more custom serialization. (you could add or remove the xml ignore attributes on relations through the designer) This would allow by default, interoperable simple web messages.

You mean, serializing the data into xml? Everything I tried made the .NET xmlserializer hang or crash. The problem is that there are cyclic references. A and B have a 1:1 relation. Serializing an A instance, which refernces a B instance... how would you do that? I would traverse what's referenced by A and serialize that too. I can't direct the Xmlserializer to do that properly, as B also references A. So that's definitely not going to work. The only thing that works in all cases is custom xml serializing code. Which is already in place.

But I'm lost now what people really REALLY want. I don't understand it anymore. At one time, webservices is about messages and you should just send messages. Perfect. At another point, people want to send large object graphs. That doesn't add up. In another situation people want to serialize the object graphs to xml but don't want to use the entity objects on the client. Ok, though why bother with entities in the webmethod then? Just pull the data in a dynamic list and send it over: you will lose change tracking and other data anyway, so why bother with entities?

Ok, i can see your confused by what i meant.

Let me start off by saying that WebServices are designed to send messages back and forth, NOT object graphs!!! If you want full type fidelity and an object graph users should be using .NET remoting PERIOD. Its not that the XML serializer is stupid, its actually quite brilliant and does the job EXACTLY how it was designed to...convert a simple object into a message...

Let me give you an example of what im talking about now... Lets say i have an OrderEntity with a bunch of OrderDetail Entities. And of course there is a property on OrderEntity referencing a collection of OrderDetailEntities.

Now, as it is, the XMLSerializer would use IXMLSerializable and output your custom xml. However, this is NOT what i want when passing a message, as its not interop friendly and it contains an object graph not a message....

So how do i get around this, well i use the generator and create a custom set of entities that contain just the fields. So in the the case of Order and OrderDetails, there is no reference in Order to Orderdetail objects and vice versa.....

When one calls GetOrder(Guid orderId), i construct a GetOrderResponse object which contains one order object, and a collection of orderDetail objects as properties. This GetOrderResponse is my message....

But ihave an extra step in there, i have to copy data from my llblgens OrderEntity into my custom order entity etc...

There is an easy way to eliminate this step. Simply get rid of the IXMLSerializable interface, and place the [XmlIgnore] attribute above EVERY property that is not a column in the DB. So in the case of OrderEntity and OrderDetailEntity, the property that references the OrderDetail EntityCollection in OrderEntity and every field that is not a column in the order table would have the [XmlIgnore] attribute above it. This would allow the XmlSerializer to work just fine in the scenario that i suggested Which would allow for clean interoperable web services.

In the event that user whats to send an object graph, which they really shouldnt be, but if they did, they could use template set that would implement IXMLSerializable as it sits now.

I hope you understand why i say to remove it by default and have a 2nd template set that would generate the IXMLSerializable inteface if a customer wanted it.....

Ok, I had a feeling you meant that . But I'm still wondering how the orderdetails data is consumed by the XmlSerializer if the OrderDetails property in order is marked with XmlIgnore ?

Btw, in v2, the concept of a 'templateset' is gone. You'll have different .templatebindings files with bindings you can set precedence for, different .tasks files and you can select the tasks to run via presets or build the queue by hand manually in the gui.

I do understand where you're going to, and I agree, though to prevent users shooting themselves in the foot, it should be the default option to have IXmlSerializable implemented. How I'll make it implement IXmlSerializable in the templates is a bit of a problem still, but I think I can fix that.

There's another problem and that's the types in where the data is stored on the client. These are classes generated by wsdl I pressume? (dumb DTO buckets)

Otis wrote:

Ok, I had a feeling you meant that . But I'm still wondering how the orderdetails data is consumed by the XmlSerializer if the OrderDetails property in order is marked with XmlIgnore ?

orderdetails is not automatically consumed. You have to manually return it. So for instance if i wanted to return both order and orderdetails, i woudl have to constuct a getorderresponse object that would have TWO properties....one order property and another property that was acollection of orderdetail objects...

thats one way you could do it, the other way would be to inherit Order, and add another order detail collection property that does not have XML ignore....that would work fine as well...but its a bit more of a pain and prolly not a good way...

There's another problem and that's the types in where the data is stored on the client. These are classes generated by wsdl I pressume? (dumb DTO buckets)

Thats correct, the wsdl shoudl be able to generate the objects...OR you could also use llblgen entities still if you implemented a schemaimporterextension But the abilty for the wsdl to autogenerate the objects is a must!

Answer wrote:

Otis wrote:

Ok, I had a feeling you meant that . But I'm still wondering how the orderdetails data is consumed by the XmlSerializer if the OrderDetails property in order is marked with XmlIgnore ?

orderdetails is not automatically consumed. You have to manually return it. So for instance if i wanted to return both order and orderdetails, i woudl have to constuct a getorderresponse object that would have TWO properties....one order property and another property that was acollection of orderdetail objects...

thats one way you could do it, the other way would be to inherit Order, and add another order detail collection property that does not have XML ignore....that would work fine as well...but its a bit more of a pain and prolly not a good way...

Though what if you want it?

IMHO It's not an easy task to solve. I agree that the xmlserializable interface is perhaps a pain on the collection etc. but it's a convenient way to support xmlwebservices for the people who want it. I can't turn that back now. Moving the code to the templates is a bit cumbersome as I then will extend the amount of generated code while I want to decrease the amount of generated code, although it's not a big piece of code to have (2 methods and an interface). The issue is also that the user has to specify somewhere to have support for this interface on the entities, and I therefore have to create different entity templates. Now, that's something I won't do.

What I can do is add a 'dataonly' xml serialization but that would also be too slow for you I think, as I have to take into account properties which aren't in the fields but added manually by the developer. So reflection has to be used.

There's another problem and that's the types in where the data is stored on the client. These are classes generated by wsdl I pressume? (dumb DTO buckets)

Thats correct, the wsdl shoudl be able to generate the objects...OR you could also use llblgen entities still if you implemented a schemaimporterextension But the abilty for the wsdl to autogenerate the objects is a must!

For now I'll put this on hold. I know it's a bit of a pain, but I also know that at the moment there's no proper solution for it which will satisfy everyone. Then there's WCF on the horizon which puts other things on the table and which might force changes to the code which can push the framework into the direction which does satisfy everyone.

fair enuf

i think the best solution is to generate a second set of objects with conversion code which is what im doing. with overloading the casting operators the conversion is almost painless

Answer wrote:

fair enuf

i think the best solution is to generate a second set of objects with conversion code which is what im doing. with overloading the casting operators the conversion is almost painless

I think with projections in v2 you'll get far In v2, you can grab an EntityView object from an entitycollection and 'project' that data onto for example your own classes, with a few lines of code.