LLBLGen Pro - generated code - XML Support (serialization, de-serialization)

Generated code - XML Support (serialization, de-serialization)

Preface

.NET contains a class called XmlSerializer. In .NET 1.0 and 1.1, this class is not well written: it can't deal with interface typed members, can't deal with cyclic references, can't handle for example Hashtable typed variables, generates XML first in temp files which is very slow (but faster with subsequent requests, however not in all cases), can't deal with event-handlers defined in the object to serialize and the interface it uses (IXmlSerializable), is undocumented for .NET 1.x due to issues with the XmlSerializer / wsdl.exe (It always expects the class implementing that interface to be a DataSet).

To make XML generation/consumption possible, all entity and entity collection classes are able to convert themselves into XML or re-instantiate themselves or related entities through consumption of an XML document. LLBLGen Pro's code offers you three different types of XML output: Verbose, Compact and for adapter specific also Compact25. Verbose is usable in .NET -> non-.NET communication, if the non-.NET side has to know complex type information of the data received. The Compact formats can be used for .NET <-> .NET communication between two systems which both have knowledge of the types used: the generated code, or you can use it for further processing, as it doesn't contain much type information to keep the XML as compact as possible.

The entity classes and entity collection classes all implement IXmlSerializable. Under the hood, the interface is connected to the compact Xml producing/consuming methods, and allow you to use entities and entity collections in an XML webservice environment. See for more details about using the code in a webservice environment: generated code - XML Webservices / WCF support.

The methods you will use to produce XML and to consume XML are: ReadXml() and WriteXml() which come with various overloads. Please consult the LLBLGen Pro reference manual for more details about these overloads. Entity classes, the EntityCollection class (Adapter) and the entity collection classes (SelfServicing) contain ReadXml() and WriteXml() methods which allow you to write a complete object hierarchy to XML, using a WriteXml() overload and instantiate a complete object hierarchy from XML using a ReadXml() overload. This section describes in brief how to establish that and it also describes the specific format elements of the XML written by WriteXml(). Not all XML tags are discussed, as most of them are self describing, however some XML elements are custom to establish cyclic references and other special aspects.

Note:

Because all inner data of an object is also written into the XML, the total XML size produced by the verbose variant of the WriteXml() methods can become quite large. Although the routines are optimized for speed, it can still take some time to write a complete object hierarchy to XML or instantiate a hierarchy from XML. If the amount of XML is too much, consider using one of the Compact variants, with Compact25 as recommended format (Adapter only). You can switch to a Compact variant by passing in settings to the WriteXml() method.

Consider using Adapter if you're working with XML and performance is an issue, as Adapter has support for the more compact Compact25 format and its XML routines are more optimized as they use an XmlWriter/Reader instead of an XmlDocument.

Writing / reading object hierarchies to / from XML

Entity objects have a unique ID, the ObjectID. This ID is specific for each instance of an entity. The XML producing/consuming code uses these ID's heavily to handle cyclic references. The ObjectID is a normal GUID. To write an entity to XML, complete with its contained entity objects, collections, and objects inside those objects, you use the entity's method WriteXml(). WriteXml() has several overloads to enable you to retrieve the XML in a form you want to work with, like a string with XML or an XmlNode, and if you want verbose (default) or compact XML. WriteXml() has overloads accepting an XmlDocument, which allows you to specify an existing XmlDocument to which you want to add the XmlNode with the created XML. This node is addable to the XmlDocument without conversions, as it is created with the passed in XmlDocument.

Instantiating an entity and its containing objects from XML (for example XML that is written by a call to WriteXml()), is done by first instantiating a new entity object (or EntityCollection object) and then calling ReadXml() of that new object, which will fill the object with data inside objects instantiated from the XML data passed to ReadXml()'s overloads in one of the formats supported by ReadXml().

Verbose XML

Below two examples how to use these methods. First a WriteXml() is performed, after that a ReadXml(). The XML is written to a file, which is then read into a new object using ReadXml(). The CustomerEntity object is filled with OrderEntity objects and each OrderEntity object is filled with its OrderDetails objects

C#
VB.NET

// [C#]
StreamWriter writer = new StreamWriter("customer.XML");
string customerXml = String.Empty;
customer.WriteXml(out customerXml);
writer.Write(customerXml);
writer.Close();

// now read it back
StreamReader reader = new StreamReader("customer.XML");
string xmlFromFile = reader.ReadToEnd();
reader.Close();
CustomerEntity customerFromXml = new CustomerEntity();
customerFromXml.ReadXml(xmlFromFile);

' [VB.NET]
Dim writer As New StreamWriter("customer.XML")
Dim customerXml As String = String.Empty
customer.WriteXml(ByRef customerXml)
writer.Write(customerXml)
writer.Close()

' now read it back
Dim reader As New StreamReader("customer.XML")
Dim xmlFromFile As String = reader.ReadToEnd()
reader.Close()
Dim customerFromXml As New CustomerEntity()
customerFromXml.ReadXml(xmlFromFile)

customerFromXml is now equal to customer, including the ObjectID. The same code works also for EntityCollection classes, the EntityCollection class has the same ReadXml() and WriteXml() overloads. SelfServicing's WriteXml() writes only already fetched related entities.

Compact XML

The same functionality can be used to produce compact XML. This XML is stripped from most type specific information and is the preferred format for communication between client and server which both have a reference to the same generated code so type information is unnecessary. To specify what you want, you can customize the XML output by specifying XmlFormatAspect values. XmlFormatAspect is an enum definition with Flags which can be 'or'-ed together and which allow you to signal the XML producing code to produce compact XML, to convert DateTime values into XML compliant date / time formats and to wrap strings which contain < and/or > characters into CDATA blocks. Please consult the reference manual for details about this enum definition and its values.

Below the same example is specified, but now we order WriteXml to produce compact XML, wrap mark-up language statements in CDATA blocks and convert datetime values to XML format instead of storing them as ticks.

C#
VB.NET

// [C#]
StreamWriter writer = new StreamWriter("customer.XML");
string customerXml = String.Empty;
customer.WriteXml(
	XmlFormatAspect.Compact | XmlFormatAspect.DatesInXmlDataType | XmlFormatAspect.MLTextInCDataBlocks, out customerXml);
writer.Write(customerXml);
writer.Close();

// now read it back
StreamReader reader = new StreamReader("customer.XML");
string xmlFromFile = reader.ReadToEnd();
reader.Close();
CustomerEntity customerFromXml = new CustomerEntity();
customerFromXml.ReadXml(xmlFromFile);

' [VB.NET]
Dim writer As New StreamWriter("customer.XML")
Dim customerXml As String = String.Empty
customer.WriteXml( _ 
	XmlFormatAspect.Compact Or XmlFormatAspect.DatesInXmlDataType Or XmlFormatAspect.MLTextInCDataBlocks, ByRef customerXml)
writer.Write(customerXml)
writer.Close()

' now read it back
Dim reader As New StreamReader("customer.XML")
Dim xmlFromFile As String = reader.ReadToEnd()
reader.Close()
Dim customerFromXml As New CustomerEntity()
customerFromXml.ReadXml(xmlFromFile)

As you can see, ReadXml doesn't have to know if the XML is written in compact or non-compact format, it can deal with that on the fly.

Adapter specific: if you want even more compact XML, you can specify XmlFormatAspect.Compact25 instead of Compact. This will greatly reduce the XML produced. See for a format example the XML format descriptions below.

Culture specific format specifications

XML is a text-based format and when data is serialized into XML and back to values from the XML, it's essential that values which have culture-specific formats can be read back properly. For example the decimal value 1 in EU formats will be 1,000. In US formats, this is then equal to 1000, and not equal to 1. It can also lead to invalid values which will cause exceptions at runtime. By default, LLBLGen Pro uses the current culture to serialize values to XML and to deserialize XML back into values. This is important for System.Single, System.Double and System.Decimal typed values. DateTime typed values are serialized to either ticks (which is of type System.Long) or the specific Xml date / time format which is regional invariant. This means that if you use the same culture when serializing as well as deserializing (e.g. on the machine with the webservice and on the client machine), you have no problems. A problem arises when you use a different culture when deserializing as when the data was serialized.

To overcome this, LLBLGen Pro supports a setting to specify the culture to use for serialization and deserialization of values to/from XML. This setting, CultureNameForXmlValueConversion, which is a static property of the XmlHelper class can be set in code or via the config file by adding an element to the appSettings element, with cultureNameForXmlValueConversion as key and the name of the culture to use as value. Below are both ways of setting this property illustrated. To specify the invariant culture, use the empty string "". For all names of all cultures, please consult the MSDN documentation of the CultureInfo class. Example
The following two examples set the XmlHelper.CultureNameForXmlValueConversion property to the culture "nl-NL". First in code, followed by the element to be add to the appSettings element in the config files of the applications which serialize and deserialize the XML data.

In code:

C#
VB.NET

XmlHelper.CultureNameForXmlValueConversion = "nl-NL";

XmlHelper.CultureNameForXmlValueConversion = "nl-NL"

Via the .config files of the applications, by adding the following element to the appSettings element:

<add key="cultureNameForXmlValueConversion" value="nl-NL" />

XML Format descriptions

The verbose XML format written by WriteXml() is pretty straightforward. As an example the CustomerEntity written by the code listed above is used. The XML shown contains both entities and EntityCollection objects. The direct child node of the EntityCollectionReference nodes are XML nodes produced by the EntityCollection's WriteXml() method.

<CustomerEntity CustomerId="CHOPS" Assembly="DALTester, Version=1.0.1479.22635, 
	Culture=neutral, PublicKeyToken=null" Type="DALTester.CustomerEntity">
	<ConcurrencyPredicateFactoryToUse Type="SD.LLBLGen.Pro.ORMSupportClasses.IConcurrencyPredicateFactory"/>	
	...
	<ObjectID Type="System.Guid">033a5969-094a-4286-9e4c-944a1fcdd914</ObjectID>
	<Validator Assembly="Northwind, Version=1.0.1478.30380, Culture=neutral, PublicKeyToken=null" 
		Type="Northwind.ValidatorClasses.CustomerValidator"/>
	...
	<Fields>
		<CustomerId>
			<CurrentValue Type="System.String">CHOPS</CurrentValue>
			<IsChanged Type="System.Boolean">False</IsChanged>
			<IsNull Type="System.Boolean">False</IsNull>
		</CustomerId>
		<CompanyName>
			<CurrentValue Type="System.String">Chop-suey Chinese</CurrentValue>
			<IsChanged Type="System.Boolean">False</IsChanged>
			<IsNull Type="System.Boolean">False</IsNull>
		</CompanyName>
		...
	</Fields>
	<EntityCollectionReference PropertyName="Orders">
		<Orders Assembly="Northwind, Version=1.0.1478.30380, Culture=neutral, PublicKeyToken=null" 
			Type="Northwind.HelperClasses.EntityCollection">
			<Entities>	
				<OrderEntity OrderId="10254" Assembly="DALTester, Version=1.0.1479.22635, Culture=neutral, PublicKeyToken=null" 
					Type="DALTester.OrderEntity">
					<ConcurrencyPredicateFactoryToUse Type="SD.LLBLGen.Pro.ORMSupportClasses.IConcurrencyPredicateFactory"/>
					<EntityReference PropertyName="Customer">
						<ProcessedObjectReference ObjectID="033a5969-094a-4286-9e4c-944a1fcdd914"/>
					</EntityReference>
					...
				</OrderEntity>
				...
			</Entities>
			<EntityFactoryToUse Assembly="DALTester, Version=1.0.1479.22635, Culture=neutral, PublicKeyToken=null" 
				Type="DALTester.OrderEntityFactory"/>
			<AllowEdit Type="System.Boolean">True</AllowEdit>
			<AllowRemove Type="System.Boolean">True</AllowRemove>
			<Count Type="System.Int32">95</Count>
			<AllowNew Type="System.Boolean">True</AllowNew>
			<SupportsChangeNotification Type="System.Boolean">True</SupportsChangeNotification>
			<SupportsSorting Type="System.Boolean">False</SupportsSorting>
			<SupportsSearching Type="System.Boolean">True</SupportsSearching>
			<Validator Assembly="Unknown"/>
			<IsReadOnly Type="System.Boolean">False</IsReadOnly>
		</Orders>
	</EntityCollectionReference>
	<IsNew Type="System.Boolean">False</IsNew>
	<LLBLGenProEntityName Type="System.String">CustomerEntity</LLBLGenProEntityName>
	<IsDirty Type="System.Boolean">False</IsDirty>
	<EntityState Type="SD.LLBLGen.Pro.ORMSupportClasses.EntityState">Fetched</EntityState>
</CustomerEntity>

Please pay special attention to the element ProcessedObjectReference, inside the Customer reference of the Order object. It shows that the Customer object referenced by the Order object is the same object, which contains the Orders collection the Order object is stored in (cyclic reference). ReadXml() will deal with this reference. Assembly types and full object types are specified to be able to fully re-instantiate a written object hierarchy from this XML.

Compact format

The compact format contains much less data, as the following example shows. it's an OrderDetails entity from Northwind with a referenced Order entity.

<OrderDetailsEntity OrderId="10289" ProductId="3">
	<EntityReference PropertyName="Products" />
	<ObjectID>f08047cb4d50483c99f9937a5f77a4a5</ObjectID>
	<IsNew>False</IsNew>
	<Fields>
		<OrderId>
			<CurrentValue>10289</CurrentValue>
			<DbValue>10289</DbValue>
			<IsChanged>False</IsChanged>
			<IsNull>False</IsNull>
		</OrderId>
		<ProductId>
			<CurrentValue>3</CurrentValue>
			<DbValue>3</DbValue>
			<IsChanged>False</IsChanged>
			<IsNull>False</IsNull>
		</ProductId>
		...
	</Fields>
	<EntityReference PropertyName="Orders">
		<OrderEntity OrderId="10289">
			<Fields>
				<OrderId>
					<CurrentValue>10289</CurrentValue>
					<DbValue>10289</DbValue>
					<IsChanged>False</IsChanged>
					<IsNull>False</IsNull>
				</OrderId>
				...
				<OrderDate>
					<CurrentValue>19960826T00:00:00.0000000+02:00</CurrentValue>
					<DbValue>19960826T00:00:00.0000000+02:00</DbValue>
					<IsChanged>False</IsChanged>
					<IsNull>False</IsNull>
				</OrderDate>
				...
			</Fields>
			<EntityReference PropertyName="Employees" />
			<EntityCollectionReference PropertyName="OrderDetails">
				<OrderDetails>
					<Entities>
						<ProcessedObjectReference ObjectID="f08047cb4d50483c99f9937a5f77a4a5" />
					</Entities>
					<AllowNew>True</AllowNew>
					<EntityFactoryToUse Assembly="Northwind.Adapter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" Type="Northwind.Adapter.FactoryClasses.OrderDetailsEntityFactory" />
					<AllowRemove>False</AllowRemove>
					<AllowEdit>True</AllowEdit>
				</OrderDetails>
			</EntityCollectionReference>
			...
			<IsDirty>False</IsDirty>
			<EntityState>Fetched</EntityState>
			<SavedFieldSets />
		</OrderEntity>
	</EntityReference>
	<IsDirty>False</IsDirty>
	<EntityState>Fetched</EntityState>
	<SavedFieldSets />
</OrderDetailsEntity>

There still are factory specifications available, these are required for the ReadXml routine to properly re-instantiate the collection classes.

Compact25 format (Adapter specific)

The Compact25 format is much more compact than the previous described formats. Below is the XML in Compact25 of a Northwind Customer and its Order entities. Compact25 assumes all type info is known on the consumer side, except when it's not determinable. It won't emit elements for references which are empty. It contains small elements for state tracking for the LLBLGen Pro runtime framework.

<CustomerEntity ObjectID="df7c7b33-1fd2-4fe9-b3b4-ec8d00eeebed" Format="Compact25">
    <CustomerId>CHOPS</CustomerId>
    <CompanyName>Foo Inc.</CompanyName>
    <ContactName>Yang Wang</ContactName>
    <ContactTitle>Owner</ContactTitle>
    <Address>Hauptstr. 29</Address>
    <City>Bern</City>
    <PostalCode>3012</PostalCode>
    <Country>Switserland</Country>
    <Phone>555-chang</Phone>
    <Fax>1313-chan</Fax>
    <Orders>
        <OrderEntity ObjectID="d8117794-675c-45ce-bd97-f23c22391039">
            <OrderId>10746</OrderId>
            <CustomerId>CHOPS</CustomerId>
            <EmployeeId>1</EmployeeId>
            <OrderDate>1997-11-19T00:00:00.0000000+01:00</OrderDate>
            <RequiredDate>1997-12-17T00:00:00.0000000+01:00</RequiredDate>
            <ShippedDate>1997-11-21T00:00:00.0000000+01:00</ShippedDate>
            <ShipVia>3</ShipVia>
            <Freight>31,4300</Freight>
            <ShipName>Chop-suey Chinese</ShipName>
            <ShipAddress>Hauptstr. 31</ShipAddress>
            <ShipCity>Bern</ShipCity>
            <ShipPostalCode>3012</ShipPostalCode>
            <ShipCountry>Switzerland</ShipCountry>
            <Customer Ref="df7c7b33-1fd2-4fe9-b3b4-ec8d00eeebed" />
            <NullableDateTime></NullableDateTime>
            <CompanyName>Foo Inc.</CompanyName>
            <NullableDec></NullableDec>
            <NulledString></NulledString>
            <_lps fs="AACAAA==" es="1" />
        </OrderEntity>
        ...
        <OrderEntity ObjectID="80a16cd2-4dd9-4e41-aae0-5a669af76257">
            <OrderId>10731</OrderId>
            <CustomerId>CHOPS</CustomerId>
            <EmployeeId>7</EmployeeId>
            <OrderDate>1997-11-06T00:00:00.0000000+01:00</OrderDate>
            <RequiredDate>1997-12-04T00:00:00.0000000+01:00</RequiredDate>
            <ShippedDate>1997-11-14T00:00:00.0000000+01:00</ShippedDate>
            <ShipVia>1</ShipVia>
            <Freight>96,6500</Freight>
            <ShipName>Chop-suey Chinese</ShipName>
            <ShipAddress>Hauptstr. 31</ShipAddress>
            <ShipCity>Bern</ShipCity>
            <ShipPostalCode>3012</ShipPostalCode>
            <ShipCountry>Switzerland</ShipCountry>
            <Customer Ref="df7c7b33-1fd2-4fe9-b3b4-ec8d00eeebed" />
            <NullableDateTime></NullableDateTime>
            <CompanyName>Foo Inc.</CompanyName>
            <NullableDec></NullableDec>
            <NulledString></NulledString>
            <_lps fs="AACAAA==" es="1" />
        </OrderEntity>
        <_lps f="7" />
    </Orders>
    <_lps fs="ACAA" es="1" />
</CustomerEntity>

LLBLGen Pro will add _lps elements to the XML to track the state of entity fields and the entity itself and also the state of entity collections. The data in these elements is compacted to keep everything as small as possible. A decription of the various elements you can encounter in an _lps element is found below.

_lps element description for Entities
The _lps attribute fs stands for ‘Field state data’ and is a bitarray, with for each field a bit for the IsChanged flag and a bit for the IsNull flag. This bitarray is then packed to a hex string. The order is by fieldindex: the field at index 0 has the first two bits (at index 0 and 1), etc. There’s no info about IsNew and IsDirty. This is because these are redundant: IsDirty can be determined from the stateflags of all the fields, and IsNew can be determined from the attribute 'es'.
The _lps attribute es stands for ‘EntityState’ and is the numeric value of the EntityState enum for the entity. If there are fields changed, or better: if the CurrentValue for a field is different from the DbValue value, and the DbValue isn’t null/Nothing, an entry is placed for that field in the dbv element as a child of the _lps element with the DbValue value. This saves data for the fields which aren’t changed. If the entity has an error string set, it’s stored under the ee element as a child of the _lps element. If there are field errors stored, they’re stored under an efes element under the _lps element and every error is stored as a sub element of efes with the key as the elementname and the errorstring as the element value.

_lps element description for EntityCollection instances
The f attribute in the _lps element of the entity collection is a bitarray with the Allow* values in this order (from index 0 upwards) AllowEdit, AllowNew, AllowRemove. At bit pos 3, the DoNotPerformAddIfPresent flag is placed.