XML Support (serialization, de-serialization)

All entity and entity collection classes are able to convert themselves into XML or re-instantiate themselves or related entities through consumption of an XML document. LLBLGen Pro's code offers you three different types of XML output: Verbose, Compact and for adapter specific also Compact25. It's highly recommended to use Compact25 if you are doing manual xml serialization through the methods available.

Verbose is usable in .NET -> non-.NET communication, if the non-.NET side has to know complex type information of the data received. The Compact formats can be used for .NET <-> .NET communication between two systems which both have knowledge of the types used: the generated code, or you can use it for further processing, as it doesn't contain much type information to keep the XML as compact as possible.

WCF Services / WebAPI, which use IXmlSerializable, will use Compact25 when using Adapter.

The entity classes and entity collection classes all implement IXmlSerializable. Under the hood, the interface is connected to the compact Xml producing/consuming methods, and allow you to use entities and entity collections in an XML webservice environment. See for more details about using the code in a webservice environment: generated code - XML Webservices / WCF support.

The methods you will use to produce XML and to consume XML are: ReadXml() and WriteXml() which come with various overloads. Please consult the LLBLGen Pro reference manual for more details about these overloads. Entity classes, the EntityCollection class (Adapter) and the entity collection classes (SelfServicing) contain ReadXml() and WriteXml() methods which allow you to write a complete object hierarchy to XML, using a WriteXml() overload and instantiate a complete object hierarchy from XML using a ReadXml() overload.

This section describes in brief how to establish that and it also describes the specific format elements of the XML written by WriteXml(). Not all XML tags are discussed, as most of them are self describing, however some XML elements are custom to establish cyclic references and other special aspects.

Writing / reading object hierarchies to / from XML

Entity objects have a unique ID, the ObjectID. This ID is specific for each instance of an entity. The XML producing/consuming code uses these ID's heavily to handle cyclic references. The ObjectID is a normal GUID.

To write an entity to XML, complete with its contained entity objects, collections, and objects inside those objects, you use the entity's method WriteXml(). WriteXml() has several overloads to enable you to retrieve the XML in a form you want to work with, like a string with XML or an XmlNode, and e.g. in which format you want the output.

Examples

Below two examples how to use these methods.

Verbose / Default format aspects

First a WriteXml() is performed, after that a ReadXml(). The XML is written to a file, which is then read into a new object using ReadXml(). The CustomerEntity object is filled with OrderEntity objects and each OrderEntity object is filled with its OrderDetails objects. As no format aspects are specified, Verbose is assumed, as that's the default (for backwards compatibility reasons).

var writer = new StreamWriter("customer.XML");
string customerXml = String.Empty;
customer.WriteXml(out customerXml);
writer.Write(customerXml);
writer.Close();

// now read it back
var reader = new StreamReader("customer.XML");
string xmlFromFile = reader.ReadToEnd();
reader.Close();
CustomerEntity customerFromXml = new CustomerEntity();
customerFromXml.ReadXml(xmlFromFile);

customerFromXml is now equal to customer, including the ObjectID. The same code works also for EntityCollection classes, the EntityCollection class has the same ReadXml() and WriteXml() overloads. SelfServicing's WriteXml() writes only already fetched related entities.

Compact and Compact25 XML

The same functionality can be used to produce more compact XML. This XML is stripped from most type specific information. To specify what you want, you can customize the XML output by specifying XmlFormatAspect values. XmlFormatAspect is an enum definition with Flags which can be 'or'-ed together and which allow you to signal the XML producing code to produce compact XML, to convert DateTime values into XML compliant date / time formats and to wrap strings which contain < and/or > characters into CDATA blocks. Please consult the reference manual for details about this enum definition and its values.

There are two compact formats, Compact and Compact25. Their formats are different, with Compact25 being the most compact XML and also the most natural format with respect to the data. Compact stays close to the object structure containing the data. Compact25 is only available in Adapter and is the format used when using entities in a WCF service, through the implementation of the IXmlSerializable interface. See for a format example the XML format descriptions below.

Below the same example is specified, but now we order WriteXml to produce Compact25 XML, wrap mark-up language statements in CDATA blocks and convert datetime values to XML format instead of storing them as ticks. It uses XmlFormatAspect.Compact. To produce Compact XML, specify XmlFormatAspect.Compact instead of XmlFormatAspect.Compact25.

var writer = new StreamWriter("customer.XML");
string customerXml = String.Empty;
customer.WriteXml(XmlFormatAspect.Compact25 | XmlFormatAspect.DatesInXmlDataType | 
                  XmlFormatAspect.MLTextInCDataBlocks, out customerXml);
writer.Write(customerXml);
writer.Close();

// now read it back
var reader = new StreamReader("customer.XML");
string xmlFromFile = reader.ReadToEnd();
reader.Close();
CustomerEntity customerFromXml = new CustomerEntity();
customerFromXml.ReadXml(xmlFromFile);

As you can see, ReadXml doesn't have to know if the XML is written in compact or non-compact format, it can deal with that on the fly.

Culture specific format specifications

XML is a text-based format and when data is serialized into XML and back to values from the XML, it's essential that values which have culture-specific formats can be read back properly. For example the decimal value 1 in EU formats will be 1,000.

In US formats, this is then equal to 1000, and not equal to 1. It can also lead to invalid values which will cause exceptions at runtime.

By default, LLBLGen Pro uses the current culture to serialize values to XML and to deserialize XML back into values. This is important for System.Single, System.Double and System.Decimal typed values. DateTime typed values are serialized to either ticks (which is of type System.Long) or the specific Xml date / time format which is regional invariant.

This means that if you use the same culture when serializing as well as deserializing (e.g. on the machine with the webservice and on the client machine), you have no problems. A problem arises when you use a different culture when deserializing as when the data was serialized.

To overcome this, LLBLGen Pro supports a setting to specify the culture to use for serialization and deserialization of values to/from XML. This setting, CultureNameForXmlValueConversion, which is a static property of the XmlHelper class can be set in code or via the config file by adding an element to the appSettings element, with cultureNameForXmlValueConversion as key and the name of the culture to use as value.

Below are both ways of setting this property illustrated. To specify the invariant culture, use the empty string "". For all names of all cultures, please consult the MSDN documentation of the CultureInfo class.

Example

The following two examples set the XmlHelper.CultureNameForXmlValueConversion property to the culture "nl-NL". First in code, followed by the element to be add to the appSettings element in the config files of the applications which serialize and deserialize the XML data.

In code:

XmlHelper.CultureNameForXmlValueConversion = "nl-NL";

Via the .config files of the applications, by adding the following element to the appSettings element:

<add key="cultureNameForXmlValueConversion" value="nl-NL" />

XML Format descriptions

Verbose format

The verbose XML format written by WriteXml() is pretty straightforward. As an example the CustomerEntity written by the code listed above is used. The XML shown contains both entities and EntityCollection objects. The direct child node of the EntityCollectionReference nodes are XML nodes produced by the EntityCollection's WriteXml() method.

<CustomerEntity CustomerId="CHOPS" Assembly="DALTester, Version=1.0.1479.22635, 
    Culture=neutral, PublicKeyToken=null" Type="DALTester.CustomerEntity">
    <ConcurrencyPredicateFactoryToUse Type="SD.LLBLGen.Pro.ORMSupportClasses.IConcurrencyPredicateFactory"/>  
    ...
    <ObjectID Type="System.Guid">033a5969-094a-4286-9e4c-944a1fcdd914</ObjectID>
    <Validator Assembly="Northwind, Version=1.0.1478.30380, Culture=neutral, PublicKeyToken=null" 
        Type="Northwind.ValidatorClasses.CustomerValidator"/>
    ...
    <Fields>
        <CustomerId>
            <CurrentValue Type="System.String">CHOPS</CurrentValue>
            <IsChanged Type="System.Boolean">False</IsChanged>
            <IsNull Type="System.Boolean">False</IsNull>
        </CustomerId>
        <CompanyName>
            <CurrentValue Type="System.String">Chop-suey Chinese</CurrentValue>
            <IsChanged Type="System.Boolean">False</IsChanged>
            <IsNull Type="System.Boolean">False</IsNull>
        </CompanyName>
        ...
    </Fields>
    <EntityCollectionReference PropertyName="Orders">
        <Orders Assembly="Northwind, Version=1.0.1478.30380, Culture=neutral, PublicKeyToken=null" 
            Type="Northwind.HelperClasses.EntityCollection">
            <Entities>    
                <OrderEntity OrderId="10254" Assembly="DALTester, Version=1.0.1479.22635, Culture=neutral, PublicKeyToken=null" 
                    Type="DALTester.OrderEntity">
                    <ConcurrencyPredicateFactoryToUse Type="SD.LLBLGen.Pro.ORMSupportClasses.IConcurrencyPredicateFactory"/>
                    <EntityReference PropertyName="Customer">
                        <ProcessedObjectReference ObjectID="033a5969-094a-4286-9e4c-944a1fcdd914"/>
                    </EntityReference>
                    ...
                </OrderEntity>
                ...
            </Entities>
            <EntityFactoryToUse Assembly="DALTester, Version=1.0.1479.22635, Culture=neutral, PublicKeyToken=null" 
                Type="DALTester.OrderEntityFactory"/>
            <AllowEdit Type="System.Boolean">True</AllowEdit>
            <AllowRemove Type="System.Boolean">True</AllowRemove>
            <Count Type="System.Int32">95</Count>
            <AllowNew Type="System.Boolean">True</AllowNew>
            <SupportsChangeNotification Type="System.Boolean">True</SupportsChangeNotification>
            <SupportsSorting Type="System.Boolean">False</SupportsSorting>
            <SupportsSearching Type="System.Boolean">True</SupportsSearching>
            <Validator Assembly="Unknown"/>
            <IsReadOnly Type="System.Boolean">False</IsReadOnly>
        </Orders>
    </EntityCollectionReference>
    <IsNew Type="System.Boolean">False</IsNew>
    <LLBLGenProEntityName Type="System.String">CustomerEntity</LLBLGenProEntityName>
    <IsDirty Type="System.Boolean">False</IsDirty>
    <EntityState Type="SD.LLBLGen.Pro.ORMSupportClasses.EntityState">Fetched</EntityState>
</CustomerEntity>

Please pay special attention to the element ProcessedObjectReference, inside the Customer reference of the Order object. It shows that the Customer object referenced by the Order object is the same object, which contains the Orders collection the Order object is stored in (cyclic reference).

ReadXml() will deal with this reference. Assembly types and full object types are specified to be able to fully re-instantiate a written object hierarchy from this XML.

Compact format

The compact format contains much less data, as the following example shows. it's an OrderDetails entity from Northwind with a referenced Order entity.

<OrderDetailsEntity OrderId="10289" ProductId="3">
    <EntityReference PropertyName="Products" />
    <ObjectID>f08047cb4d50483c99f9937a5f77a4a5</ObjectID>
    <IsNew>False</IsNew>
    <Fields>
        <OrderId>
            <CurrentValue>10289</CurrentValue>
            <DbValue>10289</DbValue>
            <IsChanged>False</IsChanged>
            <IsNull>False</IsNull>
        </OrderId>
        <ProductId>
            <CurrentValue>3</CurrentValue>
            <DbValue>3</DbValue>
            <IsChanged>False</IsChanged>
            <IsNull>False</IsNull>
        </ProductId>
        ...
    </Fields>
    <EntityReference PropertyName="Orders">
        <OrderEntity OrderId="10289">
            <Fields>
                <OrderId>
                    <CurrentValue>10289</CurrentValue>
                    <DbValue>10289</DbValue>
                    <IsChanged>False</IsChanged>
                    <IsNull>False</IsNull>
                </OrderId>
                ...
                <OrderDate>
                    <CurrentValue>19960826T00:00:00.0000000+02:00</CurrentValue>
                    <DbValue>19960826T00:00:00.0000000+02:00</DbValue>
                    <IsChanged>False</IsChanged>
                    <IsNull>False</IsNull>
                </OrderDate>
                ...
            </Fields>
            <EntityReference PropertyName="Employees" />
            <EntityCollectionReference PropertyName="OrderDetails">
                <OrderDetails>
                    <Entities>
                        <ProcessedObjectReference ObjectID="f08047cb4d50483c99f9937a5f77a4a5" />
                    </Entities>
                    <AllowNew>True</AllowNew>
                    <EntityFactoryToUse Assembly="Northwind.Adapter, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" Type="Northwind.Adapter.FactoryClasses.OrderDetailsEntityFactory" />
                    <AllowRemove>False</AllowRemove>
                    <AllowEdit>True</AllowEdit>
                </OrderDetails>
            </EntityCollectionReference>
            ...
            <IsDirty>False</IsDirty>
            <EntityState>Fetched</EntityState>
            <SavedFieldSets />
        </OrderEntity>
    </EntityReference>
    <IsDirty>False</IsDirty>
    <EntityState>Fetched</EntityState>
    <SavedFieldSets />
</OrderDetailsEntity>

There still are factory specifications available, these are required for the ReadXml routine to properly re-instantiate the collection classes.

Compact25 format (Adapter specific)

The Compact25 format is much more compact than the previous described formats. Below is the XML in Compact25 of a Northwind Customer and its Order entities. Compact25 assumes all type info is known on the consumer side, except when it's not determinable. It won't emit elements for references which are empty. It contains small elements for state tracking for the LLBLGen Pro runtime framework.

<CustomerEntity ObjectID="df7c7b33-1fd2-4fe9-b3b4-ec8d00eeebed" Format="Compact25">
    <CustomerId>CHOPS</CustomerId>
    <CompanyName>Foo Inc.</CompanyName>
    <ContactName>Yang Wang</ContactName>
    <ContactTitle>Owner</ContactTitle>
    <Address>Hauptstr. 29</Address>
    <City>Bern</City>
    <PostalCode>3012</PostalCode>
    <Country>Switserland</Country>
    <Phone>555-chang</Phone>
    <Fax>1313-chan</Fax>
    <Orders>
        <OrderEntity ObjectID="d8117794-675c-45ce-bd97-f23c22391039">
            <OrderId>10746</OrderId>
            <CustomerId>CHOPS</CustomerId>
            <EmployeeId>1</EmployeeId>
            <OrderDate>1997-11-19T00:00:00.0000000+01:00</OrderDate>
            <RequiredDate>1997-12-17T00:00:00.0000000+01:00</RequiredDate>
            <ShippedDate>1997-11-21T00:00:00.0000000+01:00</ShippedDate>
            <ShipVia>3</ShipVia>
            <Freight>31,4300</Freight>
            <ShipName>Chop-suey Chinese</ShipName>
            <ShipAddress>Hauptstr. 31</ShipAddress>
            <ShipCity>Bern</ShipCity>
            <ShipPostalCode>3012</ShipPostalCode>
            <ShipCountry>Switzerland</ShipCountry>
            <Customer Ref="df7c7b33-1fd2-4fe9-b3b4-ec8d00eeebed" />
            <NullableDateTime></NullableDateTime>
            <CompanyName>Foo Inc.</CompanyName>
            <NullableDec></NullableDec>
            <NulledString></NulledString>
            <_lps fs="AACAAA==" es="1" />
        </OrderEntity>
        ...
        <OrderEntity ObjectID="80a16cd2-4dd9-4e41-aae0-5a669af76257">
            <OrderId>10731</OrderId>
            <CustomerId>CHOPS</CustomerId>
            <EmployeeId>7</EmployeeId>
            <OrderDate>1997-11-06T00:00:00.0000000+01:00</OrderDate>
            <RequiredDate>1997-12-04T00:00:00.0000000+01:00</RequiredDate>
            <ShippedDate>1997-11-14T00:00:00.0000000+01:00</ShippedDate>
            <ShipVia>1</ShipVia>
            <Freight>96,6500</Freight>
            <ShipName>Chop-suey Chinese</ShipName>
            <ShipAddress>Hauptstr. 31</ShipAddress>
            <ShipCity>Bern</ShipCity>
            <ShipPostalCode>3012</ShipPostalCode>
            <ShipCountry>Switzerland</ShipCountry>
            <Customer Ref="df7c7b33-1fd2-4fe9-b3b4-ec8d00eeebed" />
            <NullableDateTime></NullableDateTime>
            <CompanyName>Foo Inc.</CompanyName>
            <NullableDec></NullableDec>
            <NulledString></NulledString>
            <_lps fs="AACAAA==" es="1" />
        </OrderEntity>
        <_lps f="7" />
    </Orders>
    <_lps fs="ACAA" es="1" />
</CustomerEntity>

LLBLGen Pro will add **_lps** elements to the XML to track the state of entity fields and the entity itself and also the state of entity collections. The data in these elements is compacted to keep everything as small as possible. A decription of the various elements you can encounter in an _lps element is found below.  

_lps element description for Entities

The _lps attribute fs stands for ‘Field state data’ and is a bitarray, with for each field a bit for the IsChanged flag and a bit for the IsNull flag. This bitarray is then packed to a hex string. The order is by fieldindex: the field at index 0 has the first two bits (at index 0 and 1), etc.

There’s no info about IsNew and IsDirty. This is because these are redundant: IsDirty can be determined from the stateflags of all the fields, and IsNew can be determined from the attribute 'es'.

The _lps attribute es stands for ‘EntityState’ and is the numeric value of the EntityState enum for the entity. If there are fields changed, or better: if the CurrentValue for a field is different from the DbValue value, and the DbValue isn’t null/Nothing, an entry is placed for that field in the dbv element as a child of the _lps element with the DbValue value.

This saves data for the fields which aren’t changed. If the entity has an error string set, it’s stored under the ee element as a child of the _lps element. If there are field errors stored, they’re stored under an efes element under the _lps element and every error is stored as a sub element of efes with the key as the elementname and the errorstring as the element value.

_lps element description for EntityCollection instances

The f attribute in the _lps element of the entity collection is a bitarray with the Allow* values in this order (from index 0 upwards) AllowEdit, AllowNew, AllowRemove. At bit pos 3, the DoNotPerformAddIfPresent flag is placed.

Serializing Low-level Query API / Unit of Work instances to XML

LLBLGen Pro Runtime Framework supports serializing / deserializing low-level query API elements like predicates, relations and expressions to and from XML. It also supports the serialization / deserialization of UnitOfWork2 (Adapter's Unit of Work class) to / from XML. See the UnitOfWork2 documentation for details.