LLBLGen Pro 2.6 Final (June 6th, 200 RTL 2.6.8.613 Adapter .Net 2.0 SQL 2005

Hi

We have recently upgraded from LLBLGenPro 2.5 to 2.6 and decided to switch FastSerialization on. We have encountered a problem with some generic code that is used fairly extensively throughout our application (which is still under development) and any advice would be greatly appreciated.

It is a "client/server" Windows Forms application that uses .NET Remoting and as a result a decision was taken to use datatables to populate most of the datagrids on the setup forms in an effort to minimise the size of the traffic. The following eventhandler fires on the client form when a user selects a datagrid row for editing;

private void grdPublicHoliday_PerformFetchEntity(object sender, DataGridView.PerformFetchEntityEventArgs e)
        {
        IList<PublicHolidayEntity> preLoaded = LLBLGenUtilities.CreatePreloadedEntities<PublicHolidayEntity>(e.PrimaryKeyValues);
        e.FetchedEntities.Add(Facade.CodeLibrary.FetchEntity(preLoaded[0]));
    }

The problem comes in with the call to "Facade.CodeLibrary.FetchEntity", which is on the server. The client is attempting to pass a "preloaded" entity created by the method below but although the type/entity makes it through to the server, it's "blank", the data does not get serialized, and so therefore the FetchEntity call fails because there is no primary key value available.

The following utility method is pretty self-explanatory and is used in a number of different contexts throughout the application for creating "lightweight" entities for various purposes without the need for excessive client-server round-trips;

public static IList<EntityType> CreatePreloadedEntities<EntityType>(IList<IDictionary<string, object>> values)
                where EntityType : IEntity2, new()
        {
            IList<EntityType> list = new List<EntityType>(values.Count);
            foreach (IDictionary<string, object> fields in values)
            {
                EntityType entity = new EntityType();

                foreach (KeyValuePair<string, object> fieldValue in fields)
                {
                    IEntityField2 field = entity.Fields[fieldValue.Key];
                    if (field != null)
                    {
                        field.ForcedCurrentValueWrite(fieldValue.Value);
                        field.IsChanged = false;
                    }
                }

                //this is here to support a unit of work. 
                entity.IsNew = false;
                entity.IsDirty = false;

                list.Add(entity);
            }

            return list;
        }

However, I have noticed that if I edit the "grdPublicHoliday_PerformFetchEntity" eventhandler and add twoone lines just before the "FetchEntity" call;

private void grdPublicHoliday_PerformFetchEntity(object sender, DataGridView.PerformFetchEntityEventArgs e)
        {
        IList<PublicHolidayEntity> preLoaded = LLBLGenUtilities.CreatePreloadedEntities<PublicHolidayEntity>(e.PrimaryKeyValues);
        //preLoaded[0].IsNew = true;
                                 preLoaded[0].IsDirty = true;
        e.FetchedEntities.Add(Facade.CodeLibrary.FetchEntity(preLoaded[0]));
    }

then it seems to serialize the entity completely without any trouble.

In the course of searching the archives I noticed previous postings that mention differences between standard and fast serialization in the way that saved fields are dealt with and I was wondering is this still the case and the possible cause of our problem?

The problem comes in with the call to "Facade.CodeLibrary.FetchEntity", which is on the server. The client is attempting to pass a "preloaded" entity created by the method below but although the type/entity makes it through to the server, it's "blank", the data does not get serialized, and so therefore the FetchEntity call fails because there is no primary key value available.

If you want to fetch an entity, then why passing a preloaded entity/empty entity. Why not pass the PK field value, and return the entity from the service method.

            foreach (KeyValuePair<string, object> fieldValue in fields)
            {
                IEntityField2 field = entity.Fields[fieldValue.Key];
                if (field != null)
                {
                    field.ForcedCurrentValueWrite(fieldValue.Value);
                    field.IsChanged = false;
                }
            }

If the field is not seen to be changed (IsChanged = false), then it's CurrentValue will be ignored. So you may add a check if the field is a PK, then set the IsChanged to true.

Hello Walaa, Thank you for your quick response.

Actually, the whole idea was to create one generic method to fetch any entity, rather than a separate service method for each different entity type. It has been working well up until now when we switched FastSerialization on.

And in fact I had originally experimented with the original example of fetching an entity from the LLBLGen documentation which does something like;

PublicHolidayEntity publicHolidayEntity = new PublicHolidayEntity(46);
e.FetchedEntities.Add(Facade.CodeLibrary.FetchEntity(publicHolidayEntity));

So what you are saying is that FastSerialization will only serialize fields with IsChanged = true?

I've tried setting this value explicitly, but it doesn't seem to have any effect.

//preLoaded[0].IsNew = true;
            //preLoaded[0].IsDirty = true;
            for (int i = 0; i < preLoaded[0].Fields.PrimaryKeyFields.Count; i++)
            {
                preLoaded[0].Fields.PrimaryKeyFields[i].IsChanged = true;   
            }

However, setting "entity.IsDirty = true;" in the client-side eventhandler seems to fix it and I was trying to understand the behaviour.

Thanks

Ian.

When you set a field in code IsChanged is set to true, and the entity.IsDirty is automatically set to true too. Since you hack it with ForcedCurrentValueWrite(), then you should set these flags yourself.

Hi

I think we have/are losing the point. Let me try to simplify and summarise;

1. I create a basic "preloaded" entity client-side, with only the primary key data populated (using the custom generic CreatePreloadedEntities method detailed earlier).
2. I attempt to pass this preloaded entity to a custom generic "FetchEntity" method on the server, using .NET Remoting, but the PK data is not serialized if FastSerialization is switched on.
3. This works fine as soon as I switch FastSerialization off.
4. I experiment with explicitly setting IsChanged = true on the PK field as advised but to no effect - the data is still not serialized.
5. I experiment with explicitly setting IsDirty = true on the preloaded entity that is being passed through and that results in the data being serialized.

This seems to say to me that FastSerialization will only serialize entities with IsDirty=true but surely that cannot be correct?

oneness wrote:

This seems to say to me that FastSerialization will only serialize entities with IsDirty=true but surely that cannot be correct?

I'm not sure, but the logic says me that if the field is not dirty (IsDirty == false) only DbValues for Entities that are not Dirty are serialized. In your case you are populating (forcing to write) the CurrentValue and staying that the field is not dirty, so that will result in NULLs values serialized (my guess). It might be that some serialization optimizer look at the IsDirty property to select between DBValue and CurrentValue. To workaround this you should specify that the value is dirty (IsDirty == true) so the CurrentValue would be used. Another thing you could try (following this logic) is to write the DBValue of the field.

Ok wait, let me try rephrasing and restating -

If the following code worked without any trouble when FastSerialization was switched off, what is FastSerialization doing behind the scenes that is causing the entity that is being passed not to be serialized? When I examine "preLoaded[0].PublicHolidayId" in Debug mode on the last line of the client-side eventhandler below, it has a valid integer value but it does not have a PK value when I examine it in the "FetchEntity" method on the server side?

Client-side eventhandler:

private void grdPublicHoliday_PerformFetchEntity(object sender, DataGridView.PerformFetchEntityEventArgs e)
        {
         // We use the utility function to project onto a newly created entity, the keys retrieved from the grid. 
         //It is important to note that we do NOT need to use the grid to get the selected rows etc, 
         //because the entities that need to be fetched from the db,
         // will have their primary keys in the eventargs for this function (as seen by e.PrimaryKeyValues).

         IList<PublicHolidayEntity> preLoaded = 
LLBLGenUtilities.CreatePreloadedEntities<PublicHolidayEntity>(e.PrimaryKeyValues);

         //we add the entities to the entities collection. We could have elected to rather load all the items
        // in the list in a more generic manner and cater for the fact that there could be more than one selected item that needs to be
        // fetched. However, in the context of this form, there is no need as we never need to "edit" more than one record at a time.
        // Keep in mind that this event handler is fired to any request to SelectedEntities (because the underlying datasource is a 
        // datatable and not an entitycollection. Therefore in this current implementation there will only ever be one fetched item
        // in the collection returned by SelectedEntities.

       e.FetchedEntities.Add(Facade.CodeLibrary.FetchEntity(preLoaded[0]));

}

Custom LLBLGenUtilities function:

public static IList<EntityType> CreatePreloadedEntities<EntityType>(IList<IDictionary<string, object>> values)
                where EntityType : IEntity2, new()
        {
            IList<EntityType> list = new List<EntityType>(values.Count);
            foreach (IDictionary<string, object> fields in values)
            {
                EntityType entity = new EntityType();

                foreach (KeyValuePair<string, object> fieldValue in fields)
                {
                    IEntityField2 field = entity.Fields[fieldValue.Key];
                    if (field != null)
                    {
                        field.ForcedCurrentValueWrite(fieldValue.Value);
                        field.IsChanged = false;
                    }
                }

                // This is here to support a unit of work. 
                entity.IsNew = false;
                entity.IsDirty = false;

                list.Add(entity);
            }

            return list;
        }

And for completeness here is the custom FetchEntity facade method, although the entity that is being passed through is already blank by the time it comes into this method. Also, as a matter of interest, this method is not overloaded at all;

public virtual T FetchEntity<T>(T entity, params IEntityField2[] includeFields) where T : IEntity2
        {
            this.LLBLGenPattern.PerformTryCatch(
                delegate(IDataAccessAdapter adapter)
                {
                    //Used to determine if the entity has been fetched yet, defaults to false.
                    bool fetched = false;

                    //that there are fields to include, if not default to all fields.
                    if ((includeFields != null) && (includeFields.Length > 0))
                    {
                        //Create a list of fields to include during fetch.
                        ExcludeIncludeFieldsList includeFieldList = new ExcludeIncludeFieldsList();

                        //Only include the fields in the list.
                        includeFieldList.ExcludeContainedFields = false;

                        //Add specified fields to the field include list.
                        foreach (IEntityField2 field in includeFields)
                        {
                            includeFieldList.Add(field);
                        }
                        //Check that fields where added (if not fetch all fields).
                        if (includeFieldList.Count > 0)
                        {
                            //Fetch the entity and retrieve only the specified fields.
                            adapter.FetchEntity(entity, null, null, includeFieldList);

                            //Entity has been fetched.
                            fetched = true;
                        }
                    }
                    if (!fetched)
                    {
                        //Entity was not fecthed, so retrieve all fields.
                        adapter.FetchEntity(entity);
                    }
                }
            );

            return entity;
        }

Part of the reason I'm asking what FastSerialization does differently is because previous forum postings, albeit old ones, hinted that might be the case?

http://www.llblgen.com/TinyForum/Messages.aspx?ThreadID=9341&HighLight=1

Just to point out that FastSerialization behaviour (as in this particular set of code) is different to standard LLBLGenPro serialization here - standard behaviour is to not serialize SavedFields at all - there was a thread on this ISTR. Since you have demonstrated a need to preserve SavedFields, you might want to check with Frans about what the planned behaviour for v2.1 is going to be.

Cheers Simon

PS Ironically, I did have a test for SavedFields but because my tests also use non-fast serialization for comparison and the standard do-not-serialize-saved-fields was causing a mismatch, I disabled the test 'temporarily' but forgot to add it back - that'll teach me.

You see, the CreatePreLoadedEntities method is used fairly extensively throughout the application in a number of different contexts and so modifying it is not really an option - and I see no reason why it should be at this stage if it hasn't posed any problems up to now?

Also, I don't understand the problem in the logic with the IsDirty, IsNew and IsChanged flags when creating a "preloaded" entity for the purpose of fetching an existing entity back from the dbase? We aren't creating or updating new data so why should any of those flags be true anyway?

Please bear with me, I'm really just trying to understand what is actually happenning so that I can avoid just putting a hack in.

Cheers

Ian.

These flags determine the state of the entity, and FastSerialization routine depends on these flags to determine which data to serialize to keep the size of the serialized data as lean as possible.

Hi

Ok, if that is the case, can you please confirm the rules that FastSerialization uses to determine what data will be serialized?

FastSerialization will serialize if; 1. Entity.IsDirty = true/false; 2. Entity.IsNew = true/false; 3. Entity.Fields[i].IsChanged = true/false;

Are there any other factors/properties of an entity that also affect whether or not data is serialized when FastSerialization is switched on?

Thanks

Ian

Ok, if that is the case, can you please confirm the rules that FastSerialization uses to determine what data will be serialized?

FastSerialization will serialize if; 1. Entity.IsDirty = true/false; 2. Entity.IsNew = true/false; 3. Entity.Fields[i].IsChanged = true/false;

Normally when a field is set, it will have it's IsChanged set to true, and also the entity's IsDirty will be set to true.

This should be the case whether you have created a new entity and have set one or more fields. Or you have updated one or more fields of an entity which was fetched from the database.

So the logic checks for entity.IsDirty, if true it serialized the currentValue of the entityFields. If not, it serialized the DBValue of the entityFields. (since it assumes currentValue = DBValue in this case).

So if you manually force the a field set, and also you set teh IsChanged to true, you should go all the way and set IsDirty to true. (since this is how it goes when you normally set a field).

Are there any other factors/properties of an entity that also affect whether or not data is serialized when FastSerialization is switched on?

Nope.

But, if you want to chec the code/logic, please have a look at the following methods in the ormsupportclasses source code.

1- EntityBase2.GetSerializationFlags(); 2- EntityBase2.SerializeOwnedData(SerializationWriter writer, object context); 3- SerializationHelper.SerializeEntityFields(SerializationWriter writer, IEntityFields2 fields, bool isDirty)

To give some background on the topic: The problem is mainly caused by: field.ForcedCurrentValueWrite(fieldValue.Value);

this sets the CurrentValue but not the DbValue. Fastserialization serializes the DbValue when the entity isn't dirty (as both should be the same). See SerializationHelper.cs, line 174.

To fix it, use: field.ForcedCurrentValueWrite(fieldValue.Value, fieldValue.Value);

this will set both the dbvalue and the currentvalue and it will be ok (your initial code). Normally one wouldn't run into this as DbValue is either not set (new entity, which has either the CurrentValue set (isdirty) or it's completely blank (isnew with no values set) and therefore doesn't need to be serialized at all (the fields))) or it's set with the value from the DB (after a fetch).

FastSerialization is about squeezing as much size and speed out of the serialization process, so every bit which doesn't have to be transferred is removed. Simon Hewitt who wrote the code has done a tremendous job in this area, but as it does optimize a lot, it of course has to make assumptions and you touched an edge case where its optimizations left you in the cold.

Please post back if the suggestion I mentioned above doesn't work for you. (you can just post a new message in this thread, it will be opened again automatically)

(edit) In v3's RTL this is changed to serialize CurrentValue instead of DBValue. The code is a leftover from v1's code where CurrentValue could contain the default value of a type (e.g. '0') if the field was nullable and not set to a value. In v2 we dropped that approach.

Hi Frans

Thank you, that makes complete sense and of course it seems clear, logical, simple and straightforward now. So why didn't I think of it?

I have made the change and run a quick test and it seems to work perfectly now. And I can't see that the modification will have any effect on it's usage anywhere else in the application either.

Thanks once again.

Chjeers

Ian.

p.s. and thanks also to Walaa and daelmo for your patience and perseverance, it's much appreciated!