Generated code - Setting up and using Auditing

Similar to Authorization, Auditing is a feature which should be present in any modern O/R mapper framework and LLBLGen Pro therefore supports Auditing in full. This section describes the different elements of Auditing, which actions are auditable using the LLBLGen Pro runtime framework and how to setup your own auditing code to work with your entities.

There are two parts of auditing which are essential:

1. Recording of changes and the actions causing the changes
2. Persisting the recorded data.

What exactly is a change? For auditing we consider something a change if the entity is altered in the database. So auditing, while tempting, isn't meant to simply track changes to fields of entities in-memory, it's meant to track changes to entities in the database.

Though to be able to persist / log verbose auditing information, it's necessary to track the field changes in-memory first, to know who changed what and when.

LLBLGen Pro's runtime framework will, when one of the auditable actions occurs, call into the audit logic and let the audit logic record the action and the relevant information passed in. From then on it's up to the audit logic what to do with it: store it for persistence later on, or log it directly somewhere.

When the transaction the entity participates in is committed, the auditor logic is asked for any additional entities it wants to persist inside that transaction. This mechanism is meant for audit logic to produce entities of their choice in which they'll store the recorded data and which are then persisted in the same transaction as the audited entity.

LLBLGen Pro's Auditing feature also offers you to record access to fields, even if they're not changed. This can be used as a journalling feature. To be able to store that particular information in the database however, it's essential that the entity accessed is also changed.

Auditing is designed to be transparent, i.o.w.: it's set-and-forget code, similar to Authorization.

Auditable actions

To be able to have some kind of auditing, there has to be a way to record changes and the actions causing the changes so these can be persisted later on (to the database, to a logfile etc.) as audit data. This means LLBLGen Pro has to notify the audit data recorder logic (in short: recorder) that some change took place and which action caused that change, as well as who made that change.

To be able to make auditing as flexible as possible, the recorder is notified with as much changes as possible, so the developer can make the right decision what to record as audit information and what to ignore.

The following changes and actions are recognized to be useful auditing information.

1. Setting a field value. This action is audited if an entity field is set through an entity property or through SetNewFieldValue, but also if an FK field is synced with a PK field during a recursive save action. Setting an entity field's CurrentValue property or calling an entity field's ForcedCurrentValueWrite isn't recorded.
2. De-referencing a related entity. An example of this action is when the OrderEntity instance X is no longer referencing the CustomerEntity instance C. Another example is removing OrderEntity instance X from the Orders collection of CustomerEntity instance C.
3. Referencing a related entity. An example of this action is when the OrderEntity instance X is now referencing the CustomerEntity instance C. Another example is adding OrderEntity instance X to the Orders collection of CustomerEntity instance c.
4. Saving a new entity.
5. Updating an existing entity. This action also means updating an entity or set of entities directly in the database using a batch statement (entitycollection.UpdateMulti() (SelfServicing) or DataAccessAdapter.UpdateEntitiesDirectly() (Adapter)).
6. Deleting an entity. This action also means deleting an entity or set of entities directly from the database using a batch statement (entitycollection.DeleteMulti() (SelfServicing) or DataAccessAdapter.DeleteEntitiesDirectly() (Adapter)).
7. Getting a field value. This action is audited if the value of an entity field is read through the entity property or when entity.GetCurrentFieldValue() is called. Getting a field's value through the EntityField(2).CurrentValue property isn't recorded.
8. Loading an entity. This action is audited both when an entity is loaded as an individual entity or when it is loaded inside a collection.

Action 7 and 8 are special operations, as there's nothing really changed. These operations are auditable because the complete set of actions is now useful for journaling.

Action 7 and 8 are not necessarily useful to be used to produce entities with audit data to persist into the database, because it doesn’t necessarily have to be the case that the audited entity in question is saved/deleted later on, so the audit information is in that situation not persisted to the database as well, as the audit logic, the recorder, isn't asked for entities with audit data to persist in the same transaction as there is no transaction in progress.

To be able to have reliable journaling, the information has to be logged by an external service. Also take into account for action 7, it might be a lot of audit information is logged, as entity properties can be read a lot of times during the lifetime of an entity.

Location of your Auditing logic

The actions described in the previous paragraph will make the LLBLGen Pro framework to call into the entity the action is applied on to reach the recording logic which should record the action.

There are two main choices where to store the audit logic:

1. Add your action-auditing code to the entity itself
2. Add your action-authorizing code to an Auditor class which is plugged into the entity.

The framework calls all arrive at the protected virtual (Protected Overridable) method for the particular action.

By default, these methods will check if there is an Auditor object available in the entity and if so, will call the equivalent method on that Auditor object, or if there's no Auditor object, they'll simply return. If the method is overriden by you, the logic you've added to that method is used to audit the action, so it's up to you how that logic looks like.

The following list describes all methods called in the entity classes for a given action. It's recommended to also consult the LLBLGen Pro reference manual for more details on these methods, which are defined in the base classes for entities: EntityBase (SelfServicing) and EntityBase2 (adapter).

• OnAuditEntityFieldSet. This method will audit action 1, Setting a field value. If an Auditor object is available in the entity, it will call AuditEntityFieldSet on that Auditor object.
• OnAuditDereferenceOfRelatedEntity. This method will audit action 2, De-referencing a related entity. If an Auditor object is available in the entity, it will call AuditDereferenceOfRelatedEntity on that Auditor object.
• OnAuditReferenceOfRelatedEntity. This method will audit action 3, Referencing a related entity. If an Auditor object is available in the entity, it will call AuditReferenceOfRelatedEntity on that Auditor object.
• OnAuditInsertOfNewEntity. This method will audit action 4, Saving a new entity. If an Auditor object is available in the entity, it will call AuditInsertOfNewEntity on that Auditor object.
• OnAuditUpdateOfExistingEntity. This method will audit action 5, Updating an existing entity for an individual entity update. If an Auditor object is available in the entity, it will call AuditUpdateOfExistingEntity on that Auditor object.
• OnAuditDirectUpdateOfEntities. This method will also audit action 5, Updating an existing entity, but now for updates directly on the database using entitycollection.UpdateMulti() (SelfServicing) or DataAccessAdapter.UpdateEntitiesDirectly() (Adapter). If an Auditor object is available in the entity, it will call AuditDirectUpdateOfEntities on that Auditor object. The entity which contains the changed values is the one used for auditing in this scenario.
• OnAuditDeleteOfEntity. This method will audit action 6, Deleting an entity for an individual entity deletion. If an Auditor object is available in the entity, it will call AuditDeleteOfEntity on that Auditor object.
• OnAuditDirectDeleteOfEntities. This method will also audit action 6, Deleting an entity, but now for deletes directly on the database using entitycollection.DeleteMulti() (SelfServicing) or DataAccessAdapter.DeleteEntitiesDirectly() (Adapter). If an Auditor object is available in the entity, it will call AuditDirectDeleteOfEntities on that Auditor object. Adapter users are required to use the DataAccessAdapter.DeleteEntitiesDirectly() overload which accepts a Type object instead of the one which accepts a name.
• OnAuditEntityFieldGet. This method will audit action 7, Getting a field value. If an Auditor object is available in the entity, it will call AuditEntityFieldGet on that Auditor object.
• OnAuditLoadOfEntity. This method will audit action 8, Loading an entity. If an Auditor object is available in the entity, it will call AuditLoadOfEntity on that Auditor object.

The next section will describe the Auditor classes in more detail.

Auditors

To keep concerns separated, it's often desirable to have the auditing logic placed in a separate class, even in a separate Visual Studio project. To be able to do so in the LLBLGen Pro runtime framework, the Auditing functionality has been designed around the Auditor classes, which implement a common interface, IAuditor.

The Auditor classes follow the same pattern as Authorizers do, described in Setting up and using Authorization. To make life easier for you, you can derive your Auditor classes from the common base class AuditorBase which is a class which simply implements IAuditor using virtual methods.

This means that you only have to override and implement those methods of the interface which you want to use in your particular auditor class. The AuditorBase class also contains for Adapter a couple of method stubs for XML Serialization/Derialization and a utility method which converts a .NET Type of an Entity class into an EntityType value. See for more information about the AuditorBase class the LLBLGen Pro reference manual.

You're free to add the Auditor classes to a separate Visual Studio project than the generated code.

Setting an entity's Auditor

If you've decided to use Auditor classes to place your auditing logic in, you'll be confronted with the question: how to set the right Auditor in every entity object?

You've three options:

1. Setting the AuditorToUse property of an entity object manually. This is straight forward, but error prone: if you forget to set an auditor, auditing isn't performed. Also, entities fetched in bulk into a collection are created using the factories so you have to alter these as well. You could opt for overriding OnInitializing in an entity to add the creation of the Auditor class.
2. By overriding the Entity method CreateAuditor. This is a protected virtual (Protected Overridable) method which by default returns null / Nothing. You can override this method in a partial class or user code region of the Entity class to create the Auditor to use for the entity. The LLBLGen Pro runtime framework will take care of calling the method. One way to create an override for most entities is by using a template. Please see the LLBLGen Pro SDK documentation for details about how to write templates to generate additional code into entities and in separate files. Also please see Adding Adding your own code to the generated classes for details.
3. By using Dependency Injection. Using the Dependency Injection mechanism build into LLBLGen Pro, the defined Auditors are injected into the entity objects for you. This option is the least amount of work.

The example later on in this section will show you how to write a simple Auditor and set it up to be used with Dependency Injection.

Adapter specific: Auditors in XML serialization/deserialization scenarios

If an entity containing an Auditor is passed across tiers for example, by using WCF or Webservices, the data collected inside an auditor has to be serialized into XML and restored when the Auditor is deserialized. To be able to do so, you should override in your Auditor classes the WriteXml and one of the ReadXml methods.

The WriteXml produces the XML for the collected data which is consumable as-is by your ReadXml method. It's also necessary that both sides of a service connection (i.e. client and service) have the same Auditor classes in use with their entities. The Auditor object should return true from the HasDataToXmlSerialize property if there's any data to Xml serialize.

Auto-persisting recorded data

One of the main issues with auditing is how to store the recorded audit data in the same database as the entities audited in such a way that it's transparent, automatic and thus always works? LLBLGen Pro's Auditing functionality therefore asks every Auditor object inside all entities participating in the current Transaction if they've any entities to persist additionally to the entities already in the transaction.

This is the case if the Auditor object wants to store the recorded data in audit entities, designed for that purpose. You're free to decide the layout and type of the audit entities you want to use or if you want to use audit entities at all: for example if you decide to log the audit data using a logging service, you don't need to persist the data also into the database.

When the transaction in progress is about to be committed, the LLBLGen Pro runtime framework will call the method GetAuditEntitiesToSave on all Auditor objects in the entities participating in the transaction. If your Auditor classes have any entities to persist inside the transaction, they should return them when that method is called.

Use an override of the AuditorBase class' version to do so. Do not remove the entities / audit data from the Auditor object after the call to this method, wait for the call to TransactionCommitted. This is because a transaction can fail and be rolled back, which would make your Auditor become empty in a second attempt to persist the entities.

Be sure that audit entities don't reference normal entities which are new or changed. This is because the save logic will do a graph sort and the audit entities would make entities which might have been deleted already to be inserted again.

Controlling transaction creation

With single inserts, updates or deletes, the statements executed are a transaction on its own. When you don't start a transaction manually, LLBLGen Pro will therefore not start a transaction as well in these situations.

This is the case with non-recursive single entity saves which aren't in an inheritance hierarchy and single entity deletes which aren't in a hierarchy. However, to be able to save audit entities in the same transaction as the entity action is performed, an explicit transaction is needed.

You can make LLBLGen Pro's runtime framework create this explicit transaction for you under the hood in these single statement queries by returning true from the Auditor object's method RequiresTransactionForAuditEntities. The method implementation in AuditorBase already returns true, so it's the default.

If you override this method, you can control what to do based on the query statement type passed in. It's recommended to keep audit entity save actions and the entity actions on the database in a single transaction so it's recommended to keep this method as-is and let it return true.

The main reason for that is that if you don't keep them together you can get audit information in your database which isn't reflected by the entity data. This can happen if the entity action on the database fails due to an error, leaving the entity data untouched.

Auditor example

The following example shows a basic auditor implemenation which is usable on all entities in a project. It doesn't audit all actions, as it doesn't override all available audit methods. It is injected via Dependency Injection into all IEntity2 implementing entities, which happen to be all entities in an Adapter project.

To use this auditor in a SelfServicing project, IEntity2 should be changed into IEntity. The example creates AuditInfoEntity instances which are entities mapped onto a simple table in the same database as the entities to audit. The Auditor class is marked as Serializable which means it can be transfered across a remoting boundary.

The DependencyInjectionInfo attribute keeps the defaults for the optional parameters, which means it will inject a new instance in every entity instance, which is required as the Auditor is a stateful class.

/// <summary>Example Auditor class which is usable on all entities in a project.</summary>
[DependencyInjectionInfo(typeof(IEntity2), "AuditorToUse")]
[Serializable]
public class GeneralAuditor : AuditorBase
{
    private enum AuditType
    {
        DeleteOfEntity=1,
        DirectDeleteOfEntities,
        DirectUpdateOfEntities,
        DereferenceOfRelatedEntity,
        ReferenceOfRelatedEntity,
        EntityFieldSet,
        InsertOfNewEntity,
        UpdateOfExistingEntity
    }

    private List<AuditInfoEntity> _auditInfoEntities;

    /// <summary>CTor </summary>
    public GeneralAuditor()
    {
        _auditInfoEntities = new List<AuditInfoEntity>();
    }

    /// <summary>Audits the successful delete of an entity from the database</summary>
    /// <param name="entity">The entity which was deleted.</param>
    /// <remarks>As the entity passed in was deleted succesfully, reading values from the 
    /// passed in entity is only possible in this routine. After this call, the
    /// state of the entity will be reset to Deleted again and reading the fields 
    /// will result in an exception. It's also recommended not to reference
    /// the passed in entity in any audit entity you might want to persist as the entity 
    /// doesn't exist anymore in the database.</remarks>
    public override void AuditDeleteOfEntity(IEntityCore entity)
    {
        var auditInfo = new AuditInfoEntity();
        auditInfo.AffectedEntityName = entity.LLBLGenProEntityName;
        auditInfo.ActionDateTime = DateTime.Now;
        auditInfo.ActionType = (int)AuditType.DeleteOfEntity;
        _auditInfoEntities.Add(auditInfo);
    }

    /// <summary>Audits the successful dereference of related entity from the entity passed in.</summary>
    /// <param name="entity">The entity of which the related entity was dereferenced from.</param>
    /// <param name="relatedEntity">The related entity which was dereferenced from entity</param>
    /// <param name="mappedFieldName">Name of the mapped field onto the relation from entity to related 
    /// entity for which the related entity was dereferenced.</param>
    public override void AuditDereferenceOfRelatedEntity(IEntityCore entity, IEntityCore relatedEntity, 
        string mappedFieldName)
    {
        var auditInfo = new AuditInfoEntity();
        auditInfo.AffectedEntityName = entity.LLBLGenProEntityName;
        auditInfo.ActionDateTime = DateTime.Now;
        auditInfo.ActionType = (int)AuditType.DereferenceOfRelatedEntity;
        auditInfo.ActionData = string.Format("RelatedEntityName: {0}. MappedFieldName: {1}", 
                relatedEntity.LLBLGenProEntityName, mappedFieldName);
        _auditInfoEntities.Add(auditInfo);
    }

    /// <summary>Audits the successful insert of a new entity into the database.</summary>
    /// <param name="entity">The entity saved successfully into the database.</param>
    public override void AuditInsertOfNewEntity(IEntityCore entity)
    {
        var auditInfo = new AuditInfoEntity();
        auditInfo.AffectedEntityName = entity.LLBLGenProEntityName;
        auditInfo.ActionDateTime = DateTime.Now;
        auditInfo.ActionType = (int)AuditType.InsertOfNewEntity;
        _auditInfoEntities.Add(auditInfo);
    }

    /// <summary>
    /// Audits the successful reference of related entity from the entity passed in.
    /// </summary>
    /// <param name="entity">The entity of which the related entity was dereferenced from.</param>
    /// <param name="relatedEntity">The related entity which was dereferenced from entity</param>
    /// <param name="mappedFieldName">Name of the mapped field onto the relation from entity to related 
    /// entity for which the related entity was referenced.</param>
    public override void AuditReferenceOfRelatedEntity(IEntityCore entity, IEntityCore relatedEntity, 
        string mappedFieldName)
    {
        var auditInfo = new AuditInfoEntity();
        auditInfo.AffectedEntityName = entity.LLBLGenProEntityName;
        auditInfo.ActionDateTime = DateTime.Now;
        auditInfo.ActionType = (int)AuditType.ReferenceOfRelatedEntity;
        auditInfo.ActionData = string.Format("RelatedEntityName: {0}. MappedFieldName: {1}", 
            relatedEntity.LLBLGenProEntityName, mappedFieldName);
        _auditInfoEntities.Add(auditInfo);
    }

    /// <summary>
    /// Audits the successful update of an existing entity in the database
    /// </summary>
    /// <param name="entity">The entity updated successfully in the database.</param>
    public override void AuditUpdateOfExistingEntity(IEntityCore entity)
    {
        var auditInfo = new AuditInfoEntity();
        auditInfo.AffectedEntityName = entity.LLBLGenProEntityName;
        auditInfo.ActionDateTime = DateTime.Now;
        auditInfo.ActionType = (int)AuditType.UpdateOfExistingEntity;
        _auditInfoEntities.Add(auditInfo);
    }

    /// <summary>
    /// Gets the audit entities to save. Audit entities contain the audit information stored 
    /// inside this auditor.
    /// </summary>
    /// <returns>The list of audit entities to save, or null if there are no audit entities to save</returns>
    /// <remarks>Do not remove the audit entities and audit information from this auditor when this method is 
    /// called, as the transaction in which the save takes place can fail and retried which will result in 
    /// another call to this method</remarks>
    public override IList GetAuditEntitiesToSave()
    {
        return _auditInfoEntities;
    }

    /// <summary>
    /// The transaction with which the audit entities requested from GetAuditEntitiesToSave were saved.
    /// Use this method to clear any audit data in this auditor as all audit information is persisted 
    /// successfully.
    /// </summary>
    public override void TransactionCommitted()
    {
        _auditInfoEntities.Clear();
    }
}