Data Scopes
The main point of Data scopes is to make life extremely easy for developers who have to consume and work with entities, e.g. in UIs. Data Scopes are illustrated in an example project which is available on github in the example repository. It's highly recommended you look at this example to see what Data Scopes can do for you. The example is winforms, however it can be equally used in WPF based applications.
In short, a data scope is a class which derives from the base class DataScope which defines which entities to fetch and persist, so which entities are in scope. At the heart of the data scope is a souped up Context class which is notified for every entity addition to the graph, as every entity added to the graph in the scope is added to the context automatically. Data Scopes are fully async / await aware. See for details regarding async / await the section below.
The core functionality of a data scope:
- The data scope keeps track of which entities are new, deleted and updated and will commit all the actions in one go
- The data scope is used to define which data to fetch with one method call on the scope
- The data scope is used to mark collections as read-only, edit only, or that add-new is allowed or not.
- The data scope is used to contain all data of a scope and manage that data / entities for the developer.
- The data scope is used as a controller to know whether things changed, deep inside the scoped graph.
The main goal is that the developer only has to define the query / queries to fetch the data for the scope and bind the data to controls and the rest is taken care of: all overhead of managing entities, the scope takes that work away.
Links to related documentation are pointing to Adapter variants if there is both an Adapter and SelfServicing version, however Data Scopes are supported on both SelfServicing and Adapter with the same functionality.
This section first describes the core functionality in detail, and then gives two examples of Data Scopes from the example project to illustrate what Data Scopes look like.
Core functionality
This section describes more of the core functionality offered by a data scope.
Using Data Scopes with direct or external data-access
The FetchDataImpl method can be used to fetch data directly or obtain it from a service. If the data is fetched directly, the query should be tracked, by calling TrackQuery. This makes sure the fetch will use the Data Scope's context and all data fetched (and with selfservicing, also the data fetched through lazy loading later on) is tracked by the Data Scope.
If the data has to be obtained from a service, the FetchDataImpl method implementation should be used to obtain the data from the service and should use Attach(entity) or Attach(collection) to add the entities in the graph obtained from the service to the datascope. The attach methods are recursive, so all referenced entities by the entities passed to Attach() are added as well.
The Attach methods are protected to provide a way to build a custom interface on the scope class without exposing generic logic to add every entity you want for deletion.
Fetching data for the Data Scope
To tell the scope to fetch its data is done by calling its FetchData() method. FetchData accepts zero or more parameters which are passed to the FetchDataImpl method, which is a method implemented in the DataScope derived class. FetchDataImpl is an abstract method, which leaves the complete implemention of any data fetching to you, the scope itself doesn't fetch anything.
If you need a class to fetch data for you, e.g. an IDataAccessAdapter implementation, pass it to FetchData, and it's passed to FetchDataImpl as well.
FetchData() does not reset the Data Scope. This means that subsequentially calling FetchData() will not remove data already in the scope from it. It might be this is on purpose, but it also might be you want to start with a clean scope. In that case call Reset() before calling FetchData().
Handling tracking of removed entities
To be able to keep track of entities deleted from collections, the scope automatically inserts removal tracker collections inside each collection in an entity added to the scope. Per entity type of a collection a removal tracker is used to keep the number of collections within limits. When the changes have to be committed, these tracker collections are used to gather the entities to remove.
Handling cascading deletes / updates
Consider the following example: a scope contains the graph Customer, its Orders and for each order its OrderDetail entities. All is fetched with 1 query using a prefetch path. The Customer, Orders and OrderDetails are bound to a user interface, and the user can navigate through the entities with it. The user then decides to remove an Order, by pressing DEL. This removes the Order entity from the Orders collection inside the Customer, and adds it to the removal tracker.
When the user then clicks 'Save', which makes the user interface call the Commit method on the Data Scope, a problem arises: the Order entity is to be deleted, but its contained OrderDetail entities have to be removed first, but those aren't in the set of entities to delete.
Normally, without a Data Scope, the developer adds the entities in the right order for deletion to a Unit of Work, but in this case, that's not doable. With a Data Scope, this is taken care of.
The Data Scope will build a list of actions what to delete first. Say, an Audit entity is also referencing the Order deleted in the user interface, however this time, it's inside the database and the Audit entity isn't fetched. The list of actions to perform for the removal of this particular Order therefore becomes:
- Delete, directly on the DB, the Audit entity / entities, which point to the Order we're about to delete. (it might be you want to preserve the Audit entity, in which case you'd opt for a NULL in the FK field, but in this example we go for removal)
- Delete all OrderDetails contained by the Order in memory. However, this might not be enough: perhaps we filtered the number of OrderDetail entities during the fetch, so we have to delete the OrderDetails entities directly
- Delete the Order.
In short, the Data Scope builds a list of delete queries which have to be executed directly onto the DB, and which have to be executed prior to the delete action of the actual entity. It might be entities which have to be removed first are also pointed at by entities which are in the DB (e.g. the Audit entity is on the PK side of another relationship), these entities have to be removed as well.
To help the Data Scope, a developer overrides the OnEntityDelete method to specify the relationships and what cascading action to take (update FK fields or delete).
Deleting entities which are already fetched
The entities in the scope which depend (indirectly) on the entity to delete are deleted using a Unit of work, by building a list of depending entities, traversing the tree, with the entity to delete as root. After that all entities also in the list of depending entities are added in that order to the Unit of work to be deleted.
Entities not in the scope which are depending on the entity to delete, have to be deleted using additional queries. The developer can add entity relationships in a given order, which are used to build predicates. This is done with the following rules:
- The relationships added are the PKSide / FKSide relationships
- The relationships have to be added in the order from root of graph to leaf, so starting with the entity to delete.
The scope uses the same logic used by prefetch path queries to build a predicate from the relationship. It builds a cache of filters, which are re-used when the relationships are consumed, so a dependency of A<-B<-C<-D will re-use the filter on B for the filter on C and the filter on D will re-use the filter on C (which contains the filter on B).
Unit of Work work order
The work order in which the blocks are executed is changed from the default: the direct deletes are executed before the entity deletes. This makes sure the depending entities are indeed deleted at the right spot. When depending entities are in a Target Per Entity inheritance hierarchy, they can't be deleted directly.
This means the entities have to be added to the Unit of Work to be deleted from memory. To do so, the Data Scope allows the developer to fetch a graph of entities which are then added to the Unit of Work for deletion. These entities are added before the main entities to delete, so they're not influencing the delete process.
Important is that the developer shouldn't worry about entities depending on these entities when adding the entities to the Unit of Work, so this should be done indirectly with a method in the scope.
Updating the FK to null instead of Delete
In the above example, it might be you don't want to delete Audit entities, but set their FK field to NULL, so the Order entity they reference is removed but we can keep the Audit entity without violating referential integrity constraints. Data Scopes give you the choice to opt for an Update of the FK fields to NULL instead of deleting the depending entity.
If an entity X is depending on an entity Y, and the FK field(s) in X are nullable, it's possible to set these to NULL instead of deleting X when Y is deleted. By default, the FK is set to null, if at least 1 field is nullable and the FK field isn't part of the PK. Non-null FK fields in a compound FK where one or more FK fields are nullable are set to a default value for the .NET type. E.g. 0 for int typed fk fields.
The developer can specify for a relationship specified for deletes to set the FK side to NULL instead of deleting the entity by specifying the CascadeActionType.
This is required if the relationship is a relationship with self: when a relation with self is part of the edges to traverse and resolves to delete queries, the relationship always has to have a nullable FK. If not, an error will be thrown and the cascades will stop: deleting the depending entities could whipe out the entire table. It's however up to the developer to specify this.
All entities in memory which are depending on a given entity to be deleted are also deleted, not updated.
Re-parenting 1:n relationships
When an entity on the Fk side of a 1:n relationship gets a new PK side, it's called 're-parenting': it gets a new 'parent'. Say we want to assign an order O from employee E1 to employee E2. This is done by either placing O in E2.Orders, which will automatically make sure everything is synced up properly, or by setting O.Employee to E2.
The effect of this is that O is removed from E1.Orders. This removal would normally mark it as a delete action, and it's placed in the removal tracker inside E1.Orders. As O is placed after that in E2.Orders, its MarkedForDeletion flag is then reset.
The Unit of work builder in the Data Scope will filter out entities in removal trackers which have MarkedForDeletion set to false. This will make sure re-parented objects are not seen as 'removed'.
Re-parenting 1:1 relationships
With 1:1 relationships where the FK side is set to a new PK side is a little different. The scope has no way of tracking these associations and therefore will not see it as a potential removal action. For 1:1 associated entities which have to be removed, the developer has to write custom code to add the entity to the scope as removed. See next section.
Marking single entities or full collections for deletion
If an individual entity has to be deleted, or a collection of entities, which aren't in the scope, have to be deleted, they can be added for deletion or marked for deletion by calling the method MarkForDeletion(entity/collection) on the Data Scope.
To mark an entity for deletion, the developer has to add methods to the scope class which call the protected method MarkForDeletion(entity). This will add the entity, if not present in the scope already, to the scope and store it in an internal cache so it can be deleted when the work is commit.
To mark a collection of entities, the developer calls the MarkForDeletion(collection) overload which follows the same procedure: it will add all entities in the collection to the scope and store them for deletion.
The internal cache will use a tracker collection of its own: when the entity is added to the scope for deletion, its MarkedForDeletion flag is set to true. However when the entity is placed in a collection somewhere, its MarkedForDeletion flag is reset and the entity is not deleted.
The methods are protected and not public to provide a way to build a custom interface on the scope class without exposing generic logic to add every entity you want for deletion.
Refetch strategy for saved entities after Commit
Normally, entities are saved and not refetched. This can be a sufficient strategy if the scope is abandoned after the save action. If this isn't the case, it can be a problem as entities are perhaps re-read and are then out of sync. you can specify the refetch strategy for the Data Scope. There are three options, by using the values of the enum DataScopeRefetchStrategyType:
- DoNothing (leave it as-is) and don't refetch.
- AlwaysRefetch
- MarkSavedEntitiesAsFetched. This is done after commit.
To specify what the Data Scope should do with saved entities, pass the enum DataScopeRefetchStrategyType to the Data Scope's constructor. You can also change it on-the-fly by altering the property RefetchStrategy on the Data Scope.
Controlling aspects of contained collections
Collections which are exposed through the contained entities and which are bound to controls, are exposing themselves through Entity Views. The aspects of these views are by default: editable, allow new and allow delete. This might not be what the you want.
To keep things automated, and have the user interface be created from what the collections expose, it's possible to order the scope to set allownew/edit/delete on entity views exposed by collections in entities in the Data Scope.
To do so, you pass names of navigator properties to the Data Scope by calling SetNavigatorAspects using the generated MemberNames static class of entity classes, which is available as a static property conveniently called MemberNames. This way, name changes are detected by the compiler. The method SetNavigatorAspects is a protected method of the datascope.
By default, AllowNew, AllowDelete and AllowEdit are set to true except for m:n based collections, which have AllowNew, AllowDelete and AllowEdit set to false.
Defaults can be set, which are used for every navigator, and which are overriden by the navigator specific aspects set by SetNavigatorAspects. The defaults are set through OverrideDefaultNavigatorAspects. M:n based collections are readonly by themselves, and the views won't change behavior when their allownew/edit/delete flags are set to a different value: setting aspects on these navigators has no effect.
If no aspects are defined through SetNavigatorAspects, the defaults take effect. If there are aspects set through SetNavigatorAspects, these are the aspects used on the navigator, all defaults are ignored. If the navigator specified in SetNavigatorAspects isn't a collection, the aspects are ignored.
Commiting changes
After the modifications have been done to the entities in the datascope, changes have to be committed. The Data Scope offers a direct way to do that with the CommitChanges(ITransactioncontroller) method. This method calls BuildWorkForCommit to create a Unit of Work object which contains all the actions to perform to reflect the changes made to the entities in the database.
A ITransactionController is for adapter a
DataAccessAdapter instance, and for SelfServicing, a Transaction
instance. The object specified is passed to the Commit method of the
generated Unit of Work object. If the passed in ITransactionController
has a transaction open, the CommitChanges method won't commit the
transaction, otherwise it will auto-commit the transaction, equal to
Commit on a Unit of Work object.
If the data has to be sent to a service or another tier inside the application, you should call the CommitChanges(Func<IUnitOfWorkCore, bool>) instead. This method creates a Unit of Work and passes it to the passed in func to commit. The func passed in should be a method call to a method which does the actual commit action of the Unit of Work, e.g. by passing it to a service or tier. The method called by the func should return true on success and false on failure.
Starting with v5.8.1, there's also an async variant of CommitChanges(Func), CommitChangesAsync(Func<IUnitOfWorkCore, CancellationToken, Task<bool>>, CancellationToken), which executes the passed in func asynchronously. The passed in cancellation token is passed to the func as the second parameter, so there's no need to embed the cancellation token within the func itself.
Detecting changes in entities inside the Data Scope
The Data Scope tracks changes to all the elements in the scope. If you want to detect whether something changed, e.g. the user changed a value of an entity in a collection deep in the entity graph of the Data Scope. To detect these changes (for example to enable a Save button), you bind an event handler to the Data Scope's ContainedDataChanged event. This event is raised when a change is detected in an entity which is in scope. During fetches, this event isn't raised.
Async / Await and Data Scopes
Data Scopes are fully async / await aware and offer async variants of the synchronous methods discussed above. Async methods can be used e.g. on async event handlers in WPF applications. To implement the async variants instead of the sync variants, simply implement FetchDataAsyncImpl instead of FetchDataImpl. FetchDataAsyncImpl is called by FetchDataAsync, which is the method you'll be using to fetch data, instead of the FetchData method. Calling FetchData instead will result in 'false' being returned illustrating a failed fetch.
The same is true if you don't implement FetchDataAsyncImpl and call FetchDataAsync: false is returned illustrating a failed fetch. This way you can decide what to implement, e.g. both or just one variant. You don't need to implement both if you just use e.g. async or only the synchronous variant. The async variant of CommitChanges is CommitChangesAsync. It works the same way as CommitChanges but performs the commit of the changes asynchronously.
To use the async variants, you have to reference the .NET 4.5+ build of the ORMSupportClasses, See compiling your code.
Example usage of a Data Scope
Below is an example of a Data Scope, with comments what each portion of the code does. Below the example Data Scope, it's usage is illustrated. This example is also available in the LLBLGen Pro examples on github linked at the start of this section.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using SD.LLBLGen.Pro.QuerySpec;
using SD.LLBLGen.Pro.ORMSupportClasses;
using SD.LLBLGen.Pro.QuerySpec.Adapter;
using Northwind.DAL.EntityClasses;
using Northwind.DAL.HelperClasses;
using Northwind.DAL.FactoryClasses;
using Northwind.DAL.DatabaseSpecific;
using Northwind.DAL;
namespace Northwind.BO
{
public class CustomerWithOrdersDataScope : DataScope
{
#region Class Member Declarations
private CustomerEntity _customer;
private string _customerID;
#endregion
public CustomerWithOrdersDataScope()
{
_customerID = string.Empty;
}
/// <summary>Adds the specified entity</summary>
/// <param name="toAdd">To add.</param>
/// <returns></returns>
public CustomerEntity Add(CustomerEntity toAdd)
{
if(toAdd != null)
{
_customer = toAdd;
this.Attach(toAdd);
}
return toAdd;
}
/// <summary>Called when this scope is reset.</summary>
protected override void OnReset()
{
base.OnReset();
// reset all navigator aspects of all collection navigators
this.OverrideDefaultNavigatorAspects(NavigatorAspectType.None);
// set the navigator aspects of Order.OrderDetails.
this.SetNavigatorAspects<OrderEntity>(OrderEntity.MemberNames.OrderDetails,
NavigatorAspectType.AllowNew |
NavigatorAspectType.AllowEdit |
NavigatorAspectType.AllowRemove);
}
/// <summary>
/// The implementation of the fetch method. In this method, fetch the data for the
/// scope. Use the TrackQuery methods to make sure
/// all entities fetched in this method are tracked by the scope.
/// </summary>
/// <param name="fetchMethodParameters">The fetch method parameters.</param>
/// <returns>true if the fetch was successful, false otherwise</returns>
protected override bool FetchDataImpl(params object[] fetchMethodParameters)
{
// use DI scope here to set the injectables during fetch.
using(var adapter = new DataAccessAdapter())
{
// Fetch using a query, which is tracked
_customer = adapter.FetchFirst(this.TrackQuery(CreateCustomerFetchQuery()));
// or alternatively, fetch it using a normal non-tracked query or obtain it from a service
//_customer = adapter.FetchFirst(CreateCustomerFetchQuery());
// ... and attach it. All entities reachable from _customer are attached as well.
//Attach(_customer);
}
return _customer != null;
}
/// <summary>Creates the customer fetch query.</summary>
/// <returns></returns>
private EntityQuery<CustomerEntity> CreateCustomerFetchQuery()
{
var qf = new QueryFactory();
return qf.Customer
.Where(CustomerFields.CustomerId == this.CustomerID)
.WithPath(CustomerEntity.PrefetchPathOrders.WithSubPath(
OrderEntity.PrefetchPathOrderDetails
.WithSubPath(OrderDetailEntity.PrefetchPathProduct),
OrderEntity.PrefetchPathEmployee));
}
/// <summary>
/// Called when toDelete is about to be deleted. Use this method to specify work to be done by the scope to
/// avoid FK constraint issues. workData is meant to collect this work. It can either be additional entities to
/// delete prior to 'toDelete', or a list of relations which are used to create cascading delete actions executed
/// prior to the delete action of toDelete.
/// </summary>
/// <param name="toDelete">To delete.</param>
/// <param name="workData">The work data.</param>
protected override void OnEntityDelete(IEntityCore toDelete, WorkDataCollector workData)
{
switch((EntityType)toDelete.LLBLGenProEntityTypeValue)
{
case EntityType.OrderEntity:
// only add edge for entities not already in the graph. All order details are in the graph.
workData.Add(OrderEntity.Relations.OrderAuditInfoEntityUsingOrderId);
// alternatively, we could opt for fetching the data to delete.
//var qf = new QueryFactory();
//var q = qf.OrderAuditInfo.Where(OrderAuditInfoFields.OrderId == ((OrderEntity)toDelete).OrderId);
//workData.Add(new DataAccessAdapter().FetchQuery(q));
break;
case EntityType.CustomerEntity:
workData.Add(CustomerEntity.Relations.CustomerCustomerDemoEntityUsingCustomerId);
workData.Add(CustomerEntity.Relations.OrderEntityUsingCustomerId);
workData.Add(OrderEntity.Relations.OrderDetailEntityUsingOrderId);
workData.Add(OrderEntity.Relations.OrderAuditInfoEntityUsingOrderId);
break;
}
}
#region Class Property Declarations
public string CustomerID
{
get { return _customer == null ? null : _customer.CustomerId; }
set
{
if(_customer == null)
{
_customerID = value;
}
}
}
public CustomerEntity Customer
{
get { return _customer; }
}
public EntityCollection<OrderEntity> Orders
{
get { return _customer==null ? null : _customer.Orders; }
}
#endregion
}
}
The code in the scope is straight forward: a place to define the query (FetchDataImpl), a place to define relationships for cascade deletes (OnEntityDelete) and a place to define navigator aspects (OnReset).
The scope exposes its data through two properties: Customer, and Orders, which are the Customer's Orders. The example code below, which is a form which shows the Customer data, the Orders of the customer and the OrderDetail entities of the current selected Order, contains no real logic with respect to entity management: everything is outsourced to the Data Scope.
This separates the UI code from the entity management and also doesn't force you to write a lot of code. As the Data Scope defines a clear set with behavior and it's not tied to a UI, you can re-use this scope class for multiple views / forms.
The code is winforms, but e.g. in WPF you can use similar constructs.
Additional winforms code for the actual layout of the form is omited. For the complete form/example, please see the example on github, linked in the Preface.
public partial class CustomerManagerDialog : System.Windows.Forms.Form
{
private CustomerWithOrdersDataScope _customerScope;
public CustomerManagerUsingBO()
{
InitializeComponent();
_customerScope = new CustomerWithOrdersDataScope();
// bind to the event handler so we can enable the Save button
_customerScope.ContainedDataChanged += new EventHandler(_customerScope_ScopedDataChanged);
}
public CustomerManagerUsingBO(CustomerEntity customer) : this()
{
SetCustomerAsCurrent(customer);
}
/// <summary>
/// Binds the current set customer (in _currentCustomer) to the gui's
/// controls so data entered is validated directly.
/// </summary>
private void BindCustomerToGui()
{
if(_customerScope==null)
{
return;
}
_customerBindingSource.DataSource = _customerScope.Customer;
_saveCustomerButton.Enabled = false;
}
/// <summary>
/// Sets the given customer as the current customer, adding values of
/// the customer to the controls on the form and setting
/// up eventhandlers so the gui is aware of changes of the customer or its contents.
/// </summary>
/// <param name="customer"></param>
private void SetCustomerAsCurrent(CustomerEntity customer)
{
// set new customer object.
_customerScope.Reset();
_customerScope.Add(customer);
_customerScope.FetchData();
BindCustomerToGui();
}
/// <summary>
/// Opens a customer selector for this form and sets it as the actual customer of the scope.
/// </summary>
private void selectCustomerButton_Click(object sender, System.EventArgs e)
{
using(CustomerSelector selector = new CustomerSelector())
{
var dialogResult = selector.ShowDialog(this);
if(dialogResult == DialogResult.Cancel)
{
return;
}
SetCustomerAsCurrent(selector.SelectedCustomer);
}
}
private void saveCustomerButton_Click(object sender, System.EventArgs e)
{
// save the changes to the persistent storage.
using(var adapter = new DataAccessAdapter())
{
bool succeeded = _customerScope.CommitChanges(adapter);
if(succeeded)
{
MessageBox.Show("Save was succesful!", "Save result", MessageBoxButtons.OK,
MessageBoxIcon.Information);
_saveCustomerButton.Enabled = false;
}
else
{
MessageBox.Show(this, "Save was NOT succesful!", "Save result", MessageBoxButtons.OK,
MessageBoxIcon.Information);
}
}
}
private void _customerScope_ScopedDataChanged(object sender, EventArgs e)
{
_saveCustomerButton.Enabled = true;
}
}