I'm trying to re-fetch an entitycollection that has several Prefetch Paths associated with it in order to pick up new entities that were added to the database after the initial fetch. I see the new entities coming back from the SQL prefetch queries but it appears that they do not get added to their corresponding entitycollections. Is this the expected behavior of the current release?

Also, I found this comment in http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=1871

What will be added in the (near) future is that you can use prefetch paths also on an existing loaded collection or object. This is not entirely the same as lazy loading, but makes working with existing objects in adapter more convenient. (and will probably solve your issue)

What's the status of this new behavior?

Btw, I'm wanting to refetch an existing collection and it's prefetches b/c I'm using it to store in-memory changes before I save the new data.

For example, let's say I have an Order object with a couple prefetches to get the Order detail, including its related Customer. Then assume a new Customer's (1:M) Address Entity was added to the DB. I would like to refetch (refresh) the entire Order object and it's related enties so that I can pick up the new Address with loosing any in-memory changes made to the existing Customer.

Any ideas on the subject would be greatly appreciated. (Also, Adapter with PrefetchPaths makes LLBLGen 100x better.)

acradyn wrote:

I'm trying to re-fetch an entitycollection that has several Prefetch Paths associated with it in order to pick up new entities that were added to the database after the initial fetch. I see the new entities coming back from the SQL prefetch queries but it appears that they do not get added to their corresponding entitycollections. Is this the expected behavior of the current release?

Prefetch paths work as follows: The root entity/entities were fetched with a query A. These root entities are stored in a hashtable. The childs are fetched using a subquery which uses A. The childs are then merged with the root entities based on the relation between the entity types and the hashtable. So to be able to merge a child with its parent, the parent has to be in the root hashtable.

This isn't the case with a refetch. So a refetch should refetch everything, root and childs.

Also, I found this comment in http://www.llblgen.com/tinyforum/Messages.aspx?ThreadID=1871

What will be added in the (near) future is that you can use prefetch paths also on an existing loaded collection or object. This is not entirely the same as lazy loading, but makes working with existing objects in adapter more convenient. (and will probably solve your issue)

What's the status of this new behavior?

It's still on the plate for the current upgrade in development. All designer additions are more or less done, I'm now busy with taskperformer/code generator additions and I'll then move on to runtime lib and templates.

What I want to add is that you have first fetched a set of customers for example, and now want to fetch their orders and addresses and order details all in 1 go, but you don't want to refetch the customers again so you should be able to pass the existing parents to the prefetcher. There are a few problems though: the query used for fetching the existing parents should be preserved by the caller (or formulated again). All code is already there, I just have to write some interface methods so the developer can access the code properly

Btw, I'm wanting to refetch an existing collection and it's prefetches b/c I'm using it to store in-memory changes before I save the new data.

For example, let's say I have an Order object with a couple prefetches to get the Order detail, including its related Customer. Then assume a new Customer's (1:M) Address Entity was added to the DB. I would like to refetch (refresh) the entire Order object and it's related enties so that I can pick up the new Address with loosing any in-memory changes made to the existing Customer.

Well, you'll always have to specify a special path with just the address node, as specifying the complete path will reload everything you specify in the path of course.

Any ideas on the subject would be greatly appreciated. (Also, Adapter with PrefetchPaths makes LLBLGen 100x better.)

Prefetch paths were indeed a core requirement for adapter. It was a struggle to get them into the code but it was all worth it

Frans,

Thanks for the response. Although I apologize I was either unclear or couldn't understand your response. Let me try to expain my situation in more detail.

I have a custom method called 'FillCustomerData(CustomerEntity customer)' which fetches a single Customer and several Fetch Paths of data including the Customer's Orders, Customer's Addresses and each Order's Products.

User 'Bill', calls FillCustomerData and now he has this Customer Entity data in memory. Then he adds a new Order to the Customer's Orders collection, so this new Order Entity is new and not yet saved to the database.

At the same time, user 'Jenny', on a different PC, adds a new Address associated to that Customer's entity and saves it to the database. I would like Bill's app to be able to call 'FillCustomerData' again (the re-fetch), passing in my orginally filled Customer entity with hopes to pick up the new Customer's Address without loosing the new, in-memory Order that Bill has not saved yet.

The reason I don't want to save the new Order first and then just refresh the entire Customer Entity is b/c I want to associate the new Order with the new Address that was created. In other words, I would like to pick up the new Address and associate it with the new Order before I save the Order for the first time.

I am doing exactly this in my code and I see the SQL returning the new Address but the Customer's Address collection does not get loaded with it.

I guess my technical question is: When LLBLGen loads the entities from the database resultset and the Entity already exists in the collection, do you still try to fill any PreFetch Paths on that entity? It appears that it does not, but hopefully I'm wrong. I took at a look at the source code for a while and it wasn't clear to me either way.

Thanks for the quick response, as always.

-Jeff

Could you please elaborate a bit how your FillCustomerData looks like? What I'm interested in is that the FetchEntity() code which accepts a prefetch path (the likely routine you'll use when fetching a customer with related data) is fetching the customer, putting it into an EntityCollection, so that's the root entity and the related entities will be merged with that root entity. I don't know which routine you're using to pass in that existing entity for the merge...

The childs are then merged

Children

JimFoye wrote:

The childs are then merged

Children

LOL

My example is a little simpler than my actual code. I replaced my actual code and my domain entities with Customer, etc to make life easier, but here goes...

This function returns the Addresses that are associated with the Customer's Company's Offices. I want to be able to call this a second time to fill any new Addresses that were entered in the DB without loosing any additions to the EntityCollections in memory.

public voidl FillCustomerData(CustomerEntity customer){

   IPrefetchPath2 customerFetch = new PrefetchPath2((int)EntityType.CustomerEntity);    

   // The Customer's Company
   IPrefetchPath2 companyFetch =  customerFetch.Add(CustomerEntity.PrefetchPathCompany).SubPath;

   // The Company's Offices
   IPrefetchPath2 officeFetch =  companyFetch.Add(CompanyEntity.PrefetchPathOffice).SubPath;

   // The Offices' Addresses 
   IPrefetchPath2 addressFetch =  officeFetch.Add(OfficeEntity.PrefetchPathAddress).SubPath;  
        
   FetchEntity(customer, customerFetch);

}

I hope that helps. -Jeff

Your routine should re-fetch the customer, its offices and every address of that office in the db and add them to the collections, skipping existing objects (as these are already there, so no duplicates are added).

I think that's the problem. Say I have office 'NewYork HQ' already loaded into the Offices collection. Now a new address is added to that office and I want to fetch that too. In the offices collection an object with the data of NewYork HQ is already loaded. The prefetch path loads a new entity object with the same data.

The merger loads the address and tries to find back the office object, which is the parent of the address object. It does find it back, the newyork HQ object that was loaded too in the prefetch path fetch. But that's not the object already in the collection. So the address object gets added to the wrong office object. Prefetcher is done, objects are gone, and no address is fetched into the customer graph.

What should be done is that the prefetch paths routine checks if the object is already there, and if so, it should throw away the loaded parent and replace it with the already existing parent. This way you can re-fetch child objects into the parent's collections without losing existing parent objects, as that would be a bad thing to do because the parent can hold other data not refreshed in the requery.

I'll have to look into this a bit more. It would have been solved by an object cache at the adapter's side, but that's not implemented yet. As customer is refetched, but the object isn't, it might be easy to do, but I'm not entirely sure yet, I've to do some tests.

(edit). The merger itself doesn't know the current collection to which the child object is added. As an add of an existing object results in a rejection of that object (as it already exists in the collection), the merger works with objects which aren't actually in the collection. So it actually goes wrong with the office objects, these aren't added to the collection. At the moment I can't solve this easily for you, as the merge routine doesn't actually know what the collection is to which the child is added, it calls a property set routine for that which is actually in the entity's generated code as only at that spot it is known to which collection a field is mapped on (as the merger is in generic code).

As a somewhat lame workaround, you should clear the offices collection of the customer first, then call the refetch again. This will make new office objects being added to the collection and these are the parents for the addresses, the addresses are added as well. As all the data is re-read anyway, you won't lose any performance.

Frans,

Thanks for looking into this. I suspected this to be the case. I assume this functionality will not be availble for a while, if at all. The work around you suggested would be ok if I knew there was not any existing changes to the Office data. I guess I could manage that but in general, it would be nice if I didn't have to maintain state at such a granular level.

Do you see this as something that might change or might be an enhancement? I'm asking b/c my programming paradigm right now is leaning more and more on using LLBLGen objects to maintain in memory changes, and I'll have to make some type of hybrid solution if I can't refresh new child entities.

• Jeff

It will change, I don't know when. The current upgrade feature list has a 'refetch graph on existing entity(/ies)' feature planned, so I might be forced to fix it this week in the new code. I filed it as a bug in the bugtracker and I've planned to fix all bugs before the upgrade goes RTM so if it is not a breaking change, I'll probably fix it within a month time.\

However you should be aware that the in-memory entities can get a life of their own, and it is often best to rely on database instances. I.e.: keep copies around while you're editing, but once editing is done, flush what's in memory and reload when you need to.

Great, thanks again for the incredible service along with a kickass product.

btw, is there a future enhancement list anywhere for the public to see? I'm always curious as to what's coming...

-Jeff

acradyn wrote:

Great, thanks again for the incredible service along with a kickass product.

btw, is there a future enhancement list anywhere for the public to see? I'm always curious as to what's coming...

-Jeff

This time I decided not to do that, as I wanted the freedom to move features to the april/may upgrade if I run out of time . Currently implemented for the upcoming upgrade: (I'm at the task performers todo's at the moment, so if no changes are reported about runtime libs and templates, that's because I haven't done these yet. Most of the work was done on refactoring the underlying code though)

DRIVERS:

• CHANGED: MS Access driver/Oracle DataDirect driver now use the new ConnectionElement enum values.
• CHANGED: SqlServer / Oracle drivers now shows a stored procedure selector form if ManualSelectSProcsFromSchema is set to true.
• CHANGED: SqlServer driver now only determines the resultsets of stored procedures if SqlServerAutoDetermineSProcType is set to true, otherwise it relies on the values specified in the stored procedure selector.

CORE:

• CHANGED: User preferences now have settings for strip patterns for entities, typedviews, procs, entity fields and typed view fields
• CHANGED: Entity, view and proc retrieval now strips prefixes and suffixes and users can specify more than one prefix/suffix.
• CHANGED: Moved project properties to a separate object.
• CHANGED: CorrectNameCasing is now split into 2 options: RemoveUnderscoresFromElementName and MakeElementNamePascalCasing.
• CHANGED: UtilizingPropertyName construction is now delegated to a GeneralUtils routine which uses the two new patterns FieldMappedOnManyToManyPattern and FieldMappedOnOneManyToOnePattern to construct the name.
• CHANGED: Refresher: catalog/schema names have to be found in the target database. If not, the catalog can't be refreshed.

• CHANGED: Refresher: now takes advantage of the new log framework.

• ADDED: Multiple catalog support (sqlserver) / multiple schema support (Oracle)

• ADDED: User preferences and project properties FieldMappedOnManyToManyPattern and FieldMappedOnOneManyToOnePattern patterns for specifying the format of the field mapped onto a relation. Several pattern elements are available to specify a format.
• ADDED: More values to ConnectionElement enum definition.
• ADDED: SqlServerAutoDetermineSProcType and ManualSelectSProcsFromSchema options for allowing the user to better specify which elements to retrieve.
• ADDED: Refresher options: CreateBackupBeforeRefresh, VerboseRefresh, SyncMappedElementNamesAfterRefresh, UpdateCustomPropertiesAfterRefresh, ShowReportAfterRefresh, AddNewElementsAfterRefresh, AddNewViewsAsEntitiesAfterRefresh.
• ADDED: Catalog refresh log is more detailed and contains information about actions which weren't logged before.
• ADDED: root folder tag definition for plugins to the application config: pluginsRootFolder
• ADDED: additional folder tag definition for template sets: additionalTemplateSetsFolder. This folder can be used for generic template sets not tied to a driver, like .lpt templates for BL code. This folder is also relative to the application root.
• ADDED: Plugin system, plugin base classes, plugin control interface
• ADDED: Option to ignore system elements when retrieving entities/typed views from the catalog(s).
• ADDED: Entities can now be mapped on views as well, which allows the user to create relations between an entity mapped on a table and an entity mapped on a view, add an entity mapped on a view to a typed list etc.

• ADDED: Entities can now have fields mapped on fields in a related entity, if that entity is related using an 1:1 or m:1 relation.

• REMOVED: CollectionNamePostfix property of project/user preferences has been removed.

TASK PERFORMERS

• CHANGED: Typed list targeting routines should now deal with TypedListRelations in typedList.RelationsInTypedList rather than EntityRelation objects.

• CHANGED: TDL Statement <[ TypeOfField ]> now returns the type of the field mapped on a related field if it's used in a Foreach RelatedEntityField loop.

• ADDED: Ability to log into a general Tasks log using LogNodes. This log is shown later on if the user opted for that option.

• ADDED: Ability to clean up the vs.net project file before new file references are added. This is controlled by a setting in the project properties and only effective if the file has to be altered when existent.
• ADDED: Ability to specify if the time should be generated into the TDL template output. This is controlled by a setting in the project properties.
• ADDED: Ability to specify regions in the templates which are preserved between code generation cycles. For TDL, use <[ UserCodeRegion ]> .. and for LPT templates use DotNetTemplateEngine.GetUserCodeRegion(string name, string commentToken);
• ADDED: TDL Interpreter: new statement scope parameter: _currentTypedListRelation, set in loops over relations in typed lists (Foreach EntityRelation).
• ADDED: TDL Interpreter: new statement scope parameter: _currentFieldOnRelatedField, set in Foreach RelatedEntityField loops
• ADDED: TDL statements: <[ UserCodeRegion "name" ]> <[ EndUserCodeRegion ]>, <[ JoinHint ]>, <[ StartEntityAlias ]>, <[ EndEntityAlias ]>, <[ AggregateFunction ]>, <[ EntityAlias ]>, <[ FieldCaption ]>, <[ Foreach RelatedEntityField separator ]> <[NextForeach]>, <[ If not IsReadOnly ]> <[ EndIf ]>, <[ MappedFieldNameRelatedField ]>, <[ RelatedEntityFieldName ]>

GUI:

• FIXED: Custom properties of Typed Lists weren't saved properly

• FIXED: Slow rendering of added elements when there were a lot of elements already in the project.

• CHANGED: Database Drivers are now sorted on name in the driver drop-down box in create a new project wizard.

• CHANGED: Objects in the object selector (to select participating objects) are now sorted alphabetically.
• CHANGED: Project properties are now edited in a decent property grid.
• CHANGED: Add Entity/TypedView/StoredProcCall screens now use one unified form. This form now has better tools for the developer to select which element to add and it now uses the Janus GridEX v2 grid.
• CHANGED: The form for adding new elements (entity/proc/typed view) is no longer pre-selecting all shown elements.
• CHANGED: Custom Properties now use the Janus grid and don't accept CR/LF in the editor anymore.

• CHANGED: New Typed List editor

• ADDED: Stored procedure selector form to be used by various drivers.

• ADDED: Catalog rename functionality in the catalog explorer, to overcome name changes for catalogs.
• ADDED: Catalog / schema removal functionality in the catalog explorer.
• ADDED: Unattended catalog refresh (one or more catalogs can be refreshed unattended). Unattended catalog refresh (or schema refresh, on oracle) will not obey ManualSelectSProcsFromSchema, but will retrieve all procs.
• ADDED: Better log viewer and better refresher log display with export functionality log to XML or RTF
• ADDED: Option added to ask a confirmation from the user when the gui is closed.
• ADDED: Option added to specify to open an element when doubleclicking it in the project explorer. Default behavior is expanding the node.
• ADDED: Option added to ask for confirmation from the user when deleting an element in the project explorer
• ADDED: DEL key is now shortcut for deleting an element from the project in the project explorer.
• ADDED: Forecolor and Backcolor of nodes of changed elements (also new elements) can be set by the user.
• ADDED: Option to switch off the hints in various editor screens to save space.
• ADDED: Join hint specification to relations in typed list
• ADDED: Aggregate function setting for fields in typed list
• ADDED: Multi-join of same entity in Typed list
• ADDED: Caption for typed list/typed view fields.
• ADDED: Startup position of gui is preserved.
• ADDED: Participating objects can now be saved as a group.
• ADDED: In object selector, related entities can now be checked automatically
• ADDED: In object selector, when a typed list is checked, all entities in that typed list are checked as well.
• ADDED: Plugin execution system in the GUI, plugin starter form.
• ADDED: Plugin for adding custom properties to a wide range of objects at once.
• ADDED: Plugin related options in user preferences
• ADDED: Entities mapping on views
• ADDED: PK Fields for entities can now be set/unset if the map target doesn't have any PK fields (table with no PK fields, or a view).
• ADDED: Entities can now have fields mapped on fields in a related entity, in the designer, if that entity is related using an 1:1 or m:1 relation. The EntityEditor has a new tab for this, plus the ProjectExplorer has a new node for this for each entity.
• ADDED: Ability to show a log of all task performers ran during the code generation process and export the log to XML or RTF
• ADDED: Ability to specify an option to clean up vs.net project files before adding new file references.
• ADDED: Ability to specify an option to generate the time/date into TDL template output.

ADDED: Multiple catalog support (sqlserver) Huge!!! I've been wanting this one. Thx!!

What is: ADDED: Multiple catalog support (sqlserver) / multiple schema support (Oracle)

I am wondering in what kind of scenarios this is useful? Thanks.

netLearner wrote:

What is: ADDED: Multiple catalog support (sqlserver) / multiple schema support (Oracle) I am wondering in what kind of scenarios this is useful?

Well, for example scenario's where an application wants to use data from another catalog, like a customer table, without requiring you to add the new application's data tables to the already existing catalog with teh customer table.

A real life example is our website: we have visit tracking and demo download tracking (simple session start referrer loggers) These are different catalogs, as the site is made with our CMS and uses the core system (VB6 COM + stored procs) as that was already there, and the demo download logger was written later on.

To see how a person who downloaded a demo can onto our site, I have to combine the information, but they're in 2 different catalogs. With this feature I can add the site stats entity to the demo logger project, add a custom relation and create a typed list or whatever to get this information out of the databases.

• ADDED: Entities can now have fields mapped on fields in a related entity, if that entity is related using an 1:1 or m:1 relation.

Now that's a damn handy feature. Databinding on 1:1 and m:1 relations can be a bit of a pig with some controls - have been faking lists of 1 item to keep the devexpress vertical grid happy.

Ryan