Hi,

I have read various posts (and opinions) on caching throughout the forum but none specifically relate to the scenario I have mind.

Basically I have written generic framework library and GUI controls for fast development of web sites. For pages that display read-only data the required fields are requested directly in the HTML in the control, for example

<GenericControls:Repeater Entity="OrderEntity" .......................> <Fields> <GenericControls: PersistedField Field="OrderName"/> <GenericControls: PersistedField Field="OrderDate"/> <GenericControls: PersistedRelatedField Field="Name" Entity="CustomerEntity" ParentEntity="OrderEntity"/> </Fields> <ItemTemplate> .................................... </ItemTemplate>

The code behind this will build a dynamic list and then bind the control to the results, it will relate all the entities properly etc so the dynamic list is correct. Once the framework has built this dynamic list and manages to get it from the database it caches the datatable in the cache. The cache time is customisable (a property on the web control) but will only remain in the cache for a short time (20 seconds maybe). This is acceptable in the some scenarios as a wait of 20 seconds to see new orders is OK.

The problem comes when one of the entities that is present in data table has changed in the database. Say for example the name of one of the customers in the list changes? How do people handle this?

Say for example one customer edits their details and modifies one customer entity, this entity is saved into the database but the datatable for the repeater above remains in the cache.

The solution I currently have for this is a semi intelligent cache, so before a data table is cached (as in the repeater example above) a dependency list is built up relating every entity it depends on to its cache key, for example; entries in the form (EntityPrimaryKey) dependency: OrderEntity11 cacheKey: -7778667373 dependency: OrderEntity12 cacheKey: -7778667373

where the cache key -7778667373 is the cache key for the data table underlying the repeater above.

when the framework saves an entity the cache is flushed of any data that has changed. For example if OrderEntity with primary key of 11 changed then the framework would get all cache keys that are related to that entity and flush the cache of the data related to that key. So changing OrderEntity11 or OrderEntity12 would flush the cache of the data table backing the repeater so the next time it was fetched it would have correct data.

This is working but it means if a field was changed that the repeater (data table underlying that data to be displayed) was not interested in the cache would still be flushed and the data would have to be obtained again.

Is there another technique I should be using? The controls that allow users to edit an entity use the FetchEntity method, maybe the control that displays the data and the control that allows users to edit data should both take the entity from memory? Or maybe when an entity is saved is should update all related data tables after saving?

Any feedback or experiences would be appreciated.

Thanks

Ian

A fun challenge indeed. While I do not claim my solution to be the be-all-end-all, it fits our needs.

1) I was able to absolutely say all db updates (CUD) would go through 1 central service. 2) Therefore this remoting service handles all data requests. 3) When an entity is updated, if it is data that is cached, the SQL update and cache updates are performed at the same time. 4) Each entity is wrapped in a middle entity that handles this work as well as provides thread safety. That object is marshalled by ref to the client. This was a custom LPT template.

Simple as that. Of course, it took time to perfect. However, we notice roughly 100,000% (1000 times) faster data access on reads.

In some cases I store entire tables. I then use a hashed merge engine to perform inner joins and subset operations on the cached data.

As an aside, the security model of the main application is extremely granular, requires real-time checks, and can require dozens of checks per request for data. This was the initial impetus for going down this road.