I am pleased to announce the release of Trackable Entities version 1.0 – with support for both WCF and ASP.NET Web API with Visual Studio 2012 and 2013!
The idea behind this tool is simple: enable you to build n-tier applications quickly and easily, with entities that are change-tracked on the client in a persistence-ignorant fashion, then sent to a web service where they can be updated in a batch operation within a single transaction. The classic example is an Order with OrderDetails. The parent order may or may not have any changes, but the details could include new items, removed items and modified items. These should be persisted in an atomic fashion within the same transaction. If saving one of the items fails (for example because of invalid data), then all the updates should be rolled back.
Let’s start off with getting one thing straight: these have nothing to do with the now defunct Self-Tracking Entities that were once part of the Entity Framework. The goal of that project was noble, but the implementation was flawed, resulting in bloated code with rather tight coupling to the framework, similar to when ADO.NET DataSets were used to send entities across service boundaries. The problem is that, because STE’s were implemented poorly, the very idea of client-side change-tracking and batch updates fell out of favor. To add insult to injury, the EF team decided to deprecate STE’s instead of redesigning them.
This is where my solution to the problem comes in: Trackable Entities. Contrary to STE’s, Trackable Entities are completely platform neutral and carry very little overhead: just an enum property for entity state, and optionally a list of properties that have been modified (which is needed for partial updates to the database). Another difference is that instead of inserting a change-tracker into each entity (like STE’s did), Trackable Entities sports a separate ChangeTrackingCollection<T>, which is responsible for setting entity state transparently, as items are added, removed or modified. And it has a GetChanges method, so that only entities that have changes are sent to the server, instead of all the entities (like STE’s did).
But if Microsoft dropped support for Self-Tracking Entities, doesn’t that mean change-tracking and batch updates for entities is generally a bad idea? Well, tell that to the OData team, which included these features in Data Services. That product has a client-side change tracker which adds metadata to requests for batch updates.
So why not just use OData? The purpose of OData is essentially to expose an entity data model as a REST service. But you end up basically exposing your database to ad hoc queries, which may not be what you want. My approach instead builds on the service model of ASP.NET Web API, with a T4 template for generating controller actions for data persistence. This give you a great deal more control.
The core functionality of Trackable Entities is deployed as a set of NuGet packages, but the killer feature is the productivity tools provided by the project templates installed by a VSIX Visual Studio extension. There are two child project templates: Client.Entities, a portable class library compatible with .NET 4, SL5, WP8 and Win8. It includes a custom T4 template, which is executed by the Entity Framework Power Tools when reverse engineering database tables into Code First entities. This template generates classes that implement INotifyPropertyChanged and ITrackable, so that the ChangeTrackingCollection can do its magic. Similarly, there is a Service.Entities project template includes a T4 template which the EF Power Tools uses to generate server-side entities.
While those project templates are nice, they aren’t the best part. There are also multi-project templates for creating turn-key n-tier applications based on either WCF or ASP.NET Web API. When you create a project based on one of these two templates, you get custom T4 templates for generating services with CRUD operations. For Web API, you add the same template as you would for a Controller that uses Entity Framework.
The T4 template, however, generates code that is much better than what the default template generates. For example, POST (insert) and PUT (update) methods each return an updated entity that includes database-generated values (for identity or concurrency). Also, my T4 template inserts Include operators for bringing back related entities (for example, Product would include the related Category entity). (You need to add Includes for child entities yourself.)
Here’s an example of the Web API Controller generated for the Product entity.
public class ProductController : ApiController { private readonly NorthwindSlimContext _dbContext = new NorthwindSlimContext(); // GET api/Product [ResponseType(typeof(IEnumerable<Product>))] public async Task<IHttpActionResult> GetProducts() { IEnumerable<Product> products = await _dbContext.Products .Include(p => p.Category) .ToListAsync(); return Ok(products); } // GET api/Product/5 [ResponseType(typeof(Product))] public async Task<IHttpActionResult> GetProduct(int id) { Product product = await _dbContext.Products .Include(p => p.Category) .SingleOrDefaultAsync(p => p.ProductId == id); if (product == null) { return NotFound(); } return Ok(product); } // PUT api/Product [ResponseType(typeof(Product))] public async Task<IHttpActionResult> PutProduct(Product product) { if (!ModelState.IsValid) { return BadRequest(ModelState); } try { // Update object graph entity state _dbContext.ApplyChanges(product); await _dbContext.SaveChangesAsync(); return Ok(product); } catch (DbUpdateConcurrencyException) { if (!_dbContext.Products.Any(p => p.ProductId == product.ProductId)) { return NotFound(); } throw; } } // POST api/Product [ResponseType(typeof(Product))] public async Task<IHttpActionResult> PostProduct(Product product) { if (!ModelState.IsValid) { return BadRequest(ModelState); } _dbContext.Products.Add(product); await _dbContext.SaveChangesAsync(); var ctx = ((IObjectContextAdapter) _dbContext).ObjectContext; ctx.LoadProperty(product, p => p.Category); return CreatedAtRoute("DefaultApi", new { id = product.ProductId }, product); } // DELETE api/Product/5 public async Task<IHttpActionResult> DeleteProduct(int id) { Product product = await _dbContext.Products .Include(p => p.Category) .SingleOrDefaultAsync(p => p.ProductId == id); if (product == null) { return NotFound(); } _dbContext.Products.Attach(product); _dbContext.Products.Remove(product); try { await _dbContext.SaveChangesAsync(); return Ok(); } catch (DbUpdateConcurrencyException) { if (!_dbContext.Products.Any(p => p.ProductId == product.ProductId)) { return NotFound(); } throw; } } protected override void Dispose(bool disposing) { if (disposing) { _dbContext.Dispose(); } base.Dispose(disposing); } }
While Web API integration is a sweet spot of the Trackable Entities Visual Studio extension, I’ve also included support for Windows Communication Foundation. First, entities are decorated with [DataContract(IsReference = true)]. Second, there is an item template called TrackableWcfServiceType, which inserts a WCF service contract interface and implementation for async CRUD operations. Here is the code generated for the Product service.
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)] public class ProductService : IProductService, IDisposable { private readonly NorthwindSlimContext _dbContext; public ProductService() { _dbContext = new NorthwindSlimContext(); } public async Task<IEnumerable<Product>> GetProducts() { IEnumerable<Product> entities = await _dbContext.Products .ToListAsync(); return entities; } public async Task<Product> GetProduct(int id) { Product entity = await _dbContext.Products .SingleOrDefaultAsync(x => x.ProductId == id); return entity; } public async Task<Product> UpdateProduct(Product entity) { try { // Update object graph entity state _dbContext.ApplyChanges(entity); await _dbContext.SaveChangesAsync(); return entity; } catch (DbUpdateConcurrencyException updateEx) { throw new FaultException(updateEx.Message); } } public async Task<Product> CreateProduct(Product entity) { _dbContext.Products.Add(entity); await _dbContext.SaveChangesAsync(); return entity; } public async Task<bool> DeleteProduct(int id) { Product entity = await _dbContext.Products .SingleOrDefaultAsync(x => x.ProductId == id); if (entity == null) return false; try { _dbContext.Products.Attach(entity); _dbContext.Products.Remove(entity); await _dbContext.SaveChangesAsync(); return true; } catch (DbUpdateConcurrencyException updateEx) { throw new FaultException(updateEx.Message); } } public void Dispose() { var dispose = _dbContext as IDisposable; if (dispose != null) { _dbContext.Dispose(); } } }
The nice thing about native clients, be they traditional desktop applications or handheld devices, is that they deal with data in a disconnected fashion, making batch updates a necessity. That means it would be up to you to figure out which entities are inserted, updated or deleted, then send them separately to the service for persistence. Trackable Entities gives you client-side change-tracking and an elegant model for server-side persistence, without the extra baggage of a framework like OData. Plus you get a very nice set of templates for generating server-side persistence code. Enjoy.
Nice work!
Hi Tony!
Firstable, thanks for this framework, we’ve been using it in our project pretty much since you released it and loving it 🙂
I’d like to ask your opinion on a follow-up issue. It may be I’m not fully understanding how I should be doing things, but I sort of “miss” a helper method to fully reset the status of my change tracking collections and entities within them. I mean, once I’ve sent a request to the server to persist all changes made during a specific client session, and that returns successfully, I’d want to revert the change tracking framework to a fresh state, where all entities are marked as Unchanged and all internal lists of deleted items are flushed, etc. Does this seem like a meaningful question or, as mentioned, there’s something I’m missing?
I’ve currently implemented my own method to do this refreshing, which was mostly directly using some of the extension methods already provided by the framework (SetState, SetModifiedProperties), but I was mainly missing a method to recursively clear the list of deleted entities so had to implement that one. If all this makes sense, I’ll be happy to share it obviously 🙂
Cheers!
Juan
Hello Juan,
Great to hear you’re using Trackable Entities! And really great to get your question and suggestion to add something like an AcceptChanges method. That’s something that definitely needs to be added. I think I had it in a prior incarnation and neglected to bring it forward. However, what we’ll need to do is also bring in database-generated values, such as identity and concurrency properties, from the entity that is returned by the update and create operations.
Now as to the question of how we insert this code into the project. 🙂 The source code in CodePlex uses Git. So the best practice would be for you to Fork the code, clone it to get a local copy, then create a branch to work with. After making your updates to the code, you can commit them locally and then make a “pull request.” That will notify me of the proposed changes, which I can then review, and possibly alter, before merging them into the main branch.
If the Git approach is too much bother for you, just let me know and I’ll do it the old fashioned way. But it would be cool to leverage Git. 🙂
To get started, you need to get Git, and you can do that by installing two programs: Git for Windows, and TortoiseGit. Then follow these instructions: http://codeplex.codeplex.com/wikipage?title=Forks. Also take a look at this blog on creating a local branch for your changes after cloning your fork: http://lostechies.com/jimmybogard/2012/04/02/working-with-forks-on-github-or-codeplex/.
If you’re used to TFS or Subversion, getting your head around Git may take some time but is well worth the effort. Here is a good guide on getting started with Git for Windows: http://nathanj.github.io/gitguide/tour.html. And here is a short video: http://www.youtube.com/watch?v=pp2S2lHjzZI.
Cheers,
Tony
PS. Here is the work item I added to the CodePlex site: https://trackable.codeplex.com/workitem/6.
Thanks for both the answers, Tony!
Regarding refreshing the client entities on AcceptChanges with what’s returned from the server, spot on! That’s something we also need to work on in our project… and not because we actually need to synchronize anything from the database (at this point, we don’t), but instead because we’re nevertheless returning a full model of updated entities to the client even when there’s no use of it =:-P Anyhow, it’s identified and we should change that soon.
So I’ll see if I can have my company approve installing Git on my machine, unfortunately these kinds of things are somewhat of a hassle here 😦 However going further than refreshing the client change tracking status, and bringing in changes from the database, seem like a bit of a complex feat to do in a generic way: I guess you’ll have to use a similar approach of generically iterating through the entities returned, and for each modified property replace the value in the client entity… which you’ll probably need some sort of a key to locate… and this is where I’m starting to get a bit lost about how all this could be done generically.
Anyhow, I’ll give it a thought when I get through a couple of hurdles in our project.
Cheers and thanks again!
Juan
No worries re Git – just email me what you’ve got and I’ll take it from there: tony@tonysneed.com. (Be sure to reply to my spam filter challenge, and I’ll get it.)
Here’s how I think we’d need to go about this issue. First, we’d call AcceptChanges on the server side in order to reset the tracking state on each entity. Deletes would already then be purged. Then on the client we would simply re-insert unchanged items back into the object graph on the returned entity. For that we would need something like an AddUnchanged method.
Hi Tony.
Sent you an email yesterday but never got the spam filter challenge you mentioned… which makes me wonder if you ever got the email. Delete this reply if you did 🙂
Didn’t get the email (sometimes my spam filters arbitrarily delete emails). But I did fix the problem in v1.0.1, which I just released: https://trackable.codeplex.com/releases/view/114810.
If you need to reach me in the future, the best I think is to go to my CodePlex profile and click the Contact link: http://www.codeplex.com/site/users/view/tonysneed.
Sorry! Forgot to ask regarding the version 1.0 release: where can I find some sort of release notes on changes from the previous release? Can’t seem to find any on the codeplex site, and browsing through the code I can’t seem to find much changed from the previous version (?).
Thanks!
Juan
Good point! I don’t think I listed the changes – I’ll edit the release notes on the CodePlex download page to include them. Basically, there were not many changes in the class libraries. The main one is that in ModelBase I took out the ITrackable implementation and placed it in the entity classes instead, leaving only the INotifyPropertyChanged implementation in ModelBase. The reason for this has to do with playing nice with the DataContract serializer when I added support for WCF.
The rest of the differences has to do with the tooling support for VS 2012 and 2013. Mainly, I had to split out the EF-related class libraries into EF5 and EF6 versions. That is because VS 2012’s scaffolding for EF does not support v6.
Cheers,
Tony
Here is the updated Release Notes with the changes from Beta 1:
https://trackable.codeplex.com/releases/view/114408
Hi again, Tony!
I’ve made an improvement in our Trackable Entities framework which thought I’d share with you in case you deem could be useful to others. The issue was that we needed to somehow decorate certain properties in our entities as non-trackable, so that EF did not attempt to update them because they were actually extended properties of the entities outside EF.
So, we’ve created a custom attribute to decorate NonTrackable properties of classes:
public class NonTrackable : Attribute {}
… and we avoid tracking them with this new code in OnPropertyChanged of the ChangeTrackingCollection:
private void OnPropertyChanged(object sender, PropertyChangedEventArgs e)
{
if (!Tracking) return;
// If the modified property is marked as NonTrackable, ignore it
var property = sender.GetType().GetProperty(e.PropertyName);
if (Attribute.GetCustomAttributes(property, typeof(NonTrackable)).Any()) return;
So, as mentioned, thought I’d share it in case you deem it could be a “permanent” addition to the framework.
Cheers!
Juan
Hello Juan!
Very good suggestion. I have just a couple questions. First, do your extended properties fire the PropertyChanged event for INotifyPropertyChanged? If they do not fire PropertyChanged, then they will not be tracked.
I would assume, however, the possibility of properties that *do* fire PropertyChanged, for data binding purposes, but which should not be change-tracked. In that case, what I would do would be to add an ExcludedProperties property, which allows for a list of properties that should be excluded from change-tracking. That way, the use of attributes can be avoided. I don’t have anything against attributes, per se, but it’s usually better to avoid them in order for classes to remain ignorant of persistence concerns.
One other suggestion. It would be great if you could add an issue for this on the CodePlex project site: https://trackable.codeplex.com/workitem/list/basic. That way, I can track it better. Thanks!
Cheers,
Tony
FYI, This issue has been resolved with v2 of Trackable Entities, which has an ExcludedProperties property.
I am just curious I see this with a Web Application and Console Client ; how about the Service being hosted in a windows Service and Windows forms being a client – using NetTCPbinding ; possibly even a browser as a client (webhttpbinding) ? Is this a possibility here or am I trying to make this code do what it aint supposed to.
Trackable Entities is designed to be independent of both the web service framework (WCF, Web API, MVC Core, etc) and the service host (IIS, Windows Service, etc). If you’d like to see an example of a Web API service with a Console client, you can download TrackableEntities-Samples.zip from the 2.5.2 release. There you’ll also see a sample which uses WCF, and you can add a Windows Service host if you wish and configure endpoints for both netTcpBinding and webHttpBinding. If you have further questions, please feel free to submit an issue on the GitHub repo.