UPDATE: I created a new version of the client-side T4 template for generating trackable DTO’s. This completely decouples the client from the service by producing POCO classes based on the service metadata, instead of on the entity data model used by the service. I have also updated the client T4 template to accept a max received message size parameter.
WEBINAR: On March 30, 2010, I conducted an online seminar on Trackable DTOs. The screencast video is now available to be streamed or downloaded. The slides and code for the presentation are also available.
Download updated code for this post here.
Not long ago my friend and colleague Richard Blewett wrote a blog post on Self-Tracking Entities in EF4, in which he questioned the service-orientation of Self-Tracking Entities in EF4. While STE’s are placed in an assembly that does not reference the Entity Framework, the way in which change state is preserved in an STE is overly complex because it tries make it easier for EF to transmit those changes to the ObjectStateManager on the service side. If you look closely at the ObjectChangeTracker class that is generated for the client, you’ll see that it maintains metadata for navigation properties with items that have been added or removed, as well as original values and extended properties. Requiring a Java client to implement all that is asking an awful lot, and it couples the client too tightly to the service implementation.
About a year ago I wrote an article for MSDN Magazine on how to track change-state on the client and transmit it to a service for persistence using LINQ to SQL, Entity Framework, or some other data access stack. Each entity has an ObjectState property indicating its change state (Unchanged, Added, Modified, Deleted), which is sent to the service where it is used to perform inserts, updates and deletes against a database. The beauty of this approach is that it allows multiple changes to be sent to a service in a single round trip, where they can all be saved in a single transaction. The classic example of this is an Order with Order Details that have been added, modified or removed. An UpdateOrder method in the Data Access Layer can simply read the ObjectState property to perform corresponding inserts, updates and deletes.
Fast forward to Entity Framework 4.0 and Visual Studio 2010. This week I implemented Trackable Data Transfer Objects with EF4 using the same basic architecture that I wrote about in the article. I use two sets of T4 templates (a code-generation technology built into Visual Studio): One set is used by the Data Access Layer on the service side to generate both the ObjectContext container class and POCO classes that serve as DTO’s but have an ObjectState property. On the client side there is an assembly containing another T4 template that generates DTO’s that also have an ObjectState property. On the client-side, DTO’s have a Tracking property (to turn change-tracking on and off) and navigation properties that are of type ChangeTrackingCollection<T>. This collection is placed in a separate ClientChangeTracker assembly and marks entities as Modified when they change, or as Added or Deleted when they are added to the collection or removed from it.
The goal of this design is to keep change state as minimal as possible: a simple ObjectState enum, which is exposed as a data contract by the service and can be easily implemented by a non-.NET client. The client-side DTO’s are generated by a T4 template that consumes the service metadata, which results in a clean separation between the client and any persistence concerns (see UPDATE section above).
On the service-side there is a ServiceChangeTracker assembly that has a TrackingHelper class with an ApplyChanges methods that extends ObjectContext by walking an object graph and informing the ObjectStateManager of entity state based on the ObjectState property. When a DAL method invokes SaveChanges on the ObjectContext, inserts, updates and deletes are persisted to the database in the scope of a single transaction. The DAL can then call AcceptChanges in TrackingHelper to restore objects to an Unchanged state and return the updated object to the client with database-calculated values, such as identity and concurrency fields.
Trackable DTO’s match entities defined in the conceptual model, which allows us to leverage POCO support in EF4 to avoid creating two sets of classes and manually copying data between them. As such they represent an approach that combines the simplicity of self-tracking entities with the flexibility of DTO’s, without the extra baggage and complexity of STE’s. The use of two separate sets of T4 templates allows us to decouple client DTO’s from the service DTO’s, applying the rules of data contract versioning so that they can diverge from one another in a robust fashion.
Enjoy.
Tony Sneed's Blog 
Excellent timing 🙂
I thought about using T4 for generating proxy classes as well earlier for ADO.NET DS and LINQ to SQL; now transitioning the same for EF.
Your example with nice overview article seems to be about exactly what I was looking for.
This will be then laid on top of authorizing object level access layer for providing clean authorized dataservice 🙂
Thanks!
Hello Tony:
This is Pablo, again from DR. If you do not mind I do have a question on the Tracking of the Objects through WCF.
I did your example and by the way, Excellent video, I saw your video. After you telling me how to get the datasources I Did, but I have a problem the proxy is duplicating the classes.
In you video it shows it need a reference for the Objects type before the reference I did that, but still the proxy makes the classes with the same name. I went back to the video to see what settings you got when you created the proxy, but in the video it has been created and you only updated it.
having the reference to the types objects, can you tell me what settings do I need to create the proxy, and that the proxy does not re-create the same objects?
Thanks in Advance,
Pablo.
Tony:
Thanks, I did figure it out!
Thx.
Pablo.
Hi Tony
Could you let me know where the T4 templates can be found.
Cheers
Steve
@Steve: Yes, there are two T4 templates needed by the service which are in the NorthwindData project: Northwind-Context.tt and Northwind-Service.tt. This project also contains Northwind-Client.tt, which is linked to by the NorthwindClientObjects project.
In the spirit of service-orientation, I am using separate T4 templates for client and service. Presently, both templates are based on the entity data model (edmx), but I am working on replacing the client-side template with one that is based on the service metadata (wsdl), which is even better. Stay tuned.
Hi Tony,
I am trying to implement your code, which is awesome, BTW, thank you very much for posting it.
I am running into an issue that I’m hoping you can help with: in your code you attach each navigation property to the context recursively. But it seems EF already automatically attaches related entities. So the parent entity is already attached by the time the code tries to attach it, and it returns an error.
Am I missing something? Is there a way of disabling the behavior to better control exactly what gets attached to the context?
Thanks!
Helene
@Helene: You’re saying that attaching a child entity will automatically attach the parent? I haven’t seen that behavior. You’ll notice that the AddAttachEntities method in TrackingHelper attaches child entities before attaching the parent entity, and there’s no runtime error in the sample app.
Also notice that DataAccess.SaveOrder only calls Attach on the parent entity. If you call it more than once on different objects in the same object graph, you will get an exception.
I understood what is going on by reading your answer, thanks a lot.
It turns out I am using my own modified version of the POCO T4 template, and it still has all of Microsoft’s Fixup code in it. When the entities get deserialized on the way back down to the business layer, the fixup code is triggered and sets the navigation properties of the child entities. As a result, when I add them to the context, parent entities get added as well.
I’m not sure I even understand the reasoning behind the fixup code. So I’ll start over from your POCO template, which appears to be both simpler and better…
Thanks again.
Hi Tony,
Thanks in advance for the excellent video.
I have an interesting scenario. I am implementing your approach of Trackable DTO and followed your techinque in TrackableDTO-2 sample and created a customized client version of T4 templates for generating client POCO’s.
Everything worked so far; however, once I add a service operation contract that returns a speific POCO type (generated from server’s template)the client code generation fails with the message: “Metadata contains a reference that cannot be resolved:”, But if my operation contract returns a smaller POCO type then the client code is generated fine.
I enabled the debugger in the template and stepped through the code and sure enough mexClient.GetMetadata(); throws a WCF exception as follows:
“The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element.”}
Typically I can solve this problem by bumping up the MaxReceivedMessageSize in the config file but dont know which WCF configuration the T4 tool uses.
Also is there a way to retrieve the metadata through the service assembly itself (similar to svcutil) or from a wsdl file directly.
Thanks for fantastic job,
regards,
Mark
@Mark: I’ve found a fix to the max received message size issue, and I’ll post it shortly.
I’ll also look into retrieving metadata from the service assembly or wsdl file.
Thanks for your patience.
Tony
Hi Tony. I did manage to get the large metadata by creating the NamedPipe binding with MaxReceivedMessageSize set to a huge value and pass that binding to MetadataaExchangeClient constructor.
I still would appreciate it if you could provide a way to generate metadata from wsdl file or service contract assembly directly. That would make the build process much smoother. Thanks again. I look forward to your new post.
Mark
@Mark: You’re on the right track. MaxReceivedMessageSize is actually a property of the transport binding element, so you need to create a custom binding using that element and pass it to the MetadataaExchangeClient ctor. I’ll post a modified version of the template in the next few days.
In terms of getting metadata from a wsdl file or service contract assembly, that’s not something I plan on supporting. But feel free to take the source code and have a go at it if you wish.
Hi Tony,
thanks for this article, it really helped me understand some of the POCO /EF issues.
However, I too am getting similar results to Helene where attaching a parent object is causing an exception after attaching one of its child objects.
It appears to be caused by the fact that I am sending in the original object that I received from the server, which maintains a reference to the parent from the child.
Your example avoids this problem by creating a copy of the Order object to return (from the Order GetChanges method) which sets the navigation properties of the OrderDetails child objects to null.
Colin
@Colin: Good catch. There was an issue in some helper methods in the TrackingHelper class. I fixed the problem and uploaded a new sample project: http://tonysneed.com/elinq/download/TrackableDTO-2.zip.
I also removed the GetChanges method from the Order partial class and instead simply call GetChanges on OrderDetails. It’s important to call that method if there are details that have been removed. Those items are cached in the ChangeTrackingCollection class and included in the result of GetChanges. This also ensures that unchanged details are not sent to the service for updating, which reduces the amount of data on the wire going to the service. Note, however, that the SaveOrder method calls the helper LoadOrderProperties method to send all details back to the client. That way, the client has a fresh copy of the order, including values provided by the database, such as identity or concurrency tokens.
@Mark: The latest version of the sample project accepts a parameter for max received message size in the client’s T4 template. That will accommodate scenarios where service metadata exceeds the defatul 64K limit. The technique I used is explained further here: https://blog.tonysneed.com/2010/04/12/t4-supplement-to-add-service-reference.
Pingback: Tony and Zuzana’s World » Webinar: N-Tier Entity Framework with DTOs
Hi Tony,
many thanks for this Article!!!
Whats about Many-To-Many Relationship and the insert od remove Methods from the associations from her?
User has many Roles, Roles has many Users.
When i add an existed Role to an User, the Role is added as new Object. When i add an existed User to an existed Role, the User is added as New User.
Why?
How can i solv this?
Many Greetings,
Manuel
Hi Tony,
Just want to ask if there’s any way to add MaxNameTableCharCount in the Client template. The class cannot be generated if there are lots of Tables involved. Is this possible?
Thanks,
Philip
Hi Tony,
i have two questions.
1) I’m trying to extend the project with the methods to recieve all orders using IQueryable:
public IQueryable GetOrders(Expression<Func> predicate)
{
using (var ctx = new NorthwindEntities(Settings.Default.NorthwindConnection))
{
// Retrieve Orders and eager-load related entities
return (from o in ctx.Orders
.Include(“Customer”)
.Include(“OrderDetails.Product.Category”)
select o).Where(predicate);
}
}
But when I use this, i get an exception. Is there a way to implement such method?
2) How do I extend the project with, for example, a DataAccess for Customer. I don’t want just one big file with everything in it, but want seperate classes for customer, order, etc..
Sorry for my bad english. I hope to hear from you soon.
Thx in advance,
Jochem
Tony,
This is great stuff!
Is your downloadable demo the latest version of this framework / templates? I am interested in using it in a project…
Yes, this is the latest version. Enjoy!
Tony,
Are all of the open items in the ReadMe.txt file addressed in the latest version?
Client T4 TODO:
– Include assoc fixup code
– Perform cascading deletes
– Handle change to Tracking property
– Set foreign keys on added child entities
– Implement GetChanges method
Also, I am getting errors when I try to execute your templates in my projects:
“Could not find file ‘c:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MyModel.edmx'”
It seems to be looking for the model in the wrong directory. Any ideas? Happy to take this discussion off-line if you like.
– Alan
Would there be any changes in your code if I’m using EF 4.1 with introduction of DbContext and DbSet
Nope, those classes are there to support Code-First with EF, which doesn’t have anything to do with change-tracking.
Thanks for the great post Tony. Just curious, how to modify the client side T4 template which would create client objects from Model/database instead of service.
My implementation of trackable DTO’s has a client T4 template that generates classes based on the service metadata. That’s a conscious design decision. This relieves the client and service from needed to reference a common assembly, which decouples them and is a more ‘service-oriented’ approach. EF Self-Tracking Entities take the approach you’re asking about, which is to generate the client entity classes from the data model.
Cheers,
Tony
Hi Tony,
I appreciate the great post.
I was hoping you could provide a little background on the decision to use the navigation property name as the entityset when attaching to the context. I’m new to EF and I’m looking at how I could modify the solution to support entity per type where the navigation property name is not used as the entityset but is instead deterministically found. I thought that getting the dependent (FK) of the navigation would allow me to then determine its base type and it turn get the base type’s entityset (and do so for each child in the graph).
I updated the service template to code generate the base type inheritance with the trackable interface.
For example, I have a People entityset of type Person. When modifing an object of base type Person I would like to attach it to the correct entityset (People).
e.g. context.AttachTo(“People”, objectWithPersonBaseType);
Any pointers or gotchas would be appreciated.
Thanks.
Morgan
As an update to my post. I added the following extension method to the ServiceChangeTracker…called from AddAttachEntities and DeleteEntites using the following invocation (I’m still testing it) –
// Assume entity set name is the same as the nav property name
string navEntitySetName = context.DefaultContainerName + “.” + context.GetNavigationEntitySet(entity, navPropertyName); /*navPropertyName*/
// Get EntitySet for the specified entity’s navigation
private static string GetNavigationEntitySet(this ObjectContext context, object entity, string navPropertyName)
{
EntityType entityType = context.MetadataWorkspace
.GetCSpaceEntityType(entity.GetType());
EntityContainer entityContainer = context.MetadataWorkspace
.GetEntityContainer(context.DefaultContainerName, DataSpace.CSpace);
NavigationProperty navProp = entityType.NavigationProperties.Where(n => n.Name == navPropertyName).SingleOrDefault();
string navEndTypeName = navProp.ToEndMember.GetEntityType().Name;
string navEndRelationKeyName = navProp.RelationshipType.RelationshipEndMembers[navEndTypeName].DeclaringType.FullName;
EntitySetBase entitySetNavRelation = entityContainer
.BaseEntitySets.Where(bes => bes.ElementType.FullName == navEndRelationKeyName).SingleOrDefault();
RelationshipSet relationshipSet = null;
AssociationSet associationSet = null;
if (entityContainer.TryGetRelationshipSetByName(entitySetNavRelation.Name, true, out relationshipSet))
{
associationSet = relationshipSet as AssociationSet;
}
if (associationSet == null)
{
throw new ArgumentException(String.Format(“Unable to find a CSpace AssociationSet for type {0} given navigation name {1}”, entityType.Name, navPropertyName));
}
AssociationSetEnd associationSetEnd = associationSet.AssociationSetEnds.Where(ase => ase.Name == navEndTypeName).SingleOrDefault();
string navEndEntitySetName = associationSetEnd.EntitySet.Name;
return navEndEntitySetName;
}
First I’d like to say that I really enjoyed your point of view in the webinar on this topic. I have been struggling with this the past few weeks. And I’ve finally gotten a few good pointers to continue my journey with the Entity Framework in N-Tier architectures.
However, I’m unable to download the source code associated with this blogpost. Could you please check if the downloads are still working? I’d like to take a look at the T4 templates, but can’t find them anywhere.
Thanks for sharing with the community!
Looks like the zip file got corrupted. I’ll repair the link and post here to let you know it’s been fixed. Thanks!
Tony
Hi Tony,
Thanks a lot for this contribution.
I’ve been navigating in the source code of this post and i’ve found a possible error on ChangeTrackingHelper.OnItemChanged method. I think that the string “TrackingState” should be changed for “ObjectState”. Could be a infiltrated portion of code from your MSDN article where the tracking property was called TrackingState…
Regards.
Thanks, I’ll be sure to have a look.
Pingback: Trackable Entities: N-Tier Support for Entity Framework | Tony Sneed's Blog
Your links have been erased. I want to know if object lock is needed in ef4 when using updatechanges.
I’ll be sure to fix the links.