Tony Sneed's Blog

My last post made the case for building async services in .NET 4.5 with the Task-based Asynchronous Pattern, or TAP for short. I profiled an ASP.NET Web API service that uses synchronous methods, showing how long-running services with high throughput result in the production of extra threads and the associated overhead carried by all those threads – both in terms of memory (1 MB stack space per thread) and CPU utilization (context switching). However, thanks to Tasks and their support in the C# programming language with the async and await keywords (also included with VB.NET), building asynchronous services with the ASP.NET Web API is straightforward and relatively painless.

Download the code for this post here.

The idea is to avoid blocking the service thread when performing IO-bound asynchronous operations, such as accessing the network, file system, or a database.  In these and similar scenarios, a a dedicated thread is not required while the device performs its work.  Luckily, many API’s have been updated to include Task-based methods, making it easy to await them from within an async service operation.  This includes Entity Framework 6 (available at the time of this writing as a pre-release NuGet package), HttpClient (for calling other HTTP-based services), and WCF 4.5 (for SOAP-based services).

Adapting EAP Methods

I also described how you can use Task.Factory.FromAsync to convert legacy Begin/End methods from the APM pattern to a Task you can await on.  In addition, TAP provides a class called TaskCompletionSource, which lets you take any API and wrap it in a Task and affect its status (Cancelled, Faulted, or RanToCompletion, etc).  Here you can adapt non-async operations, such as reading from a queue or responding to Timer ticks, to simulate IO-bound Task-based operations. Here is an example of using TaskCompletionSource to adapt the Event Async Pattern to a Task by setting the task state in the event handler for a WebClient.

public async Task<string> WebServiceEventAsync(int id)
{
    // Wire up event handler to task completion source
    var address = string.Format("{0}{1}", WebServiceAddress, id);
    var client = new WebClient {BaseAddress = address};
    var tcs = new TaskCompletionSource<string>();
    client.DownloadStringCompleted += (s, e) =>
        {
            if (e.Cancelled) tcs.TrySetCanceled();
            else if (e.Error != null)tcs.TrySetException(e.Error);
            else tcs.TrySetResult(e.Result);
        };

    // Run async method and await tcs's task
    client.DownloadStringTaskAsync(address).NoWarning();
    string json = await tcs.Task;
    var result = JsonConvert.DeserializeObject<string>(json);
    return result;
}

NoWarning is an extension method you can use to suppress the compiler warning for calling an awaitable method without awaiting on it.

public static class TaskExtensions
{
    // Silences compiler warning
    public static void NoWarning(this Task task) { }
}

Help Page and Test Client

It’s easy to test ASP.NET Web API operations without creating a manual test client.  First, make sure to install the ASP.NET and Web Tools 2012.2 Update, so that you get the Web API help page.

Then, add the WebApiTestClient NuGet package to your project, and add the following lines to Areas/HelpPage/Views/Help/Api.cshtml:

@Html.DisplayForModel("TestClientDialogs")
@section Scripts {
<link type="text/css" href="~/Areas/HelpPage/HelpPage.css" rel="stylesheet" />
    @Html.DisplayForModel("TestClientReferences")
}

Select an operation, then click Test API, enter parameter values and/or a body, and click Send.

The response will appear in a friendly box.

Multiple Downstream Services

Now that we’ve looked at adapting APM methods to TAP with TaskCompletionSource, let’s have a little more fun with a service that calls multiple downstream services.  You might, for example, want to call two or more other services sequentially from within your async Web API service.  Because await obviates the need for continuation or callback methods, it’s simply a matter of awaiting the first service call, then immediately awaiting the second service call.

public async Task<string> SequentialServicesAsync(int id)
{
    // Start tasks sequentially
    int id1 = id;
    int id2 = id1 + 1;
    var results = new string[2];
    results[0] = await WebServiceAsync(id1);
    results[1] = await SoapServiceAsync(id2);
    var result = string.Join(" ", results);
    return result;
}

The C# compiler magically transforms the code after the first await into a non-blocking continuation.  The same thing is done with code following subsequent awaits, all without nested lambdas.

This pattern is useful for scenarios where you might predicate the second service operation on results from the first, in which case you wouldn’t want to begin the second operation until the first has completed.  There are cases, however, when you would instead want to call multiple downstream operations in parallel.  Your friends here are the combinator methods, Task.WhenAll and Task.WhenAny.  Use Task.WhenAll when you want to process the results after all the operations have completed; use Task.WhenAny when you want to process each operation as it completes.  Here is an example of the latter.

public async Task<string> ParallelServicesAsync(int id)
{
    // Start tasks in parallel (with delay)
    int id1 = id;
    int id2 = id1 + 1;
    var task1 = SoapServiceAsync(id2);
    await Task.Delay(TimeSpan.FromSeconds(3));
    var task2 = WebServiceAsync(id1);

    // Harvest results as they arrive (in lieu of Task.WhenAll)
    var results = new List<string>();
    var tasks = new List<Task<string>> { task1, task2 };
    while (tasks.Count > 0)
    {
        Task<string> task = await Task.WhenAny(tasks);
        tasks.Remove(task);
        string r = await task;
        Debug.Print("Response received: {0}", r); // process
        results.Add(r);
    }
    var result = string.Join(" ", results);
    return result;
}

First, we start each task, with a slight delay to offset execution. Then we iteration over the list of tasks, calling Task.WhenAny to get the first task that has completed, so we can remove it from the list and process the result.

Lastly, we’ll look at a scenario where we want to retry an individual operation a specific number of times, aborting only when the maximum number of retries has been met.

public async Task<string> RetryServiceAsync(int id)
{
    // Create http client that has a timeout
    var client = new HttpClient
    {
        BaseAddress = new Uri(WebServiceAddress),
        Timeout = TimeSpan.FromSeconds(3)
    };

    string result = null;
    const int maxRetries = 3;
    for (int i = 0; i < maxRetries; i++)
    {
        try
        {
            // Succeed only on last attempt
            if (i == maxRetries - 1)
                client = new HttpClient { BaseAddress = new Uri(WebServiceAddress),
                    Timeout = TimeSpan.FromSeconds(6) };
            string json = await client.GetStringAsync(id.ToString());
            result = JsonConvert.DeserializeObject<string>(json);
            break;
        }
        catch (Exception ex)
        {
            Debug.Print("Attempt {0} Error: {1}", i + 1, ex.Message); // log
            if (i == maxRetries - 1)
                throw new Exception(string.Format("Max retries of {0} exceeded", maxRetries));
        }
    }
    return result;
}

Here we see how much easier async exception handling is made with the await keyword, because the exception is handled in the continuation that takes place after the await.  In this example, we create an HttpClient with a 3 second timeout.  Because the service operation is coded to take at least 5 seconds (not shown here), the call to client.GetStringAsync will fail with an OperationCancelledException, in which case we simply log the error and continue the retry loop, or throw an exception if we’ve exceeded the max retries.  We can avoid that by increasing the timeout to 6 seconds for the last iteration – we can also comment out that code to see the retry max exceeded.

The important thing to note about each of these examples is that we are not blocking the service thread when calling multiple IO-bound async operations. When we await an async service operation, the remainder of the method is re-written by the compiler as a callback, which in this case will execute on a Thread Pool thread. But during the time in which the downstream service operation is executing, our service thread will quickly exit, so that it can be used for another incoming service request. That is an essential ingredient for scalable services that can handle high-latency IO with high levels of throughput while creating the minimum number of threads needed to service each request.

If you are building web services that interact with a database, chances are they are not written in a scalable fashion.  Web services based either on WCF, which supports both SOAP and REST, or on ASP.NET Web API, which exclusively supports REST, use the .NET Thread Pool to respond to requests.  But just because services are inherently multithreaded does not make them scale when numerous requests are made simultaneously.  The very reason why threads are pooled, instead of created on the fly, is because they are an extremely expensive resource, both in terms of memory and CPU utilization.  For example, each thread consumes about 1 MB of stack space, in addition to the register set context and thread properties.

Download the code for this post here. (Updated with code from this post.)

So once a thread has finished its work, it stays alive for about a minute, in the off-chance another request will arrive and the waiting thread can be used to service it.  This means that if during the time a service request is being executed, another request arrives, an additional thread will be retrieved from the Thread Pool to service the second request.  If there are no available threads, one will be created from scratch, which can take up to 500 milliseconds, during which time the request will block.  If you have numerous requests for operations that take a long time to complete, more and more threads will be created, consuming additional memory and negatively affecting your service’s performance.

The moral of the story is: do not block the executing thread from within a service operation.

Yet, this is precisely what happens when you perform an IO-bound task, such as when you retrieve or save data from a database, or invoke a downstream service. Here is an example of one such operation.

public class ProductsController : ApiController
{
    public IEnumerable<Product> Get()
    {
        using (var ctx = new NorthwindSlimContext())
        {
            List<Product> products =
                (from p in ctx.Products.Include("Category")
                    orderby p.ProductName
                    select p).ToList();
            return products;
        }
    }
}

If the call to the database were to take a few seconds or more, and another call comes in (even for another method) an additional thread would need to be procured from the Thread Pool. If there are no available threads, then one would have to be created.  To get a rough picture of this phenomenon, I opened up the Windows Performance Monitor (Run, perfmon), and added a counter from the .NET CLR LocksAndThreads category, which shows the number of current physical threads for the ASP.NET Development Web Server, WebDev.WebServer.40 (aka Cassini).

Running a client that issues a request every 100 milliseconds, I found that the number of current threads for the web server process increased to an average of about 55.

Thanks to support for asynchronous programming in .NET 4.5 and C# 5, it is extremely easy to write asynchronous methods for an ASP.NET Web API service.  Simply set the return type either to Task (if the synchronous version returns void) or to Task<T>, replacing T with the return type of the synchronous method.  Then from within the method, execute a non-blocking IO-bound async operation.  Marking the method as async  allows you to await an asynchronous operation, with the compiler converting the remainder of the methods into a continuation, or callback, that executes on another thread, usually taken from the Thread Pool.

public async Task<IEnumerable<Product>> Get()
{
    using (var ctx = new NorthwindSlimContext())
    {
        List<Product> products = await
            (from p in ctx.Products.Include("Category")
                orderby p.ProductName
                select p).ToListAsync();
        return products;
    }
}

What makes this code possible is async supported added to Entity Framework 6, available at the time of this writing as a Prerelease NuGet package.  In addition to the ToListAsync method, there are async versions of SingleOrDefault and other query methods, as well as SaveChanges – as described in the EF6 Async Spec.  What is important to emphasize here is that the async call to the database is IO-bound versus compute-bound.  In other words, the currently executing thread is returned immediately to the thread pool and becomes available to service other incoming requests.

When I profiled the async version of the service operation, the number of current threads hovered at around 16 — a tremendous improvement over the synchronous version!

Besides database operations, another candidate for async IO would be calling downstream web services.  HttpClient, which comes with ASP.NET Web API, already has a task-based API for http requests, which makes it super easy to use from within an async service operation.

public async Task<string> WebServiceAsync(int id)
{
    var client = new HttpClient { BaseAddress = 
        new Uri(WebServiceAddress) };
    string json = await client.GetStringAsync(id.ToString());
    var result = JsonConvert.DeserializeObject<string>(json);
    return result;
}

If you are using an older web client, such as WebRequest, which has Begin and End methods, Task.Factory has a FromAsync method to help you convert it to a task-based model.

public async Task<string> WebServiceFromAsync(int id)
{
    var address = string.Format("{0}{1}", WebServiceAddress, id);
    var webRequest = WebRequest.Create(address);
    var webResponse = await Task<WebResponse>.Factory.FromAsync
        (webRequest.BeginGetResponse(null, null), 
         webRequest.EndGetResponse);
    using (var reader = new StreamReader
        (webResponse.GetResponseStream()))
    {
        string json = reader.ReadToEnd();
        var result = JsonConvert.DeserializeObject<string>(json);
        return result;
    }
}

If you are calling a SOAP service, WCF 4.5 has added support for async task-based service operations.

    [ServiceContract(Namespace = "urn:examples:services")]
    public interface IValuesService
    {
        [OperationContract(Action = "GetValue", 
            ReplyAction = "GetValueResponse")]
        Task<string> GetValue(int id);
    }

The code creating a client channel looks the same, but we can now await the call to GetValue, which frees up the currently executing thread for the async service operation.

public async Task<string> SoapServiceAsync(int id)
{
    var factory = new ChannelFactory<IValuesService>
        (new BasicHttpBinding());
    var address = new EndpointAddress(SoapServiceAddress);
    var client = factory.CreateChannel(address);
    using ((IDisposable)client)
    {
        string result = await client.GetValue(id);
        return result;
    }
}

The addition of async support for EF6 is a quantum leap forward for developers striving to write scalable services that need to perform data access. Task-based asynchrony with C# language support have been woven into both WCF and ASP.NET Web API, simplifying the effort it takes to compose asynchronous service operations.  Enjoy.