One of the great things about working at Google is that almost all of my colleagues are smarter than me. True, they don’t generally know as much about C#, but they know about language design, and they sure as heck know about distributed/parallel/async computing.
One of the great things about having occasional contact with the C# team is that when Mads Torgersen visits London later in the week, I can introduce him to these smart colleagues. So, I’ve been spreading the word about C# 5’s async support and generally encouraging folks to think about the current proposal so they can give feedback to Mads.
One particularly insightful colleague has persistently expressed a deep concern over who gets to control how the asynchronous execution works. This afternoon, I found some extra information which looks like it hasn’t been covered much so far which may allay his fears somewhat. It’s detailed in the Task-based Asynchronous Pattern documentation, which I strongly recommend you download and read right now.
More than ever, this post is based on documentation rather than experimentation. Please take with an appropriately large grain of salt.
What’s the problem?
In a complex server handling multiple types of request and processing them asynchronously – with some local CPU-bound tasks and other IO-bound tasks – you may well not want everything to be treated equally. Some operations (health monitoring, for example) may require high priority and a dedicated thread pool, some may be latency sensitive but support load balancing easily (so it’s fine to have a small pool of medium/high priority tasks, trusting load balancing to avoid overloading the server) and some may be latency-insensitive but be server-specific – a pool of low-priority threads with a large queue may be suitable here, perhaps.
If all of this is going to work, you need to know for each asynchronous operation:
- Whether it will take up a thread
- What thread will be chosen, if one is required (a new one? one from a thread pool? which pool?)
- Where the continuation will run (on the thread which initiated the asynchronous operation? the thread the asynchronous operation ran on? a thread from a particular pool?)
In many cases reusable low-level code doesn’t know this context… but in the async model, it’s that low-level code which is responsible for actually starting the task. How can we reconcile the two requirements?
Controlling execution flow from the top down
Putting the points above into the concrete context of the async features of C# 5:
- When an async method is called, it will start on the caller’s thread
- When it creates a task (possibly as the target of an await expression) that task has control over how it will execute
- The awaiter created by an await expression has control (or at the very least significant influence) over how where the next part of the async method (the continuation) is executed
- The caller gets to decide what they will do with the returned task (assuming there is one) – it may be the target of another await expression, or it may be used more directly without any further use of the new language features
Whether a task requires an extra thread really is pretty much up to the task. Either a task will be IO-bound, CPU-bound, or a mixture (perhaps IO-bound to fetch data, and then CPU-bound to process it). As far as I can tell, it’s assumed that IO-bound asynchronous tasks will all use IO completion ports, leaving no real choice available. On other platforms, there may be other choices – there may be multiple IO channels for example, some reserved for higher priority traffic than others. Although the TAP doesn’t explicitly call this out, I suspect that other platforms could create a similar concept of context to the one described below, but for IO-specific operations.
The two concepts that TAP appears to rely on (and I should be absolutely clear that I could be misreading things; I don’t know as much about the TPL that all of this is based on as I’d like) are a SynchronizationContext and a TaskScheduler. The exact difference between the two remains slightly hazy to me, as both give control over which thread delegates are executed on – but I get the feeling that SynchronizationContext is aimed at describing the thread you should return to for callbacks (continuations) and TaskScheduler is aimed at describing the thread you should run work on – whether that’s new work or getting back for a continuation. (In other words, TaskScheduler is more general than SynchronizationContext - so you can use it for continuations, but you can also use it for other things.)
One vital point is that although these aren’t enforced, they are designed to be the easiest way to carry out work. If there are any places where that isn’t true, that probably represents an issue. For example, the TaskEx.Run method (which will be Task.Run eventually) always uses the default TaskScheduler rather than the current TaskScheduler – so tasks started in that way will always run on the system thread pool. I have doubts about that decision, although it fits in with the general approach of TPL to use a single thread pool.
If everything representing an async operation follows the TAP, it should make it to control how things are scheduled "from this point downwards" in async methods.
ConfigureAwait, SwitchTo, Yield
Various "plain static" and extension methods have been provided to make it easy to change your context within an async method.
SwitchTo allows you to change your context to the ThreadPool or a particular TaskScheduler or Dispatcher. You may not need to do any more work on a particular high priority thread until you’ve actually got your final result – so you’re happy with the continuations being executed either "inline" with the asynchronous tasks you’re executing, or on a random thread pool thread (perhaps from some specific pool). This may also allow the new CPU-bound tasks to be scheduled appropriately too (I thought it did, but I’m no longer sure). Once you’ve got all your ducks in a row, then you can switch back for the final continuation which needs to provide the results on your original thread.
ConfigureAwait takes an existing task and returns a TaskAwaiter – essentially allowing you to control just the continuation part.
Yield does exactly what it sounds like – yields control temporarily, basically allowing for cooperative multitasking by allowing other work to make progress before continuing. I’m not sure that this one will be particularly useful, personally – it feels a little too much like Application.DoEvents. I dare say there are specialist uses though – in particular, it’s cleaner than Application.DoEvents because it really is yielding, rather than running the message pump in the current stack.
All of these are likely to be used in conjunction with await. For example (these are not expected to all be in the same method, of course!):
// Now get back to the dispatcher thread to manipulate the UI
var task = new WebClient().DownloadStringTaskAsync(url);
// Don’t bother continuing on this thread after downloading; we don’t
// care for the next bit.
await ConfigureAwait(task, flowContext: false);
foreach (Job job in jobs)
// Do some work that has to be done in this thread
// Let someone else have a turn – we may have a lot to
// get through.
// This will be Task.Yield eventually
Is this enough?
My gut feeling is that this will give enough control over the flow of the application if:
- The defaults in TAP are chosen appropriately so that the easiest way of starting a computation is also an easily "top-down-configurable" one
- The top-level application programmer pays attention to what they’re doing, and configures things appropriately
- Each component programmer lower down pays attention to the TAP and doesn’t do silly things like starting arbitrary threads themselves
In other words, everyone has to play nicely. Is that feasible in a complex system? I suspect it has to be really. If you have any "rogue" elements they’ll manage to screw things up in any system which is flexible enough to meet real-world requirements.
My colleague’s concern is (I think – I may be misrepresenting him) largely that the language shouldn’t be neutral about how the task and continuation are executed. It should allow or even force the caller to provide context. That would make the context hard to ignore lower down. The route I believe Microsoft has chosen is to do this implicitly by propagating context through the "current" SynchronizationContext and TaskScheduler, in the belief that developers will honour them.
A complex asynchronous system is like a concerto played by an orchestra. Each musician is responsible for keeping time, but they are given direction from the conductor. It only takes one viola player who wants to play fast and loud to ruin the whole effect – so everyone has to behave. How do you force the musicians to watch the conductor? How much do you trust them? How easy is it to conduct in the first place? These are the questions which are hard to judge from documentation, frankly. I’m currently optimistic that by the time C# 5 is actually released, the appropriate balance will have been struck, the default tempo will be appropriate, and we can all listen to some beautiful music. In the key of C#, of course.