Testing Strategies Involving Async Functions

Some things to keep in mind when writing units tests for code that use
async methods: You’re not trying to test the framework’s “awaitability”
and you’re not trying to test framework methods that are “awaitable”. 
You want to test your code in certain isolation contexts.  One context,
of course, is independent of asynchronicity–do individual units of code
that don’t depend on asynchronous invocation “work”…  e.g.
“Task<string> MyMethodAsync()”, you want to have a unit test that
make sure this method does what it’s supposed to do (one being it
returns a “valid” Task<string> object, the other being that
individual side-effects occur, if any).  Another context is dependent on
asynchronicity–do individual units of code that do depend on
asynchronous invocation (or depend on something being invoked
asynchronously) “work”. 



It’s the second context that seems to be the hardest to grasp for most
people to grasp and action.  
Let’s take this example code:


	private async void startButton_Click(object sender, EventArgs e)
{
try
{
startButton.Enabled = false;
var webRequest = WebRequest.Create("http://google.ca");
using (var response = await webRequest.GetResponseAsync())
using (var stream = response.GetResponseStream())
{
if (stream == nullreturn;
using (var reader = new StreamReader(stream))
{
textBox.Text = await reader.ReadToEndAsync();
}
}
}
finally
{
startButton.Enabled = true;
}
}

A couple invariants that we want to test might be that the button is disabled during the asynchronous operations and enabled after the asynchronous operations.  It’s not immediately obvious what to in order to validate these invariants.


When the compiler encounters the await operator, it actually goes searching for method that can act upon the type of the object being returned by the method used with the await operator.  In our first use of await (on GetResponseAsync()) the return type is Task<T> (Task<WebResponse> specifically, but for our example I’ll use Task<T>).  There’s various criteria the computer uses to search for this method, one method is to search for extension methods that match the name and return an “awaiter” type that has the following methods: BeginAwait and EndAwait (more details can be found in the Aync CTP documentation).  The compiler doesn’t allow us to inject an awaiter type in the normal run-time dependency injection semantics; but it does allow us to implement an awaiter that does support dependency injection at run-time.  To do this I would create an ITaskAwaiter interface like this:


	public interface ITaskAwaiter<out T>
{
bool BeginAwait(Action continuation);
T EndAwait();
}

I then need to create a GetAwaiter(this Task<T>) extension method to return an implementation of this type.  The built-in System.Runtime.CompilerServices.TaskAwaiter<T> is public; but, unfortunately it’s constructor (and the ability to pass in a Task<T>) object is internal.  So, we can’t simply wrap the built-in awaiter and delegate to it as a default.  We actually have to write an awaiter that mimics what the built-in one does.  For example:


	public class TaskAwaiter<T> : ITaskAwaiter<T>
{
private readonly Task<T> task;

public TaskAwaiter(Task<T> task)
{
this.task = task;
}

public bool BeginAwait(Action continuation)
{
if (task.IsCompleted) return false;
var synchronizationContext = SynchronizationContext.Current;
Action<Task> action = theTask => 
                       {
                       if (synchronizationContext != null)
                       {
                       synchronizationContext.Post(state => continuation(), null);
                       }
                       else
                       {
                       continuation();
                       }
                       };

task.ContinueWith(action, CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Current);
return true;
}

public T EndAwait()
{
return task.Result;
}
}

This class is given the Task<T> in question, it implements a BeginWait method that sets up an Action delegate to invoke the continuation given to it in a specific synchronization context, then tells the task to use that new action as it’s continuation via a call to ContinueWith.  When the task is completed EndAwait will be called and we simply return the result of the task.


Now, in order to add the ability to inject a customer awaiter, we need to provide the GetAwaiter extension method.  For example:


	namespace PRI.Extensions
{
static class TaskExtensions
{
static public ITaskAwaiter<TResult> GetAwaiter<TResult>(this Task<TResult> task)
{
return TaskAwaiterFactory<TResult>.CreateTaskAwaiter != null
TaskAwaiterFactory<TResult>.CreateTaskAwaiter(task) : new TaskAwaiter<TResult>(task);
}
}
}

This extension method invokes a factory method delegate to create the ITaskAwaiter<T> object (or simply creates our new default awaiter).  This factory method looks like this:


	internal static class TaskAwaiterFactory<TResult>
{
private static Func<Task<TResult>, ITaskAwaiter<TResult>> createTaskAwaiter = task =>
                                                                               {
                                                                               Current = new TaskAwaiter<TResult>(task);
                                                                               return Current;
                                                                               };
public static Func<Task<TResult>, ITaskAwaiter<TResult>> CreateTaskAwaiter
{
get { return createTaskAwaiter; }
set
{
createTaskAwaiter = task =>
                     {
                     Current = value(task);
                     return Current;
                     };
}
}
public static ITaskAwaiter<TResult> Current { getprivate set; }
}

We can override this factory method and give a delegate to code that creates another type of ITaskAwaiter<T> implementation.  We also decorate the call to that provided delegate with something that keeps track of the Current awaiter (more on that later).  Now, we simply need to add a using statement within our namespace for the compiler to user our GetAwaiter when it compiles an await operator and use our factory.  For example:


	namespace PRI.MainApplication
{
using PRI.Extensions;

//...

We’re now all set to inject a new awaiter and be able to isolate our code from the framework and validate some invariants.  In our case, we’d like to check to make sure that the button is disabled during invocation of asynchronous operations and enabled when done.  So, we can create a spy awaiter that basically does nothing and sends back a fake or dummy object.  In our example we actually have two different types of Task<T> objects being awaited; so, we need to create a couple of awaiter spys.  One is generic and one is specific.  For example:


	public class WebResponseTaskAwaiterSpy : ITaskAwaiter<WebResponse>
{
private class FakeWebResponse : WebResponse
{
public override Stream GetResponseStream()
{
return new MemoryStream(System.Text.Encoding.ASCII.GetBytes("got some text"));
}
}
private readonly MainForm form;
public bool ButtonEnabledState { getprivate set; }
public WebResponseTaskAwaiterSpy(MainForm form)
{
this.form = form;
}

public bool BeginAwait(Action continuation)
{
return false;
}

public WebResponse EndAwait()
{
ButtonEnabledState = form.startButton.Enabled;
return new FakeWebResponse();
}
}

public class TaskAwaiterSpy<T> : ITaskAwaiter<T>
{
private readonly MainForm form;
public bool ButtonEnabledState { getprivate set; }
public TaskAwaiterSpy(MainForm form)
{
this.form = form;
}

public bool BeginAwait(Action continuation)
{
return false;
}

public T EndAwait()
{
ButtonEnabledState = form.startButton.Enabled;
return default(T);
}
}

Both classes don’t actually execute the task because we know task execution works and we know it works asynchronously–we’re not trying to test that.  The clases return false from BeginAwait to tell the generated code that the asynchronous code “completed” (we lie).  The generated code the calls the EndAwait method to get the result of the so-called asynchronous operation.  In both cases, we get the state of the button in EndAwait.  In the case of the generic spy, we simply return a dummy–the default value for that type parameter.  In the case of the WebResponse spy, we can’t send back a dummy because that would be a null value and we’d get NullReferenceEexceptions that would abort our test.  So, we create a FakeWebResponse object and send that back that basically creates a canned response stream in memory and sends it back and that will be used in our second await.


Now, we can write a unit test for startButton_Click method.  For example:


	MainForm form = CreateForm();
TaskAwaiterFactory<WebResponse>.CreateTaskAwaiter = task => new WebResponseTaskAwaiterSpy(form);
TaskAwaiterFactory<String>.CreateTaskAwaiter = task => new TaskAwaiterSpy<String>(form);

form.startButton_Click(nullEventArgs.Empty);

var spy1 = TaskAwaiterFactory<WebResponse>.Current as WebResponseTaskAwaiterSpy;
var spy2 = TaskAwaiterFactory<String>.Current as TaskAwaiterSpy<String>;

Assert.IsTrue((spy2 != null && !spy2.ButtonEnabledState) && (spy1 != null && !spy1.ButtonEnabledState));
Assert.IsTrue(form.startButton.Enabled);

This code injects our two types of awaiters (our spies), invokes the method, then checks the spies for the values we’re looking for as well as the current state of our button (we assume the current state at this point is the same state immediately after the asynchronous operation completed) to validate our invariants.


 


 


kick it on DotNetKicks.com

Deep Dive on Closure Pitfalls

I’ve blogged about closures in C# and their pitfalls before.  I keep seeing problems with closures–more now that lambdas expressions and statements (“lambdas”) are becoming more widespread–even with experienced developers. So, I’d thought i’d post about some of the details surrounding where the C# compiler generates closures in the hopes that people will recognize more where they write code that creates a closure and its context.


The C# language spec does not refer specifically to “closures”, with regard to capturing state for anonymous methods (including lambdas)–it refers to “outer variables” and “captured outer variables”.  The captured outer variables for a specific anonymous method are the “closure”.


The compiler captures the state of variables within the local scope of the anonymous method in a compiler-generated class.  The compiler effectively also captures the declaration scope of the state within the closure.  If a variable is declared outside the local scope, that is echoed in the use of that closure.  If the variable is declared within the local scope, that is captured in use of the closure.  So, an anonymous method that uses state (a variable) declared outside the local scope of the anonymous method, the compiler will generate a closure use that closure so that variable is only instantiated once.  For example: 


	ICollection<Func<string>> Funcs = new List<Func<string>>(); 
String[] texts = new String[] { "one""two""three" };
foreach (String text in texts)
{
Funcs.Add(delegate { return text; });
}

We have an anonymous method used within the local scope of the body of the foreach loop.  The anonymous method uses the text variable declared outside the scope of the foreach loop.  The compiler generates a class to contain the state and the code that uses that state (i.e. the anonymous method)–otherwise the highlighted code–and make use of that generated class.  For example:


	Closure closure = new Closure();
for (int i = 0; i < texts.Length; ++i)
{
closure.text = texts[i];
Funcs.Add(new Func<string>(closure.Func));
}

The compiler-generated class may look something like this:


	internal class Closure
{
public string text;
public string Func()
{
return text;
}
}

The compiler instantiates that class outside the generated for loop (because there is no “foreach” loop in .NET IL–what we see here for generated code is a C# view of IL) to match the single declaration of the text variable (as the body of the foreach loop sees it).  So, as we can tell from this code, each of these Func delegates are accessing the same variable.  So, if we invoked each of those delegate, they would all return “three”.  While the code looks like each Func delegate is dealing with a different value (“one”, then “two”, then “three”); it’s not and you’ve probably got a logic error with this code.


So, as I pointed out in a previous post, we get around the problem by bringing the value from the outer scope into the inner scope of the anonymous method.  For example:


	foreach (String text in texts)
{
var temp = text;
Funcs.Add(delegate { return temp; });
}

Now, the compiler needs to generate a class to model this local scope–or the highlighted code.  This local scope includes the creation of an object and assigning it a specific value.  So, what the compiler generates now would be similar to the following:


	for (int i = 0; i < texts.Length; ++i)
{
Closure closure = new Closure();
closure.temp = texts[i];
Funcs.Add(new Func<string>(closure.Func));
}

With a closure class similar to:


	private class Closure
{
public string temp;
public string Func()
{
return temp;
}
}

As we can see here, the compiler is now instantiating a Closure class once for each iteration of the loop, assigning a specific value to the temp field, then passing a delegate to the Func instance method to a new Func<T> delegate.  Now, if we invoke each of our Func delegates we get what we expect “one”, then “two”, then “three”.


I’ve specifically chosen deferred execution in these examples to separate parallel processing and what’s going on.  The same issue occurs when we use anonymous delegates with multiple threads.   For example: 


	var texts = new[] { "one""two""three" };
foreach (var text in texts)
{
ThreadPool.QueueUserWorkItem(v => Trace.WriteLine(text));
}

 This anonymous method (as a lambda) would generate code similar to:


	Closure closure = new Closure();
for (int i = 0; i < texts.Length; ++i)
{
closure.text = texts[i];
ThreadPool.QueueUserWorkItem(new WaitCallBack(closure.WaitCallback));
}

As you can see, each thread is actually dealing with a single variable (Closure.text). At worst case when each thread runs, it outputs “three”–otherwise it’s undefined what you’ll see.  We fix that the same way we did previously by adding a variable into the local scope:


	foreach (var text in texts)
{
var temp = text;
ThreadPool.QueueUserWorkItem(v => Trace.WriteLine(temp));
}

This then generates something like this:


	for (int i = 0; i < texts.Length; ++i)
{
Closure closure = new Closure();
closure.temp = texts[i];
ThreadPool.QueueUserWorkItem(new WaitCallback(closure.WaitCallback));
}

This means each thread is operating on an independent variable (the Closure.temp field of each instance of Closure that was instantiated).


I hope this makes it more clear what the compiler is doing when you use anonymous methods (including Lambdas).


kick it on DotNetKicks.com