Category Archives: 16622

Review of ‘Software Testing using Visual Studio 2012’ from Packt Publishing

I have just been reading Software Testing using Visual Studio 2012 by Subashni. S and Satheesh Kumar. N from Packt Publishing

9540EN_cov-stwvs2012

This book does what it says on the cover, it is a general introduction to the testing tools within the Visual Studio 2012 family. My comment is not about how well it is done, it is a clear enough introduction, but why produce a book that really just covers what is in MSDN, Channel9, numerous podcasts, blogs and ALM Rangers documentation?

I suppose this is a question of target audience, some people like to browse a physical book for ‘new’ technology, I can see that (though I tried it on Kindle, more of that later). This book certainly does cover the core areas, but sits strangely between a technology briefing for a manager/person who just needs an overview (it is all a bit long winded, list all the features and flags of tools) and not enough detail for the practitioner (the exercises do not go deep enough unlike those provide by Microsoft in Brian Keller VS/TFS demo VM series)

Given this concern I wonder who the target audience really is?

A real issue here is that Microsoft have gone to quarterly updates, so the product is always advancing, faster than any print book can manage (Microsoft’s own MSDN documentation has enough problems keeping up, and frequently is play catch up). For a book on testing this is a major problem as ‘test’ has been a key focus for the updates. This means when the book’s contents is compared to Visual Studio/TFS 2012.3 (the current shipping version at the time of this review) there are major features missing such as

  • The improvements in Test Explorer to support other non Microsoft test framework, playlists etc.,
  • SKU changes in licensing, MTM dropping down to Premium form Ultimate
  • Azure based load testing
  • The test experience in the web browser (as opposed to MTM)

The list will always grow while Microsoft stick to their newer faster release cycle. This was not too much of a problem when Microsoft shipped every couple of years, a new book opportunity, but now how can any book try to keep up on a 12 week cycle?

One option you would think is Kindle or eBooks in general, as at least the book can be updated . However there is still the issue of the extra effort of the authors and editors, so in general I find these updates are not that common. The authors will usually have moved onto their next project and not be focused on yet another unpaid update to a book they published last quarter.

As to my experience on the Kindle, this was the first technical book I have read on one. I have used the Kindle App on a phone for a couple of years for my novel reading, but always felt the screen was too small for anything that might have a diagram in it. I recently bought a Kindle Paperwhite so though I would give this book a go on it. I initially tried to email the book from the Packt site straight to my Kindle, but this failed (a file size issue I am told by Packt customer support), but a local copy of USB was fine.

So how was the Kindle experience? OK, it did the job, everything was clear enough,  it was not a super engaging reading experience but it is a technical book, what do you expect? It was good enough that I certainly don’t see my getting too many paper books going forward whether thet be novels or technical books.

So in summary, was the book worth the effort to read? I always gauge this question on ‘did I learn something?’ and I did. There is always a nugget or two in books on subjects you think you know. However, ‘would I say it is a really useful/essential read for anyone who already has a working knowledge in this subject?’, probably not. I would say their time is better spent doing a hand on lab or watching conference recordings on Channel9.

Leave this book to anyone who wants a general written introduction to the subject of Microsoft specific testing tooling.

TFS Test Agent cannot connect to Test Controller – Part 2

I posted last week on the problems I had had getting the test agents and controller in a TFS2012 Standard environment talking to each other and a workaround. Well after a good few email with various people at Microsoft and other consultants at Black Marble I have a whole range of workarounds solutions.

First a reminder of my architecture, and note that this could be part of the problem, it is all running on a single Hyper-V host. Remember this is a demo rig to show the features of Standard Environments. I think it is unlikely that this problem will be seen in a more ‘realistic’ environment i.e. running on multiple boxes

 

image

 

The problem is that when the test agent running on the Server2008 should request the test controller (running the on VSTFS server) should call it back on either it 169.254.x.x address or on abn address obtained via DHCP from the external virtual switch. However the problem is it is requesting a call back on 127.0.0.1, as can be seen in the error log

Unable to connect to the controller on ‘vstfs:6901′. The agent can connect to the controller but the controller cannot connect to the agent because of following reason: No connection could be made because the target machine actively refused it 127.0.0.1:6910. Make sure that the firewall on the test agent machine is not blocking the connection.

The root cause

It turns out the root cause of this problem was I had edited the c:\windows\system32\drivers\etc\hosts file on the test server VM to add an entry to allow a URL used in CodedUI tests to be resolved to the localhost

127.0.0.1   www.mytestsite.com

Solution 1 – Edit the test agent config to bind to a specific address

The first solution is the one I outlined in my previous post, tell the test agent to bind to a specific IP address. Edit

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\QTAgentService.exe.config

and added a BindTo line with the correct address for the controller to call back to the agent

<appSettings>
     // other bits …
      <add key="BindTo" value="169.254.1.1"/>
</appSettings>

The problem with this solution you need to remember to edit a config file, all seems a bit complex!

Solution 2 – Don’t resolve the test URL to localhost

Change the hosts file entry used by the CodedUI test to resolve to the actual address of the test VM e.g.

169.254.1.1   www.mytestsite.com

Downside here is you need to know the test agents IP address, which depending on the system in use could change, and will certainly be different on each test VM in an environment. Again all seems a bit complex and prone to human error.

Solution 3 – Add an actual loopback entry to the hosts file.

The simplest workaround which Robert Hancock at Black Marble came up with was to add a second entry to the hosts file for the name loopback

127.0.0.1   localhost
127.0.0.1   www.mytestsite.com

Once this was done the test agent could connect, I did not have to edit any agent config files, or know the address the agent need to bind to. By far the best solution

 

So thanks to all who helped get to the bottom of this surprisingly complex issue.

Experiences upgrading an MVC 1 application to MVC 3

I have recently had to do some work on a MVC 1 application and thought it sensible to bring it up to MVC 3, you don’t want to be left behind if you can avoid it. This was a simple data capture application written in MVC1 in Visual Studio 2008 and never needed to be touched since. A user fills in a form, the controller then takes the form contents and stores it. The key point to note here is that it was using the Controller.UpdateModel<TModel> Method (TModel, IValueProvider) method, so most of the controller actions look like

[AcceptVerbs(HttpVerbs.Post)]
       public ActionResult PostDataToApplication(FormCollection form)
       {
           NewApplication data = new NewApplication();
           try
           {
               UpdateModel(data, form.ToValueProvider());
              
               // process data object
               bool success = DoSomething(data);

               if (success)
               {
                   return RedirectToAction("FormSuccess", "Home");
               }
               else
               {
                   return RedirectToAction("FormUnsuccessful", "Home");
               }
           }
           catch (InvalidOperationException)
           {
               return View();
           }
       }

This worked fine on MVC 1 on VS2008, but I wanted to move it onto MVC3 on VS2012 if possible; with a little changes as possible as this is a small web site that has very few changes so not worth a major investment in time to keep updated to current frameworks. So these were steps I took and gotcha’s I found

The upgrade itself

First I opened the VS2008 solution in VS2010 and it automatically upgraded to MVC2, a good start!

I then used the MVC2 to MVC3 tool on Codeplex, this initially failed and it took me a while to spot that you can only use this tool if your MVC2 application targets .NET 4. Once I changed the MVC2 to target .NET 4 as opposed to 3.5 this upgrade tool worked fine.

I could now load my MVC3 application in either VS2010 or VS2012.

Using the Web Site

At this point I though I had better test it, and instantly saw a problem. Pages that did not submit data worked fine, but submitting a data capture forms failed with Null Exception errors. Turns out the problem was a change in default behaviour of the models between MVC releases. On MVC1 empty fields on the form were passed as empty strings, with MVC 2(?) and later they are passed as nulls.

Luckily the fix was simple. Previously my model had been

public class SomeModel: IDataErrorInfo
{
     public string AgentNumber { get; set; }
     …..
}

I needed to add a  [DisplayFormat(ConvertEmptyStringToNull = false)] attribute on  each string property to get back to the previous behaviour my controller expected

public class SomeModel: IDataErrorInfo
{
    [DisplayFormat(ConvertEmptyStringToNull = false)]
     public string AgentNumber { get; set; }
     …..
}

Now my web site ran as I had expected.

Unit Tests

I had previously noticed my unit tests were failing. I had expected the change to the model would fix this too, but it did not. On the web there a good many posts as to how unit testing of MVC2 and later fails unless you mock out the controller.context. You see errors in the form

Test method Website.Tests.Controllers.HomeControllerTest.DetailsValidation_AlphaInValidAccountID_ErrorMessage threw exception:
System.ArgumentNullException: Value cannot be null.
Parameter name: controllerContext
Result StackTrace:   
at System.Web.Mvc.ModelValidator..ctor(ModelMetadata metadata, ControllerContext controllerContext)
   at System.Web.Mvc.ModelValidator.CompositeModelValidator..ctor(ModelMetadata metadata, ControllerContext controllerContext)
   at System.Web.Mvc.ModelValidator.GetModelValidator(ModelMetadata metadata, ControllerContext context)
   at System.Web.Mvc.DefaultModelBinder.OnModelUpdated(ControllerContext controllerContext, ModelBindingContext bindingContext)
   at System.Web.Mvc.DefaultModelBinder.BindComplexElementalModel(ControllerContext controllerContext, ModelBindingContext bindingContext, Object model)
   at System.Web.Mvc.DefaultModelBinder.BindComplexModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
   at System.Web.Mvc.DefaultModelBinder.BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
   at System.Web.Mvc.Controller.TryUpdateModel[TModel](TModel model, String prefix, String[] includeProperties, String[] excludeProperties, IValueProvider valueProvider)
   at System.Web.Mvc.Controller.UpdateModel[TModel](TModel model, String prefix, String[] includeProperties, String[] excludeProperties, IValueProvider valueProvider)
   at System.Web.Mvc.Controller.UpdateModel[TModel](TModel model, IValueProvider valueProvider)
   at Website.Controllers.HomeController.Details(FormCollection form)

The fix is to not just new up a controller in your unit tests like this

HomeController controller = new HomeController();

But to have a helper method to mock it all out (which is created for MVC associated test projects for you, so it is easy)

private static HomeController GetHomeController()
{
   IFormsAuthentication formsAuth = new MockFormsAuthenticationService();     
   MembershipProvider membershipProvider = new MockMembershipProvider();
   RoleProvider roleProvider = new MockRoleProvider();

   AccountMembershipService membershipService = new AccountMembershipService(membershipProvider, roleProvider);
   HomeController controller = new HomeController(formsAuth, membershipService);
   MockHttpContext mockHttpContext = new MockHttpContext();

   ControllerContext controllerContext = new ControllerContext(mockHttpContext, new RouteData(), controller);
   controller.ControllerContext = controllerContext;
   return controller;
}

However, this problem with a missing context was not my problem, I was already doing this. The error my test runner was showing did not mention the context, rather binding errors.

Test method CollectorWebsite.Tests.Controllers.HomeControllerTest.CardValidation_AlphaInValidAccountID_ErrorMessage threw exception:
System.NullReferenceException: Object reference not set to an instance of an object.
Result StackTrace:   
at CollectorWebsite.Models.CardRecovery.get_Item(String columnName) 
   at System.Web.Mvc.DataErrorInfoModelValidatorProvider.DataErrorInfoPropertyModelValidator.Validate(Object container)
   at System.Web.Mvc.ModelValidator.CompositeModelValidator.<Validate>d__5.MoveNext()
   at System.Web.Mvc.DefaultModelBinder.OnModelUpdated(ControllerContext controllerContext, ModelBindingContext bindingContext)
   at System.Web.Mvc.DefaultModelBinder.BindComplexElementalModel(ControllerContext controllerContext, ModelBindingContext bindingContext, Object model)
   at System.Web.Mvc.DefaultModelBinder.BindComplexModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
   at System.Web.Mvc.DefaultModelBinder.BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
   at System.Web.Mvc.Controller.TryUpdateModel[TModel](TModel model, String prefix, String[] includeProperties, String[] excludeProperties, IValueProvider valueProvider)
   at System.Web.Mvc.Controller.UpdateModel[TModel](TModel model, String prefix, String[] includeProperties, String[] excludeProperties, IValueProvider valueProvider)
   at System.Web.Mvc.Controller.UpdateModel[TModel](TModel model, IValueProvider valueProvider)
   at CollectorWebsite.Controllers.HomeController.CardRecovery(FormCollection form)

I got stuck here for a good while………

Then it occurred to me if the behaviour has changed such that on the web site I see nulls when I expect empty strings, I bet the same is happening in unit tests. It is trying to iterate though what was a collection of strings and is now at best a collection of nulls or just an empty collection. The bind failed as it could not match the form to the data.

The fix was to make sure in my unit tests I passed in a FormCollection that had all the expected fields (with suitable empty values e.g string.empty). This meant my unit tests changed from

[TestMethod, Isolated]
public void ApplicationValidation_CurrentPostcodeNumbersOnly_ErrorMessage()
{

           // Arrange
           HomeController controller = GetHomeController();
      
    FormCollection form = new FormCollection();
           form.Add("CurrentPostcode", "12345");

           // Act
           ViewResult result = controller.Application(form) as ViewResult;

           // Assert
           Assert.IsNotNull(result);
           Assert.AreEqual("Please provide a valid postcode", result.ViewData.ModelState["CurrentPostcode"].Errors[0].ErrorMessage);

       }

To

[TestMethod, Isolated]
public void ApplicationValidation_CurrentPostcodeNumbersOnly_ErrorMessage()
{

           // Arrange
           HomeController controller = GetHomeController();
          
FormCollection form = GetEmptyApplicationFormCollection();
           form.Set("CurrentPostcode", "12345");

           // Act
           ViewResult result = controller.Application(form) as ViewResult;

           // Assert
           Assert.IsNotNull(result);
           Assert.AreEqual("Please provide a valid postcode", result.ViewData.ModelState["CurrentPostcode"].Errors[0].ErrorMessage);

       }

where the GetEmptyApplicationFormCollection() helper method just creates a FormCollection with all the forms fields.

Once this was done my unit test passed.

Summary

So I now have an MVC3 application that works and passes unit tests. You could argue I should do more work so it does not need these special fixes, but it meets my needs for now.

Why Typemock Isolator does not work on TFS 2012 Build and what you can do about it

If you are using Typemock Isolator in your unit testing then you be wanting to include then in your automated build process. Unlike other mocking frameworks you have to do a bit of work to achieve this with Isolator, this is because to enable its advanced features of mocking items like sealed private classes you have to start the interceptor that does the magic prior to running the tests and the switch it off afterwards.

In the past I posted on how you could use a build activity I wrote to wrapper MStest in a TFS 2010 build using TMockRunner. Since I wrote this Typemock released a set of build activities to do the starting and stopping of the inceptor as separate activities, a much more flexible solution which I would always recommend for TFS.

However when you try to use either with TFS 2012 you get a problem, the activities fail to load. This is a problem we saw on the TFS Extensions Codeplex project; you have to build you activities against either the 2010 TFS API or the 2012 TFS API. You don’t need to alter the code in your activities, but you do need to make a specific build.

So at this time there is no solution, one or both of these activities need to be rebuilt. For the MSTest wrapper I wrote the source is available so you can do it yourself if you want to, but the way Typemock have implemented their activities is a better solution. This is because it is not reliant on TMockRunner and MStest, it can wrapper any activities. This is important as to be able to use the new ‘any framework’ unit testing features in VS2012 you want to use Microsoft’s new test running activities and not just the old MSTest activity.

I understand that Typemock are looking at releasing a TFS2010 version of their activities soon, but I know of no release date as yet. If you want an immediate solution you will need to do a bit of work.

  • You could rebuild my MSTest based activity
  • You could use the standard InvokeMethod activity and put the contents of my MStest activity’s generated command line into this
  • But the one I favour is to use a dissembler such as Telerik JustDecompile to get the code from the Typemock.TFS2010.DLL and build a new activity.

However, it must be said I see this as just a temporary measure until the official Typemock 2012 activity is released. I am not sure I will get around to doing this before the Typemock release, we shall see.

DDD South West session on Unit testing in VS11

Thanks to everyone who attended my session at DDDSW today. The session was completely demo driven so no slides to share, but the contents of the session is covered in the blog posts

More on using the VS11 fake library to fake out SharePoint

I recently posted on how you could use the new fakes tools in VS11 to fake out SharePoint for testing purposes. I received comments on how I could make my Shim logic easier to read so though I would revisit the post. This led me down a bit of a complex trail, and to Pete Provost for pointing the way out!

When I did the previous post I had used SP2007, this was because I was comparing using Microsoft Fakes with a similar sample I had written ages ago for Typemock Isolator. There was no real plan to this choice, it was just what had to hand at the time. This time I decided to use SP2010, this was the process used that actually worked (more on my mistakes later) …

  1. Using a Windows 7 PC that did not have SP2010 installed, I created a new C# Class Library project in VS11 Beta
  2. I added a reference to Microsoft.SharePoint.DLL (this was referenced from a local folder that contained all the DLLs from the SP2010 14 hive and also the GAC)
  3. THIS IS THE IMPORTANT BIT – I changed the project to target .NET 4.0 not the default 4.5. Now, I could have changed to .NET 3.5 which is what SP2010 targets, but this would mean I could not use MSTest as, since VS2010, this has targeted .NET 4.0. I could of course have changed to another testing framework that can target .NET 3.5, such as nUnit, as discussed in my previous post in the VS11 test Runner.
  4. You can now right click on the Microsoft.SharePoint.DLL reference and ‘add fakes assembly’. A warning here, adding this reference is a bit slow, it took well over a minute on my PC. If you look in the VS Output windows you see a message the process is starting then nothing until it finishes, be patient, you only have to do it once! I understand that you can edit the .fakes XML file to reduce the scope of what is faked, this might help reduce the generation time. I have not experimented here yet.
  5. You should now see a new reference to the Microsoft.SharePoint.14.0.0.0.Fakes.DLL. and you can start to write your tests

image

So why did I get lost? Well before I changed the targeted framework, I had tried to keep adding extra references to DLLs that were referenced by the DLL I was trying to fake, just as mentioned in my previous post. This went on and on adding many SharePoint and supporting DLLs, and I still ended up with errors and no Microsoft.SharePoint.14.0.0.0.Fakes.DLL. In fact this is a really bad way to try to get out of the problem as it does  not help and you get strange warnings and errors about failures in faking the are not important or relevant e.g.

“\ShimTest\obj\Debug\Fakes\msp\f.csproj" (default target) (1) ->1>  (CoreCompile target) –> “\ShimTest\f.cs (279923,32): error CS0544: ‘Microsoft.SharePoint.ApplicationPages.WebControls.Fakes.StubAjaxCalendarView.ItemType': cannot override because ‘Microsoft.SharePoint.WebControls.SPCalendarBase.ItemType’ is not a property

The key here is that you must be targeting a framework that the thing your are trying to fake targets. For SP2010 this should really be .NET 3.5 but you seem to get away .NET 4.0 but 4.5 is certainly a step too far. If you have the wrong framework you can end up in this chain of added dependency references that you don’t need and are confusing at best and maybe causing the errors nor fixing them. In my case it seem a reference to Microsoft.SharePoint.Library.DLL stops everything working, even if you then switch to the correct framework. When all is working you don’t need to add the dependant references this is all resolved behind the scenes, not by me adding then explicitly.

So once I had my new clean project, with the correct framework targeted and just the right assemblies referenced and faked I could write my tests, so now to experiment a bit more.

Fix for problem faking two SPLists in a single unit test with Typemock Isolator has been released

A blogged a while ago about a problem faking multiple SPList with Typemock Isolator in a single test. With the release of Typemock Isolator 7.0.4.0 you no longer have to use the workaround I documented.

You can now use the code if the originally planned, and it works as expected

   1: public partial class TestPage : System.Web.UI.Page
2: {
3: public TestPage()
4: {
5:  var fakeWeb = Isolate.Fake.Instance<SPWeb>();

7: Isolate.WhenCalled(() => SPControl.GetContextWeb(null)).WillReturn(fakeWeb);
8: 
9: // return value for 1st call
10: Isolate.WhenCalled(() => fakeWeb.Lists["Centre Locations"].Items).WillReturnCollectionValuesOf(CreateCentreList());
11: // return value for all other calls
12: Isolate.WhenCalled(() => fakeWeb.Lists["Map Zoom Areas"].Items).WillReturnCollectionValuesOf(CreateZoomAreaList());
13: }
14: 
15: private static List<SPListItem> CreateZoomAreaList()
16: {
17: var fakeZoomAreas = new List<SPListItem>();
18: fakeZoomAreas.Add(CreateZoomAreaSPListItem("London", 51.49275, -0.137722222, 2, 14));
19: return fakeZoomAreas;
20: }
21: 
22: private static List<SPListItem> CreateCentreList()
23: {
24: var fakeSites = new List<SPListItem>();
25: fakeSites.Add(CreateCentreSPListItem("Aberdeen ", "1 The Road, Aberdeen ", "Aberdeen@test.com", "www.Aberdeen.test.com", "1111", "2222", 57.13994444, -2.113333333));
26: fakeSites.Add(CreateCentreSPListItem("Altrincham ", "1 The Road, Altrincham ", "Altrincham@test.com", "www.Altrincham.test.com", "3333", "4444", 53.38977778, -2.349916667));
27: return fakeSites;
28: }
29: 
30: private static SPListItem CreateCentreSPListItem(string title, string address, string email, string url, string telephone, string fax, double lat, double lng)
31: {
32: var fakeItem = Isolate.Fake.Instance<SPListItem>();
33: Isolate.WhenCalled(() => fakeItem["Title"]).WillReturn(title);
34: Isolate.WhenCalled(() => fakeItem["Address"]).WillReturn(address);
35: Isolate.WhenCalled(() => fakeItem["Email Address"]).WillReturn(email);
36: Isolate.WhenCalled(() => fakeItem["Site URL"]).WillReturn(url);
37: Isolate.WhenCalled(() => fakeItem["Telephone"]).WillReturn(telephone);
38: Isolate.WhenCalled(() => fakeItem["Fax"]).WillReturn(fax);
39: Isolate.WhenCalled(() => fakeItem["Latitude"]).WillReturn(lat.ToString());
40: Isolate.WhenCalled(() => fakeItem["Longitude"]).WillReturn(lng.ToString());
41: return fakeItem;
42: }
43: 
44: private static SPListItem CreateZoomAreaSPListItem(string areaName, double lat, double lng, double radius, int zoom)
45: {
46: var fakeItem = Isolate.Fake.Instance<SPListItem>();
47: Isolate.WhenCalled(() => fakeItem["Title"]).WillReturn(areaName);
48: Isolate.WhenCalled(() => fakeItem["Latitude"]).WillReturn(lat.ToString());
49: Isolate.WhenCalled(() => fakeItem["Longitude"]).WillReturn(lng.ToString());
50: Isolate.WhenCalled(() => fakeItem["Radius"]).WillReturn(radius.ToString());
51: Isolate.WhenCalled(() => fakeItem["Zoom"]).WillReturn(zoom.ToString());
52: return fakeItem;
53: }
54: 
55: }


A check of the returned values shows



  • web.Lists["Centre Locations"].Items.Count returns 2
  • web.Lists["Map Zoom Areas"].Items.Count) returns 1