Implement custom Claim based Authorization in ASP.NET MVC Web Application

Introduction

Claim-based authorization is a new model of authorization introduced in Windows Communication Foundation. This model addresses more scenarios than the popular role based  security model (IIdentity, IPrincipal). This is useful when an application requires complex and fine grained control on expressing access control decisions. Role based security model may not be powerful or flexible enough and is often too coarse when we reach complex scenarios – where custom roles are often necessary to represent different combinations of permissions or rights. For example – if you wanted to build a simple patient list page and wanted to have access control on CRUD operations based on roles you will end up creating several roles to support different combinations of access rights i.e. Administration clerks can only Add and Edit a patient but not Delete. Where the Supervisor can Add, Edit and Delete Patients. However you may want to restrict a Supervisor from certain access rights and then – end up creating custom roles depending on particular use cases. Claim based authorization framework attempts to address these limitations by providing a more generic model capable of addressing heterogeneous credential types, the ability to specify authorization policies through configuration at deployment time, and the flexibility to express arbitrary access control decisions based on the available evidence. .NET Framework starting in version 3.0 introduced a new identity and access-control API and can be found in the System.IdentityModel assembly and namespace.

Some of the terms and concepts that we need to get familiar with when we talk about claim based authorization – are: Claim, ClaimSet and IAuthorizationPolicy.

Claim : A claim is a piece of information that describes an individual right or action applicable to a particular resource. A claim consists of three pieces of information: a claim type, resources and right. The datastructure is represented by a class System.IdentityModel.Claims.Clam.

ClaimSet: A claim set is an enumeration of claims with a common issuer.

IAuthorizationPolicy: Authorization policyis an extensiblility point which allows new claim sets to be added to the context representing the web service or application. Claims are added to an EvaluationContext by authorization policies.

For more information please check “A Guide to Claims-Based Identity and Access Control” and “Building a Claims-Based Security Model in WCF”.

Claims are flexible and can be assigned based on authenticated users or authenticated user’s roles, although claims are guaranteed by a trusted issuer, which has added security benefits. The association between claim and the resources to which it grants rights seems to be more useful as it rarely changes, and the rules for how users and roles are assigned claims can freely change without impacting any authorization logics. For example one can design – Create, Read, Edit, Delete claims for Patients and can assign them to preferred list of users or roles freely.

FictitiousHospital.com

Today we will look at building custom Claim based Authorization in a ASP.NET MVC application. For this discussion I will keep the application requirements simple – we will build a simple patient list page that lists inpatients and outpatients – however when an user is not authenticated the patient list page do not display the user the options to create, edit or delete patients.

fictitioushospital

Fig 1 : FictitiousHospital.com

To build this application we will use the ASP.NET Membership and FormsAuthentication – to authenticate users. Then we will implement a custom authorization policy (that implements the type IAuthorizationPolicy ) to serve a “Issuer” and the “set of claims” that can be attached to the securitycontext. Finally we will hook up the claim based security model within the ASP.NET MVC by extending different parts of the MVC Framework.

Authentication

The project template of ASP.NET MVC Web Application ships with some basic pages for authentication, e.g. Login, Registration, Change Password etc. including the web.config file configured with ASP.NET Membership provider and FormsAuthentication. This is handy as the application is already configured and ready to use with Authentication in place. Just a change of connectionstring that points to the correct aspnetdb database makes us ready to start Authenticating the users and we can start concentrate on the next step – Authorization.

webconfig

Fig 2: web.config

formauthentication

Fig 3: FormsAuthenticationService

 

Authorization

The first thing to identify is to what types of claims make sense for our fictitioushospital.com website. Lets think of claims as permissions that are required to access the features. If we consider patients as resources we may have (CRUD) permissions like Patient Create, Patient View, Patient Edit and Patient Delete. So the possible list of claims may be:

1. Patient Create

Right: http://schemas.xmlsoap.org/ws/2005/05/identity/right/possessproperty

Claim Type: http://schemas.fictitioushospital.com/2010/02/identity/claims/create

Resources: http://schemas.fictitioushospital.com/2010/02/identity/resources/patients

2. Patient View

Right: http://schemas.xmlsoap.org/ws/2005/05/identity/right/possessproperty

Claim Type: http://schemas.fictitioushospital.com/2010/02/identity/claims/read

Resources: http://schemas.fictitioushospital.com/2010/02/identity/resources/patients

3. Patient Edit

Right: http://schemas.xmlsoap.org/ws/2005/05/identity/right/possessproperty

Claim Type: http://schemas.fictitioushospital.com/2010/02/identity/claims/update

Resources: http://schemas.fictitioushospital.com/2010/02/identity/resources/patients

4. Patient Delete

Right: http://schemas.xmlsoap.org/ws/2005/05/identity/right/possessproperty

Claim Type: http://schemas.fictitioushospital.com/2010/02/identity/claims/delete

Resources: http://schemas.fictitioushospital.com/2010/02/identity/resources/patients

To work with these claims in code we define list of representative constants as follows:

resources

Fig 4: Resources

claimtypes
Fig 5: ClaimTypes

We have already established our requirements for claims-based authorization for our fictitioushospital.com and now we are going to write an implementation of type System.IdentityModel.Policy.IAuthorizationPolicy interface (i.e. CustomAuthorizationPolicy). This interface requires us to implement properties, Id of type string, Issuer of type ClaimSet and the Evaluate() method that returns bool.

IAuthorizationPolicy

Fig 6: IAuthorizationPolicy interface

CustomAuthorizationPolicy

Fig 7: CustomAuthorizationPolicy

The CustomAuthorizationPolicy implements necessary interface. The Id property returns an unique identifier. The issuer property returns ClaimSet describing the issuer associated with this authorization policy, note that we have used issuerName = http://www.fictitioushospital.com/2010/02/issuer here. Finally the Evaluate() method that is the most important part of this implementation is responsible to inspect the claims based on the credentials provided.

evaluate

Fig 6: Evaluate() method

Here we have written some code that decides to attach Create, Read, Update and Delete claims to the evaluationContext when the user is authenticated otherwise only attaches the Read claim. We can implement different other ways to figure out the associated claims for the particular User.Identity e.g. based on roles or based on predefined access rights that is stored in database or xml.

The hard part is quite done – now we have to hook up this CustomAuthorizationPolicy into our ASP.NET MVC pipeline to start performing claim based authorization. Step 1: to do this we start by defining our custom IUser interface that extends IPrincipal interface and wraps an extra property named “Issuer” of type ClaimSet within, that is intended to carry the Issuer details returned from the CustomAuthorizationPolicy.

IUser

Fig 8: IUser interface

Step 2: we implement the User class of our custom IUser interface.

User

Fig 9: User

Step 3: we implement a custom authorization filter of type System.Web.Mvc.IAuthorizationFilter interface. OnAuthorization() we instantiate our custom User (that implements IPrincipal) using the information obtained from “filterContext.HttpContext.User.Identity”. Then we create a list of AutorizationPolicy and add our CustomAuthorizationPolicy to create a default instance of AuthorizationContext (authContext), this under the hood goes and call the Evaluate() method of CustomAuthorizationPolicy and populates the authContext.ClaimSets. Then we iterate through the authContext.ClaimSets and locate the issuer that we are interested in and assign the issuer to our custom User object. We then replace the HttpContext.User and Thread.CurrentPrincipal with our custom User. Please also note that we keep the customUser in Session() to avoid repeatative calls to retrieve the claimSet on every single request. A ClaimSet may be retrieved from database and this may be an expensive call to perform on every single request, populating it once and storing it in session seems feasible.

AuthorizationFilter

Fig 10: AuthorizationFilter

Step 4: we add this custom AuthorizationFilter in the controllers AuthorizationFilters list  by extending the ControllerActionInvoker.

CustomControllerActionInvoker

Fig 11: CustomControllerActionInvoker

The default implementation of IActionInvoker is ControllerActionInvoker and does quite a bit behind the scene. Firstly it obtains parameter values for the parameters on the action methods using the ModelBinder and then it executes various filters. There are different types of filters:

Authorization Filter – IAuthorizationFilter interface
Action Filter – IActionFilter interface
Result Filter – IResultFilter interface
Exception Filter – IExceptionFilter interface

and out of them Authorization filter is the first set of filters to be executed. This is where our custom AuthorizationFilter will get executed and that will replace the IPrincipal user identity with our custom custom User that we discussed above.

Step 5: we extend the DefaultControllerFactory and implement CustomControllerFactory, that assigns the controller.ActionInvoder to an instance of CustomControllerActionInvoker instead of  the ControllerActionInvoker.

CustomControllerFactory

Fig 12. CustomControllerFactory

Step 6: we tell our Mvc ControllerBuilder.Current.SetControllerFactory() to use the extended CustomControllerFactory insead of the default DefaultControllerFactory, and we normally do this once during the Application_Start as following.

application_start

Fig 13: Application_Start

Step 7: finally we are ready to check for claims during our Action invocation. Consider the PatientController, Index() action that returns list of patients. However this checks whether callers are granted to the Create, Edit and Delete role using the issuer.ContainsClaim() method and populates the PatientViewModel s CanCreate, CanEdit and CanDelete property, and the viewModel object is passed to the View.

PatientList

Fig 14: PatientController Index() action.

Later on by checking the PatientViewModels properties in the View one can easily decide on what links to display or not.

PatientListView

Fig 15: View

I have mentioned earlier that the association between claim and resources are highly unlike to change frequently. Consider this example where the Create() action demands the PatientClaims.PatientCreate Claim before this allows to create a new Patient, and similar with Update() action where it demands the PatientClaims.PatientUpdate claim. And thus the rules for how users and roles are assigned claims can freely change without impacting any the authorization logics.

PatientCreate_Update

Fig 16 Create, Update example

Claims-Based Permission Demands

You can take this further and implement Claims-Based Permission Demands, thus you will be able to write something like this:

[CreatePatientsClaim(SecurityAction.Demand)]
public void Create()

{
    //protected code
}

[UpdatePatientsClaim(SecurityAction.Demand)]
public void Update()

{
    //protected code
}

[DeletePatientsClaim(SecurityAction.Demand)]
public void Delete()

{
    //protected code
}

Permission demands are implemented with the PrincipalPermission or PrincipalPermissionAttribute and relies on the security principal attached to the executing thread to authorize calls. The idea is that when the permission demand is invoked the security principal attached to the request thread checks necessary values. Like with the PrincipalPermission type, you can write your own custom implementation of ClaimsPrincipalPermission that implements IPermission, ISecurityEncodable and IUnrestrictedPermission interface. This article “Building a Claims-Based Security Model in WCF – Part 2” by Michele Leroux Bustamente explains this in great detail.

 

Conclusion

Here we looked at how to implement claims-based security model in an ASP.NET MVC application. So how all this works in ASP.NET MVC according to the above implementation is,

  1. OnApplication_Start we SetContollerFactory to use CustomControllerFactory.
  2. CustomControllerFactory assigns instance of CustomControllerActionInvoker to the controller.ActionInvoker property of every controller instances.
  3. CustomControllerActionInvoker adds our AuthorizationFilter to the controllers AuthorizationFilters list.
  4. OnAuthorization of AuthorizationFilter is always invoked as the first thing during “InvokeAction()”. This calls the CreateDefaultAuthorizationContext() passing the list of  IAuthorizationPolicies, i.e. the CustomAuthorizationPolicy. The CustomAuthorizationPolicy instantiates itself with an unique Id.
  5. The Evaluate method of the CustomAuthorizationPolicy is called and that retrieves the ClaimSet for particular issuer for a particular user.
  6. A custom user is created of type User (that implements IUser –> IPrincipal ). This custom user is assigned to the HttpContext.User and Thread.CurrentPrincipal.
  7. The controller gets the custom user ( of type User – that implements IUser –> IPrincipal ) in its context.
  8. The action methods decides on access rights using the User.Issuer.ContainClaims() method.

Claim based authorization model is not to replace the role based authorization model and in many cases role based authorization may turn out to be more than sufficient. Claim based authorization models definitely supports more complex scenarios and allows fine grained control on expressing access control decision. Thank you for being with me so far and I hope that this discussion will help you in some way during your implementation of claims based security model. Happy coding!

Use IoC slash Dependency Injection framework to manage hierarchical object structure better

Introduction

Recently I have been working with some hierarchical object structures that are a composition of several other nested classes and elements, that we call “template objects”. To give you an idea of this, consider “Patient” class as a container that holds patient demographics in a hierarchical object structure.

hierarchical-template
Fig: hierarchical template.

 

These template objects are very robust and flexible, someone can go and design a template like above in a hierarchical structure with unlimited nesting and we have a tool that goes and generates these template objects automatically for us. e.g.

NestedClass
Fig: nested template objects.


Problem

I have been developing some web based CRUD interface pages with these template objects / container classes. However I faced some problem with the hierarchical nature of the template objects. Speaking of the above template as an example, two major problems that slowed down the development are

1. If I wanted to populate current address of the Patient class I had to instantiate several nested classes of the template object and then finally was able to assign the value to the desired  address property. e.g.

problem1

2. If I wanted to display the “current address” of the Patient to the page, I had to go through all sorts of null checking on several layers of the hierarchical object structure.

problem2 

 

As you can realize, that any kind of get and set operation to a nested object turns into a big task for the developers and the more deeper nested the object is in the hierarchy the more codes has to be written to deal with it.

Before I suggest a solution to this problem , lets look at different ways to maintain/represent hierarchical objects in C#.NET.

General class

This is the most common approach that we take, we create different dependent classes in separate physical file under same namespace.

general-class

Fig: classes in separate physical files.

However when we auto generate classes from templates there is high chances of class name conflicts. To overcome that scenario we can take one of the following approaches: Namespace separated class or Nested class.

Namespace separated class

In this approach every class can be maintained in separate physical files or all related classes can be dumped in one single physical file. However the key difference of this pattern to the above is every single class gets its own different namespace that also resembles the hierarchy. I made up the  Namespace conventions to be the parent class name suffixed by the string “Ns”.

patient-class 

patient-demographics-class 

patient-address-class

Fig: namespace separated classes.

When classes are stored in separate physical files, there is potential of physical filename conflict of the generated classes. This is easy to solve though, the related classes are sometimes grouped and stored in separate folders.

Nested class

The other approach is to create nested classes like we have seen above in “Fig : nested template objects”. Normally when one class is entirely dependent upon another, we decide to implement it using a nested class. Nested classes are declared within the scope of an existing class and receive special benefits when accessing private and protected members of their parent class.

Inversion of control (IoC) slash Dependency Injection 

Speaking about the problem that we identified earlier, here are some thoughts to solve the issues. I have some experiences with Inversion of control (IoC) slash Dependency Injection on test driven developments and I am aware the IoC frameworks provides simplified object creation, especially for hierarchical object structures and dependencies. I looked into this further to come up with a solution. For those who are new to DI and IoC, I would recommend you to read Martin Fowlers article : “Inversion of Control Containers and Dependency Injection pattern”.

In fact any of the following conditions justifies using the DI pattern.

  • You want to decouple your classes from their dependencies so that these dependencies can be replaced or updated with minimal or no changes to your classes’ source code.
  • You want to be able to write classes that depend on classes whose concrete implementation is not known at compile time.
  • You want to be able to test your classes in isolation, without using the dependencies.
  • You want to decouple your classes from being responsible for locating and managing the lifetime of dependencies.

    source:
    http://msdn.microsoft.com/en-us/library/cc707845.aspx

There are lots of IoC containers out there and some of the popular ones are StructureMap, Unity, Ninject, AutoFac, Castle Windsor and Spring.NET.

 

Solution

So by using the DI, we can inverse the dependencies on the template object classes and make them less coupled. e.g. by getting the PatienDemographics injected as a constructor parameter on the Patient class we can inverse the dependency and now Patient class depends on someone else to pass in the dependency. We can decouple and make this more flexible by declaring an interface for PatientDemographics and then pass the IPatientDemographics instead, something like this.

refactored-class

Fig: refactored Patient class.

We continue to inverse the dependencies for all other template objects in the hierarchy e.g.

refactored-tdo-class

Fig : refactored classes.

The next thing to do, is to register all our dependencies in IoC container by passing the container an interface that the dependency implements, and the concrete class that will be instantiated for request of that interface. In other words during the registration we tell the IoC container what and how it should build or find or serve requested services. I will register my dependencies using StructuredMap here.

structuremap-registry

Fig : structuremap registry.

By default, as long as an object is being created by invoking its constructor function, StructureMap will try to create/resolve/find an object for non-primitive dependency in the requested concrete type. This is known as autowiring, IoC will take care of instantiating all necessary dependent objects automatically. If StructureMap doesn’t know how to find a requested dependency it throws exception. This solves our problem, here is how:

1. If I wanted to populate current address of the Patient class I do not need to worry about instantiatiating several nested classes of the template object, because that will be magically handled by the IoC and I will be able to assign the value to the desired  address property with one line of code like the following.

solution1

2. If I wanted to display the “current address” of the Patient to the page, I do not need to go through all sorts of null checking on several layers of the hierarchical object structure, because IoC will automatically instantiate all dependent objects. I can go to the desired property directly and display the value when available.

solution2

You can see now clearly, how inversing dependencies and use of IoC container improves developers productivity significantly.

Before I finish I must also introduce the Common Service Locator library that contains a shared interface for service location which application and framework developers can reference. The library provides an abstraction over Ioc containters and service locators, which helps developers to use IoC without tying them down to a specific implementation. Currently service locator adapter implementations are available for StructureMap, Unity, Castle Windsor, Sprint.NET, Autofact, MEF and Linfu. The full list can be found here.

I hope this discussion will solve some programming nightmare with hierarchical object structure. Thank you for being with me so far and happy coding.

ASP.NET MVC, ASP.NET Tips: Provide immediate feedback to the browser on Long Running Tasks – using IFrame and web handlers

Download to download the source code please click here


The <iframe> tag defines an inline frame that contains another document and is supported in all major browsers.
Internet Explorer Firefox Opera Google Chrome Safari

The <iframe> can be placed anywhere in the document flow. The iframe properties such as  height and width can be easily modified by CSS and JavaScript. Data can be passed back and forth between the parent container and iframe using Javasrcipt. The document that is displayed inside an iframe can access the main window by the parent keyword. Iframe can have their own source URL that is distinct from the parent page and we are interested on this feature. We first declare an Iframe in the page and keep it hidden. We then point the Iframe to our preferred web handler, during page load iframe initiate a request to the server and the server start the lengthy task, and flushes state change information. The server flushes the response wrapped inside <script> tag to perform DOM manipulation on the parent container – similar technique that we have seen in one of my post earlier. Here are the steps to follow.

  1. Declare Iframe in the page.
  2. Hide the Iframe.
  3. Point the src of the Iframe to the web handler.
  4. Get Iframe to initiate request from the the browser client to the web server when the page loads.
  5. Get the web handler to flush response on task state changes.
  6. Perform DOM manipulation on the parent container.


 Iframe

Here I have declared two javascript functions in the parent page – “updateContent” and “hide”. “UpdateContent” replaces the data on “ReportDiv” element. “Hide” hides the loader image when lengthy task ends. Then down below on the same page I have declared the iframe tag, pointed the src of the iframe to a regular web handler “JsWrappedDataHandler.ashx”, and used the style elements to hide the iframe. Lets look at the web handler and what is cooking there.

jswrappedhandler

This is a regular web handler and you may be wondering why am I sending 256 dummy bytes. This is a workaround to make IE work properly because IE accepts a message from the server after the first 256 bytes.

This technique is handy when we do not know about the message size and we can avoid any unexpected event by sending 256 dummy bytes before sending the – message.

Next thing to look at is the script block – where it invokes the javascript functions that we have defined in the parent container. Example "window.parent.updateContent", "window.parent.hide".

Finally the server flushes the response directly to the browser. The Iframe receives the progressive flushed response chunk/script block and executes the javascript code – which invokes the javascript functions from the parent container and updates the content of of “ReportDiv” element until the task completes.

The end user gets immediate feedback on task state changes and do not need to wait until the entire response download. When the task is complete it invokes the "hide" function to hide the loader image. This is what it looks like.


 iframefeedback

 

Asynchronous web handlers

I discussed a little bit about asynchronous web handlers  in earlier post and I promised that I will talk about it later – so I do so. It is recommended to use asynchronous web handlers for long/ lengthy tasks. An asyncchronous web handler implements IHttpAsyncHanddler – this means it implements two additional methods BeginProcessRequest and EndProcessRequest. The idea here is to spawn custom threads by hand to run the lengthy portion of the task and not block important ASP.NET work threads from the threadpool. I have discussed about why we should spawn an additional thread in great length in this post, and I am not going to go through that again here, I am jumping straight in to the implentation of Asynchronous web handler.

IframeAsyncPage

This page is exactly same as the first one in this post where I declared 2 javascript function in the parent container such as “undateContent” and “hide”. The only difference is the iframe is now pointing to a different web handler and this time an asynchronous handler.

jswrappedAsynchandler

Here this asynchronous web handler implements the IHttpAsyncHandler. The BeginProcessRequest performs all the lengthy bits inside request.BeginProcessRequest method asynchronously and signals when the task is completed. This executes the EndProcessRequest method. The AsycWebHandlerRequest class is a custom hand written class and I paste the code below.

AsyncwebhandlerRequest

Here I spawn custom Threads by hand to invoke the LongTask method. Long task has exact same code as we have seen in the regular web handler ProcessRequest method above, except the last line where it signals that the task is complete by invoking "requestState.CompleteRequest" method. Here the Response.Flush method does all the hard work to notify the connected browser about the task state changes and end user can enjoy immediate feedback from the server. You may be wondering about the AsyncRequestState class. I have used the exact same class during implementation of the Asynchrounous Page and can be found here.

I have indicated earlier that a web handlers is recommended for performance critical web pages. I have done some load tests to compare the result of regular web handler, asynchronous web handler and the asynchronous page that pretty much does the exact same operation. The test result of load tests that I have performed in exact same environment can be found below.

IframeAsync

Fig: Load test of Iframe and asychronous web handler that uses custom thread

IframeRegular

Fig: Load test of Iframe and regular web handler

WebPageAsync

Fig: Load test of asynchronous web page that implements custom thread

You can see that the asynchronous web handler that spawns custom thread is the clear winner. And when you compare Asynchronous page with web handler – we see web handler is superior.

Please note that the different techniques that I have been discussing  in last several posts  is also known as Comet that describe the web application model in which a long-held HTTP request allows a web server to push data to a browser, without the browser expicitly requesting it. However holding a connection open for infinite period has negative impact on browser usability. The problem is the javascript will keep accumulating and the page will have to preserve them in the page itself, resulting in a unbound page size and memory implications – at worst the page becomes totally unresponsive and will crash the browser. These is ok to choose for long task that has a definite end, however we need to be a bit careful when it comes to a “real-time” applications that keeps running for indefinite period. For example an airlines time table,

For a “real-time” application ( i.e. chat application, airline timetable ) a polling / long polling mechanism is more suitable where the server keeps looping for infinite period – produces the latest content and store it in a persistent layer – example a database/file, or in the application memory. The browser polls the latest response periodically, performs DOM manipulation to updates the page – and the user gets to see the latest record. Lets look at the polling techniques in my next post.

ASP.NET MVC, ASP.NET Tips: Provide immediate feedback to the browser on Long Running Tasks – using XMLHttpRequest call

XMLHttpRequest (XHR) is a DOM API that can be used inside a web browser scripting language, such as JavaScript, to send an HTTP or an HTTPS request directly to a web server and load the server response data directly back into the scripting language. The technique that we are going to discuss here takes advantage of XMLHttpRequest objects readyState == 3 status (Interactive – Downloading) and responseText property to perform DOM manipulation in a browser. Here is how it works

  1. A request is initiated from the the browser client to the web server using the XMLHttpRequest object.
  2. The webserver flushes response during task state changes.
  3. XMLHttpRequest object holds partial data in responseText property when readyState is 3 ( Interactive – Downloading ).
  4. DOM manipulation is perfomed during readystate == 3 to interactively update the web content.

 

Download to download the source code please click here

 

Please note that XMLHttpRequest based technique will not work in all browsers. Microsoft clearly mentions in the specification that – “readyState =  3, some data has been received. responseText is not available. responseBody is not available.” So this will not work in IE. However I found it working in Firefox and Chrome. Here is the code.

XmlHttpRequest

Here I have created a XmlHttpRequest object and configured it to perform a GET request to my DataHandler.ashx that I have created earlier. The piece of code that does the magic to notify the browser on state changes is the part where I am updating the content of “ReportDiv” element with req.responseText when req.readyState == 3. The responseText dynamically loads the flushed response and this piece of code updates the content of the browser. Then finally I hide the loader image when the req.readyState turns 4 that means – loaded, all the data has been received. This entire process is initiated by the getData() method which is invoked during the pageload even, but you can do the same on onclick of a button or anchor tag or by raising some other event that suits your need. For a refresher on what is happening inside DataHandler.ashx here is the web handler codes.

RegularHandler

The web handler simulates a long running task by putting the Thread to sleep. However it keeps flushing the response when the browser is connected to the server, until the task is complete. Please check my earlier post where I have done detailed discussion on Response.IsClientConnected property, Response.Flush method etc.

You may be wondering that this is still not the perfect solution that we are looking for as responseText is not available during readyState==3 in all browsers, we cannot do much about this however we can implement the IFrame Call that works in most of the browsers. Lets look at that.

ASP.NET MVC, ASP.NET Tips: Provide feedback to the user on Long Running Tasks – using loader image – MS AJAX and JQuery

It is quite common to display a loader-image or some kind of progress bar image in a browser when the server is busy serving the request. Before we go deep into page streaming techniques with XMLHTTPRequest call and IFrame, lets look into a simple loader icon implementation – the idea is to display “loading” image until the task completes, display the response in the browser when task ends and hide the loader image.

loading

Download to download the source code please click here

 

Invoke a long task and display loader image to notify task progress – using MS AJAX

Normally in a ASP.NET page we drop the ScriptManager control, put a reference of the webservice that we want to point to. The Scriptmanager goes and creates necessary proxy class for the webmethods. We can also add static methods to an ASP.NET page and qualify them as web methods and invoke them using javascript. This tutorial shows how to do all these by configuring the ScriptManager easily. However I will play with the Sys.Net.WebRequest class directly here. The Sys.Net.WebRequest provides the script API to make Web request. It becomes handy when we need to partially update a little section of the page and it is quite straight forward to use the Web Request class anyway. 

 

msajax

It may look like a lots of code above but when you look closely you find that all I have done is setup a get request to asynchronously invoke my DataHandler.ashx web handler that I created earlier. During page load the “PageLoad” method is called and this initiates the entire process. You can also do the same on the onclick event of an input button or anchor tag or by raising some other events that you prefer. The image that I used for the loader has been generated from this website Ajaxload.info,  this website has some nice loader image designs available. You can choose your indicator, background color, foreground color and generate your preferred loader. The webrequest invokes the web handler and the loader image is displayed until the full response is available. I populate the “ReportDiv” element with the completed response data, and hide the loader image with loader.style.display = “none”. There is also some codes above that handles timeout and abort situations.

 

Invoke a long task and display loader image to notify task progress – using JQuery

The JQuery based implementation of the same above is even more simpler and is done with a lot less coding.

Jquery

All I do here is I configure HTTP GET request call to my DataHandler.ashx inside the “$(document).ready” function -this function is infact the first thing to learn about jQuery if you are new to JQuery. If you want an event to work on your page, you should call it inside the $(document).ready() function. Everything inside it will load as soon as the DOM is loaded and before the page contents are loaded. If you have not worked with JQuery yet I would strongly recommend to have a look at it. The getData() function makes the HTTP GET call to the web handler and on success I put the response to the “ReportDiv” element. Then I hide the loader icon with style display = “none”;

Thats pretty much it, however this is not a robust solution and I am writing all the post in this series to render realtime states of a task in the browser. When we use this technique all we can display is the loader image to notify users that server is busy processing his/her request – we cannot display any real time update on the tasks state changes to the browser. The MS AJAX and JQuery web request calls do not allow to check XMLHttpRequest s readyState==3 status (will discuss about XMLHttpRequest object in the next post ). The web applications are stateless and request can only be initiated from the client, however XMLHttpRequest or a IFrame call can initiate a parallel request to the server and we can take advantage of that to stream contents to the browser. Lets look at them.

ASP.NET MVC, ASP.NET Tips: Provide immediate feedback to the browser on Long Running Tasks – in Asynchronous ASP.NET pages

Download to download the source code please click here

In the earlier post we have discussed little bit about the limited number of worker threads available in the thread pool of ASP.NET and we identified why we should always try to avoid to block those important worker threads. ASP.NET 2.0 introduces Asyn=”true” attribute in the page to handle asynchronous scenarios and under the hood this attribute tells ASP.NET to implement IHttpAsyncHandler in the page. There are more than one ways available in ASP.NET 2.0 and ASP.NET 2.0 (plus) to register asynchronous tasks in the page,

  • by AddOnPreRenderCompleteAsync method.
  • by declaring PageAsynTask tasks and register them by RegisterAsyncTask method.


However PageAsyncTask provides some extra flexibility over AddOnPreRenderCompleteAsync method, for example

  • it offers a timeout option.
  • allows to register more than one async task.
  • allows to configure async tasks to be executed in parallel or sequentially
  • RegisterAsyncTask passes HttpContext.Current to the End and Timeout methods
  • RegisterAsyncTask allows to pass object state to the Begin methods.

Lets look at how we can implement an asynchronous page to serve our scenario that we are focusing here, that is, to run long tasks asynchronously and at the same time provide immediate feedback to the browser. 

AsyncPageAspx
Fig: aspx page

AsyncPage

Fig: code behind

Here in the above code block I have left the AddOnPreRenderCompleteAsync code commented which can be used alternative to PageAsyncTask object & RegisterAsyncTask method. I have registered a PageAsyncTask task using RegisterAsyncTask method.  The “task” object is configured to point to – BeginAsyncOperation, EndAsyncOperation and TimeoutAsyncOperation functions. The BeginAsyncOperation does all the lengthy bits inside request.BeginProcessRequest() method and signals when completed, this executes the EndAsyncOperation method. However if the lengthy task is not completed within the timeout period, the TimeoutAsyncOperation executes.  Lets look at the AsyncRequest class now.

 AsyncRequest

The most important thing to note here in the above piece of code is, I spawn an additional custom thread by hand and I execute the lengthy bits i.e. the “Process” Method in the custom thread. The Process method simulates the lengthy task by putting the thread to sleep. However note that the Response.Flush() does the magic to notify the connected browser about the task state immediately. It is a good idea to always check the Response.IsClientConnected property we discussed this in length in one of our earlier post.

As soon as the lengthy task completes I call the CompleteRequest method of the requestState object.

 

AsyncRequestState
The CompleteRequest singnals that the asynchronous task is complete and it invokes the the callback method if available. In this example the callback invokes EndAsyncOperation method that we have configured in our PageAsyncTask object.

You may be wondering why I had to spawn a custom thread to run the lengthy operation where we can simply use asynchronous delegate invocation. You must know that Delegate.Invoke consumes worker thread from the ASP.NET thread pool. Even though the page releases a worker thread into the pool but immediately that thread ( or another worker thread ) is again taken from the thread pool. Now this precious thread is occupied until the long task finishes hence you gain nothing rather add overhead of switching thread from the pool. 

Fritz Onion in his popular article “Use Threads and Build Asynchronous Handlers in Your Server-Side Web Code” mentioned the same, “Asynchronous delegate invocation completely defeats the purpose of building an asynchronous handler because it draws from the same process-wide CLR thread pool that ASP.NET uses to service requests. While the primary request thread is indeed returned to the thread pool, another thread is taken out to perform the asynchronous delegate execution, which means that there is a net gain of zero threads for servicing additional requests, rendering the handler useless. You will run into the same problem if you use ThreadPool.QueueUserWorkItem to perform the asynchronous request processing because it also draws threads from the same process-wide CLR thread pool”.

Jeff Prosise also quoted him in his article by saying “Asynchronous delegates, are counterproductive in ASP.NET applications because they either steal threads from the thread pool or risk unconstrained thread growth. A proper asynchronous page implementation uses a custom thread pool, and custom thread pool classes are not trivial to write.

He also warned about it in his conclusion “A final point to keep in mind as you build asynchronous pages is that you should not launch asynchronous operations that borrow from the same thread pool that ASP.NET uses. For example, calling ThreadPool.QueueUserWorkItem at a page’s asynchronous point is counterproductive because that method draws from the thread pool, resulting in a net gain of zero threads for processing requests”.

After all these warnings if you were still implementing the frequently recommended asynchronous delegation invocation ( unfortunately, most of the google search result on this topic of asynchronous programming in ASP.NET will lead to an implementation of asynchronous delegation ) it would look something like this:

asyncdelegate

I do not want to discuss about delegate invocation because it completely defeats the purpose of implementing Asynchronous pages. I have done some load testing on the above two implementations and found the following result:

testresultdelegateinvocation

Fig: Load test result of asynchronous handlers with a delegate.

 

testresultthread

Fig: Load test result of asynchronous handler with custom thread.

Clearly we see that we gain significantly on asynchronous handler that uses custom thread in both cases,

  • requests/sec_total and
  • avg. Respons_Total.


I got distracted a bit from our original subject matter again, but it was worth doing this one. I have already discussed here how we can run long running task asynchronously and how to provide immediate feedback to the connected browser immediately. In this post and previous posts I was implementing solutions by taking advantage of ASP.NET page and web handlers however we can also give immediate feedback to clients using MS AJAX, JQuery, XMLHTTPRequest, IFrame etc. in ASP.NET pages. Lets look at them one by one.

ASP.NET MVC, ASP.NET Tips: Provide immediate feedback to the browser on Long Running Tasks – by streaming regular ASP.NET page

Here in this series of post I am discussing few techniques to give client instant feedback on task progress in ASP.NET page. In the previous post I have discussed about displaying task progress – using regular handler, here I will discuss how to do the same in standard asp.net page.

Download to download the source code please click here

Display task progress – in standard asp.net page

In this technique we write some regular ASP.NET script in the page itself, that performs streaming to the original page response directly. We put the lengthy task related code in the page and keep flushing the response when task state changes. The data is pushed to the the pages outgoing stream and the browser immediately displays the output. The connection to the browser is kept open until the task is complete.

PageStream

 

You see above I have written a regular ASP.NET page with some C# scripts that simulated a long running task by forcing the thread to sleep. However before the thread goes to sleep it Flushes the response using Response.Flush method.  I discussed about the Flush method in earlier post. It basically facilitates to send buffered output to the browser immediately. The other thing to note here is the the Response.IsClientConnected property which indicates the server whether the browser is still connected or not. The Response.IsClientConnected property can determine if the browser is still connected. So when we perform a lengthy operation it is a good idea to periodically test  whether the browser is still connected.

You may have already noticed that I am writing embedded javascript codes that manipulates the DOM. In this case this the ReportDiv elements innerHTML is assigned to latest flushed response. However to keep this example simple, I have been flushing the “report” string over and over again to reflect the changes in the browser and the same data is sent across the wire over and over. This may not be a good idea when you are flushing out a large report with 1000 records. You can reduce the amount of response traffic just by sending the progressive latest content only. To do this you will need to change your DOM manipulation code to perform append instead of replacement. Something like below.

PageStreamOptimized

Note that I have changed the codes to flush progressing latest content only, I have also modified the rendered javascript codes that is now appending latest flushed content to the existing content instead of replacement.

The above example is serving the long task in a regular synchronous manner however It is a good idea to implement long running tasks asynchronously, lets look at that in the next post.

ASP.NET MVC, ASP.NET Tips: Provide immediate feedback to the browser on Long Running Tasks – using regular web handler

As an ASP.NET developer we frequently deal with long running tasks for example:

  • Making long-lasting webservice calls.
  • Making prolonged calls to database that runs complex queries.
  • Do lengthy file operation
  • Calling remote objects etc.

When the server is busy performing the lengthy time consuming task/tasks, the poor user has to wait for the response. The most common technique used to interact with user is to display a loader icon or show some kind of progress bar on the page until the task is complete. However we can provide user immediate feedback of the task progress/state while the task is running in the Server and make the page more interactive/informative.

 

 iframefeedback

Here we will discuss several techniques to give end user instant feedback on the browser about task progress in  ASP.NET pages.

such as display:

 

Download


Display task progress using regular web handler:

This is very easy to code, all we need to do is write a regular web handler that runs the long running task and we have to keep flushing the response when state changes, in other words we decide to flush when the task reaches certain stage. The data is pushed to the the outgoing stream and the browser immediately displays the output. The connection to the browser is kept open until the task is complete.

RegularHandler

We have created a regular web handler that inherits from IHttpHandler and simulated a long running task by forcing the thread to sleep. However before the thread goes to sleep it Flushes the response using Response.Flush() method. The Flush method sends buffered output immediately. Normally all of the response is sent to the browser only when page completes its processing. This may be a issue for a long running task as the end user will not see anything until the page is complete. The Flush method comes handy in this situation and can send buffered output periodically to the browser. Imagine you will have to display 500 records and you can flush the buffer every 20 records until it is complete. The good thing of doing this is – user can see some data immediately, visualize the progress of the report and the server memory is also released on every Flush operation.

One more thing to note is the Response.IsClientConnected property which indicates the server whether the browser is still connected or has already left.  A very common scenario is user starts the long expensive task but decided to close the browser before the task is complete. When a browser requests the page from the server but does not wait for the entire page to be downloaded, the server continues to process the request, wasting CPU cycles. The Response.IsClientConnected property can determine if the browser is still connected or not. So when we perform a long task it is always a good idea to periodically test  whether the browser is still connected or not.

UseHandler 
A web handler file works just like a aspx page and can be directly browsed via a browser. The web handler above, returns response type text/html and we have pointed an anchor tag to the web handler. Please note a web handler is recommended for a performance critical web page.

I have started with a regular synchronous web handler, however it is recommended to use asynchronous web handlers that implements IHttpAsyncHandler for long running tasks. ASP.NET has limited number of threads in the thread pool and a normal synchronous page holds on to the thread for the duration of the request, and this thread is prevented to be used to process any other request during that period. As a result the thread gets stuck until the lengthy request is complete. If all the threads are stuck doing lengthy tasks the subsequent additional requests starts to queue up and when the queue fill up ASP.NET starts spitting 503 “Server Unavailable” errors.

ASP.NET provides a neater solution to this problem via Asynchrounous web handlers that implements the IHtttpAsyncHandler. The interface suggests to implement two additional methods BeginProcessRequest method and  EndProcessRequest. The idea is to spawn  an additional thread by hand in response to BeginProcessRequest, then perform the lengthy task in this additional thread and notify when the task is complete so the worker thread can return a response to the user.  We will look at Asynchronous web handlers in a later post in more details. Lets continue to look at other techniques now.

Microsoft WebsiteSpark – Receive Windows Web Server and SQL Server Web Edition at no cost to host new websites

Now help is on the way in the form of Microsoft WebsiteSpark, a program announced today that will empower Web site design and Web application development firms with 10 or fewer employees. WebsiteSpark provides development tools, production licenses for server products, technical support, business development support, and access to the expertise and services of Microsoft’s community of partners and hosters.

visit http://www.microsoft.com/web/websitespark to learn more

ASP.NET, ASP.NET MVC Tips: JSON Handler, Aspx template, Asynchronous JQuery treeview with ashx generic handler and many more.

In my previous blog post, I have discussed about how to get asynchronous JQuery treevew to work with ASP.NET Webservices, and we identified that, to make a successful ASP.NET webservice call using JQuery the request must be POST request, the content type of the request must be “application/json; charset=utf-8”, and the data parameter of the the $.ajax() method must be passed as a string. We also looked at the tweaks that we need to do to the jquery.treeview.async.js file to get that working.

However I found the JQuery, $.ajax() method can consume JSON objects straightway from the ASP.NET Generic Handler (ashx) that can be designed to serve a JSON response. If you are using ASP.NET MVC framework you will find that $.ajax() can also consume JSON response without hassle from MVC controller actions that returns a JsonResult. I will demonstrate both scenario with my FictitiousWebsite.com.

fictitiouswebsite

Fig: FictitiousWebsite.com

Lets look at the ASP.NET MVC Fictitious website application first,

ASP.NET MVC Application

  1. I have put together two classes (TreeService, TheTreeNode) that generates story categories and subcategories as seen in the screenshot above – [categories: “.NET, ASP.NET MVC, AJAX, ADO.NET”, subcategories: “Latest articles, Forum post”]. The TreeService class returns story catetories and subcategories as List<TheTreeNode>.

    TreeService
     
  2. Then I have designed an MVC controller action to return JsonResult. This serves the Jquery treeview control with the list of story categories and subcategories as JSON response.

    ControllerAction
    I have used “d” variable here to replicate the security feature that has been added to ASP.NET 3.5 JSON serialization, you will notice the JSON response is encapsulated within a parent object to address a nasty vulnerability. You can read more about this vulnerability that Phil Haack has discussed in this post “Json Hijacking”. Encapsulating the JSON response results the following response .

    JsonResponse  
  3. The I have created a View – “LatestStories.aspx” that hosts the JQuery treeview control.

    ViewPage  

    Note that , the treeview control is communicating to my MVC controller discusssed in previous step, through this URL, “/Home/GetStoryList”. You will also notice that I have decoraded a fileClick event for the treeView. When user clicks the subcategories, it asynchronously/on demand gets the relevant list of stories from this URL “/Home/Stories” and renders to the right side panel. This fileclick event do not ship with treeview out of the box, I had to inject 2 lines of code to the original “jquery.treeview.async.js” file to get this happening.

    asyncjs

  4. Finally I decorated a aspx template – “Stories.aspx” that serves the necessary html content for the right side panel.

    template
    I prefer using templates as aspx files, as I can take full advantage of the intellisense and design support of the Visual Studio. Plus I have the entire ASP.NET MVC framework to back me up to produce and render dynamic contents on demand.

It is very straight forward to consume JSON response in the ASP.NET MVC framework, however the same is not as easy when we deal with ASP.NET Webservices as discussed in my previous post. ASP.NET Webservices is not the only way we can serve JSON response, lets look at how we can design a ASP.NET Generic Handler (ashx) to serve the same. Plus you will also find in a moment that ashx can be more easily consumed by the JQuery $.ajax() method.

ASP.NET 3.5 Website

I have created an exact replica of the FictitiousWebsite.com in an ASP.NET 3.5 website project. This website project has exact same files as the ASP.NET MVC application, with very minor differences.

 

  1. Here look at the ASP.NET solution explorer snap to get a quick rundown of the similarities and the differences.

    aspnet
  2. The (TreeService.cs, TheTreeNode.cs) files are exactly same as before and generates list of story categories and subcategories.
  3. The ASP.NET Generic Handler (ashx) file “GetStoryListJsonHandler.ashx” serves JSON response to the Jquery treeview control with the list of story categories and subcategories.

    GenericHandlerJson

    You may already recall that we have written similar code in the MVC controller that was invoking the TreeService and  returning JsonResult. I have decorated this Generic Handler (ashx) in the similar way. Note I have defined the ContentType = “application/json”, and serialized the data with JavaScriptSerializer. I have also used the “d” variable to replicate the security feature that has been added to ASP.NET 3.5 JSON serialization as before. This results exactly same output as we have seen in the MVC framework earlier. The best part is JQuery $.ajax() method can consume this without any hassle as we have faced earlier to communicate to the ASP.NET Webservice method.

    ashxoutput
  4. The “LatestStories.aspx” hosts the treeview control, the only difference you will note here is it now communicates with the ASP.NET Generic Handler (ashx) file, and the fileClick communicates to the “Stories.aspx” template.

    latestStories 
  5. The “Stories.aspx” file (template) serves the necessary html content to display the list of stories to the right side panel. This page is now backed up by the ASP.NET web forms framework, WYSIWYG designer support in Visual Studio and allows to render dynamic contents easily.

Conclusion

We have discussed, how we can serve JSON response in both ASP.NET MVC and ASP.NET 3.5 website framework. We have identified that consuming a JSON response from ASP.NET Webservice requires some extra steps in the JQuery world, we have demonstrated an alternative approach to serve JSON response using Generic Handler (ashx) and found it easier to consume via JQuery $.ajax() method. While designing the JSON response we used “d” variable to replicate the security feature that has been added to ASP.NET 3.5 JSON serialization. We also looked at rendering dynamic content asynchronously on demand, and identified the advantage of aspx templates as it can be designed using the WYSIWYG designer, intellisense, compilation, dubug, sourceview, splitview, codebehind and all the features of Visual studio and the the ASP.NET framework.

The ficitiouswebsite.com demonstrates everything that we have discussed above and is available to

Thank you for being with me so far, Happy Coding.

Just another Microsoft MVPs site