SharePoint WCM in the finance sector

You might have been wondering where I’ve been for the last few weeks, so I thought I’d show rather than just tell. I rarely talk about specific ‘daytime’ projects I work on, but I guess I’m quite proud of this one and you might be interested in what we’ve done. I’ve been working with LBi (Europe’s biggest design/technology agency) recently, leading a team of 5-7 developers on a WCM (Web Content Management) project for one of the global investment banks. You can see some aspects of what we’ve done with SharePoint in this post, and I’ll follow up with some developer-to-developer info about techniques we used in the next one. Additionally, for UK-based folks I think Riaz Ahmed (my current boss and Mr SharePoint at LBi) is hoping to do a real show and tell at the UK SharePoint user group in a couple of months, where there’ll be the opportunity for Q&A and to pick up far more info/lessons learnt (both technical and non-technical) than I can provide here.

The site is targeted at a closed audience over the internet (our client’s clients), so unfortunately you can’t see it for yourself – the site provides analysts with high-value information on financial markets, primarily in the form of market reports and newsflashes, though there are other content types. The user experience we were tasked with implementing (created by LBi’s design team) means the user gets to have some pretty impressive functionality around all this (IMHO) which differentiates the service from others.

The centrepiece is the market report data, which is held for many countries and topics:

MarketReport_b

This is imported nightly from a master datasource (thus lending itself nicely to caching), and our client can also add comments to the imported data via SharePoint WCM page editing. Perhaps the slickest piece of functionality is the facility for the user to create personalized ‘smart reports’ with specific markets and topics which interest him/her. These are saved against the user’s profile for quick access:

ConfigureSmartReportSmall_b

At runtime we build the personalized page from the same cache layer used by the standard pages in the first shot.

Since financial markets change quickly, our client’s authoring teams (distributed around the world), can quickly create newsflashes via SharePoint publishing functionality for display on the site – this is controlled via SharePoint workflow. Users can search and filter by country, date, high priority flag and can also store in a ‘My newsflashes’ bucket for quick retrieval:

NewsflashesSmall_b

Users also have the option of receiving e-mail alerts when newsflashes are added to the site (immediate for high priority newsflashes, nightly batch for others), and can tailor their e-mail preferences by country…

MyAccountCountryPreferencesSmall_b

..and topic:

MyAccountMailPreferencesSmall_b

The site also has a ton of other good stuff, such as:

  • single sign-on with a sister site
  • integration with a higher-level portal which can authenticate users itself (meaning users do not log-in separately on our site)
  • WCAG ‘A’ -level accessibility – accessibility wasn’t particularly a focus of the client for this site, but (happily) is just how things are done here anyhow. We use a lot of JavaScript to enhance the user experience but it always provides equivalent functionality when disabled – I have to say the LBi front-end developers use some incredibly innovative techniques to achieve this, I’ve rarely seen anything like it (see below for some examples).
  • integration with Endeca for search (including the newsflash landing page you see above – this is powered by search)
  • custom admin functionality for user creation/management
  • many many smaller but ‘interesting’ requirements such as newsflashes having user-selected ‘related items’, attachments etc etc!

Rich, accessible controls

In addition to all the personalization, one of the things that I love about the user experience is the funky controls we have. So why have a dull dropdown like this:

StandardDropdown

…when you can have one like this:

CountryControlExpandedMid 

This allows us to present only countries the user has told us they are interested in in the dropdown (thus avoiding clutter), whilst allowing them to easily see data for other countries if they have the occasional need to. To do this, clicking the ‘More…’ link expands the dropdown to show a second section containing the other countries (or more specifically, the other countries they are permitted to see):

CountryControlExpandedFull

Similarly for multiple selections, why have this:

StandardListbox

…when you can have this:

CountryControlMultiExpandedMid 

Again, clicking the ‘More…’ link allows the user to select a country not in their ‘preferred’ countries:

CountryControlMultiExpandedFull

Personally I think this makes for a great user experience, and it’s all accessibility-compliant.

Project plan : caffeine

The development time for everything you see here (and everything you don’t) was around 7-8 weeks, though this excludes some of the front-end development (HTML, CSS, JS). We’re not quite over the finishing line yet, but are in the final stages of user acceptance testing and security testing. To say it’s been tough would be something of an understatement, but I’m incredibly proud of my guys and it’s been a great team effort – fortunately for me they’re very talented 🙂

Next time I’ll detail some of the technical decisions we took and go through some developer bits which we felt worked well when building/deploying the site.

SPDataSource – every developer’s friend (part 2)

SPDataSource – a refresher

So last time in part 1 of this 2-part series, we saw how SPDataSource is a great option for fetching data from lists since not only does it do the actual work of fetching the items for you, it also does this without any C#/VB.Net code. We can retrieve all the items from the list, or optionally supply a CAML query along with the list details if we wish to filter/sort the items. So as I showed in part 1, we can easily display a dropdown containing the items from a list with the following markup:

<SPWebControls:SPDataSource runat="server" ID="dsPersonTitles" DataSourceMode="List" 
SelectCommand="<Query><OrderBy><FieldRef Name='SortOrder' Ascending='true' /></OrderBy></Query>"
<SelectParameters>
<asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
<asp:Parameter Name="ListName" DefaultValue="PersonTitles" />
</SelectParameters>
</SPWebControls:SPDataSource>

<asp:DropDownList runat="server" ID="ddlPersonTitles" CssClass="title" DataSourceID="dsPersonTitles" DataTextField="Title" DataValueField="ID">
</asp:DropDownList>

We also saw how SPDataSource offers more than just retrieving items from a list, with the following other ‘modes’ (see part 1 for more details on these):

  • CrossList – similar to doing a query with SPSiteDataQuery across all lists in a site collection
  • ListItem – show field values from a single list item
  • Webs – lists all webs in a site collection
  • ListOfLists – lists all lists in a web

What I want to focus on this time is how to use parameters with SPDataSource, since when using the control for real you’ll often want to do this.

Parameters with SPDataSource

SPDataSource works like other .Net data source controls, so the key to passing parameters is using one of the .Net 2.0 parameter classes, either declaratively (in markup) or in code. The example above shows using the basic Parameter class by setting the DefaultValue property to a known string, but the following parameter types can also be used (though note I haven’t tried them all and the SPD Team Blog article I mentioned says somewhat vaguely that "most" of them can be used!):

Effectively using one of these saves you writing code to read the value from the respective location and passing it to SPDataSource yourself. I think that ControlParameter and QueryStringParameter are possibly of the most value, though all could be interesting depending on your requirements. As an example, to get the value a user selected earlier from a dropdown containing a list of lists and pass it to a SPDataSource control I would need:

<SelectParameters>
<asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
    <asp:ControlParameter Name="ListName" ControlID="ddlLists"
PropertyName="SelectedValue"/>
</SelectParameters>

I seemed to have some control execution lifecycle fun with ControlParameter but it did work in the end. To pull the value from a ‘userID’ querystring parameter I would need:

<SelectParameters>
<asp:QueryStringParameter Name="userId" QueryStringField="userId" />
</SelectParameters>

And that’s not all – I can also drop a parameter value into a string such as a CAML query used in the earlier example of ‘List’ mode:

<SPWebControls:SPDataSource runat="server" ID="dsPersonTitles" DataSourceMode="List"
SelectCommand="<Query><OrderBy><FieldRef Name='{SortField}' Ascending='{SortAsc}' /></OrderBy></Query>">
<SelectParameters>
<asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
<asp:Parameter Name="ListName" DefaultValue="PersonTitles" />
<asp:QueryStringParameter Name="SortField" QueryStringField="SortField" DefaultValue="SortOrder" />
<asp:QueryStringParameter Name="SortAsc" QueryStringField="SortAsc" DefaultValue="true" />
</SelectParameters>
</SPWebControls:SPDataSource>

Notice here I’m dynamically specifying the sort field and direction from querystring parameters. Cool stuff.

Even cooler is the idea of building your own parameter control – one that strikes me is one for retrieving values from a user’s SharePoint profile (since the ProfileParameter listed above refers to ASP.Net profiles). Scott Mitchell has a great guide to this at Accessing and Updating Data in ASP.NET 2.0: Creating Custom Parameter Controls.

Summary

So we’ve covered a lot of ground there, so here’s a recap of why the SPDataSource is your friend:

  • Can bind to any control which can do data-binding – DropdownList, ListBox, CheckBoxList are some obvious candidates, but consider also Repeater, DataGrid and SharePoint DataView/SPGridView controls
  • Can easily get parameters from other controls, querystring, session, form values etc. (or write some code to fetch from another location)
  • Can use a variety of modes to make different queries e.g. items in a list, properties of a list item, webs in a site etc.

If you want to see more I recommend reading SPDataSource and Rollups with the Data View – this emphasises using SPDataSource with SharePoint DataViews but also has some good examples I haven’t covered, such as using the ‘CrossList’ mode.

SPDataSource – every SharePoint developer’s friend (part 1)

Sooner or later, nearly every SharePoint developer needs to write code to retrieve all the items from a list and display them – either on the page, or often in a control like a dropdown list control. An example could be retrieving a list of countries, or perhaps person titles for a form:

DropdownFromList

Clearly there are benefits from storing such values in a list, since we (or the client) can then add/edit/delete items easily. To implement this, the developer’s first thought might be to write code like this:

private void bindPersonTitles()
{
// the name of the DropDownList control we are populating is ddlPersonTitles..
using (SPWeb configWeb = SPContext.Current.Site.AllWebs["configuration"])
{
SPList titlesList = configWeb.Lists["PersonTitles"];

foreach (SPListItem titleItem in titlesList.Items)
{
ddlPersonTitles.Items.Add(new ListItem(titleItem.Title, titleItem.ID.ToString()));
}
}
}

Which is fine. Except if we want to filter or sort the list items, we really should use a CAML query instead of iterating through all the items – the code below shows this (sorting on a column called ‘SortOrder’), and also has a slight twist on the previous example in that I’m using data-binding rather than looping through the items ‘manually’:

private void bindPersonTitlesByCamlQuery()
{
using (SPWeb configWeb = SPContext.Current.Site.AllWebs["configuration"])
{
SPList titlesList = configWeb.Lists["PersonTitles"];
SPQuery query = new SPQuery();
query.Query = "<OrderBy><FieldRef Name=\"SortOrder\" /></OrderBy>";
SPListItemCollection titlesItems = titlesList.GetItems(query);

// showing alternative of using data-binding rather iterating through items..
ddlPersonTitles.DataSource = titlesItems;
ddlPersonTitles.DataTextField = "Title";
ddlPersonTitles.DataValueField = "ID";
ddlPersonTitles.DataBind();
}
}

Fine again. Except I can’t help thinking "all this just to take the items from a list and put them in a dropdown??" Surely there must be a better way. And what did Microsoft do every time they needed to do this, surely they didn’t repeat the above code in every place in SharePoint? The answer is no, they used this:

SPDataSource

In short, SPDataSource is a web control which implements IDataSource and saves you writing code like the above. The great thing is that it is extremely flexible, and as we’ll see, can be used for more than you might think. The best thing though, is that being a control it can be used declaratively, so I can bind my dropdown to the list without writing a single line of C# or VB.Net code – all I have to do is set properties correctly. We’ll go through the detail in a second, but the markup to replace the last code sample would look like:

<SPWebControls:SPDataSource runat="server" ID="dsPersonTitles" DataSourceMode="List" 
SelectCommand="<Query><OrderBy><FieldRef Name='SortOrder' Ascending='true' /></OrderBy></Query>"
<SelectParameters>
<asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
<asp:Parameter Name="ListName" DefaultValue="PersonTitles" />
</SelectParameters>
</SPWebControls:SPDataSource>

<asp:DropDownList runat="server" ID="ddlPersonTitles" CssClass="title" DataSourceID="dsPersonTitles" DataTextField="Title" DataValueField="ID">
</asp:DropDownList>

So we’re doing the following:

  • setting the ‘DataSourceMode’ for the SPDataSource is set to ‘List’ – we’ll examine other possible values shortly
  • setting the ‘SelectCommand’ to the CAML query we want to use – here we’re just sorting on the ‘SortOrder’ field once more
  • telling the SPDataSource which list we want to use by supplying ASP.Net 2.0 parameter objects named ‘WebUrl’ and ‘ListName’ – we’ll dive more into this later
  • finally we bind the dropdown to the data by specifying the DataSourceID to the ID we gave our SPDataSource, and also say which fields in the resultset we want to use for the ‘Text’ and ‘Value’ of the dropdown. Incidentally I recommend using the ID for the value and the Title for the text (as shown above) so that it’s easy to create an SPLookupValue to update a list item – more on this in part 2

We’re only just starting to see the power of SPDataSource, but I love this approach for a couple of reasons:

  • Details of my query aren’t specified in compiled code, so if anything about the list changes (e.g. we restructure our site) I don’t have to recompile and redeploy assemblies
  • Less custom code, less bugs!
  • Setting properties is arguably simpler than writing code

So let’s dive deeper into what we can do – we’ll cover ‘modes’ of SPDataSource in this article and how to dynamically pass parameters to control the query in part 2.

The different ‘modes’ of SPDataSource

SPDataSource isn’t just limited to fetching the items from a list (DataSourceMode = ‘List’). Other possibilities are:

  • CrossList – similar to doing a query with SPSiteDataQuery across all lists in a site collection (for a sample of this see the SharePoint Designer Team blog post linked at the end of this article)
  • ListItem – show field values from a single list item
  • Webs – lists all webs in a site collection
  • ListOfLists – lists all lists in a web

For the last 2 modes I was able to bind but had some difficulty working out values to use for the ‘DataTextField’/’DataValueField’ of my control. I was trying to simply get the web or list name, but none of the obvious values such as ‘Title’, ‘ListName’ etc. were in the resultset. I was unable to find any other information on this but I’m sure some more trial and error would solve it. The ‘ListItem’ mode can be interesting – in the following sample I’m binding a single list item to a DataGrid to show selected fields from the item, which itself is selected by using the ‘ListItemID’ property:

<SPWebControls:SPDataSource runat="server" ID="dsPeople" DataSourceMode="ListItem" UseInternalName="true"> 
<SelectParameters>
<asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
<asp:Parameter Name="ListID" DefaultValue="34F91B0C-FCF2-455A-ABBA-67724FB4024A" />
<asp:Parameter Name="ListItemID" DefaultValue="1" />
</SelectParameters>
</SPWebControls:SPDataSource>

<asp:GridView ID="grdPeople" runat="server" DataSourceID="dsPeople"
AutoGenerateColumns="False">
<Columns>
<asp:BoundField DataField="FullName" HeaderText="Blogger name" />
<asp:BoundField DataField="WorkCity" HeaderText="City" />
<asp:BoundField DataField="Blog_x0020_URL" HeaderText="Blog URL" />
</Columns>
</asp:GridView>

This would give something like this, based on an underlying list item which has these fields (no formatting yet applied):

SPDataSource_ListItem_DataGrid

I found I had to specify UseInternalName = "true" and use the ListID rather than ListName parameter in this mode.

Next time – passing parameters to SPDataSource

So far we’ve looked at supplying parameters by using a standard ASP.Net parameter control and specifying the value in the DefaultValue’ property. In fact, ASP.Net has a whole range of parameter controls which can do the work of retrieving a parameter from somewhere and passing it to the SPDataSource, so that’s what we’ll look at next time. Additionally, since we’re often using SPDataSource to bind data to form controls, we’ll look at the common scenario of getting the selected item out of the form control to save back to a lookup field in SharePoint.

<updated>That link to the SPD blog article I mentioned but forgot to link to is http://blogs.msdn.com/sharepointdesigner/archive/2007/04/24/spdatasource-and-rollups-with-the-data-view.aspx – the focus here is mainly on using SPDataSource with a DataView, but there’s some great info and samples.</updated>

Disposing SharePoint objects – what they don’t tell you

If you’re a SharePoint developer and find yourself regularly writing code to work with SPListItem objects, here’s some info which might help you factor your code in a slightly neater way in some scenarios.  Interestingly this doesn’t get mentioned in any of the key papers/articles – I can think of a good reason for this (which we’ll look at), but in my mind this is a safe technique. If you have other ideas however, I’d love to hear from you.

Most developers are now fairly well-versed in the techniques needed to write efficient SharePoint code (key resources are listed at the end of the article), specifically in terms of disposing of any SPSite/SPWeb objects your code creates. The thing is, having to always be mindful of disposing these objects can sometimes restrict us in the way we want to write our code – consider the following simplified example:

public void DoSomething()
{
SPList list = getList();

// do something with list..
foreach (SPListItem item in list.Items)
{
processItem(item);
}

// ** PROBLEM - how do we now dispose of the SPSite/SPWeb objects we created earlier? **
}

private SPList getList()
{
// can't dispose of these objects here if we're returning a list - we'll be attempting to use
// objects which have already been disposed of..
SPSite site = new SPSite("http://cob.blog.dev");
SPWeb web = site.OpenWeb("/MyWeb");
SPList list = web.Lists["MyList"];

return list;
}

The problem here is we created our SPSite/SPWeb objects in a different method to where we need to dispose them – this can happen a lot when writing library code. To extend this example, here’s a pattern I regularly use to ensure I’m getting my objects the most efficient way – using SPContext if it’s present, but instantiating them myself if it isn’t (i.e. the code is called from an event receiver, console app etc.):

public void DoSomething()
{
bool bDisposalsRequired = false;

// get list from SPContext if we have one..
SPList list = getListFromContext();
if (list == null)
{
// otherwise get list from objects we create..
list = getInstantiatedList();
bDisposalsRequired = true;
}

// do something with list..
foreach (SPListItem item in list.Items)
{
processItem(item);
}

if (bDisposalsRequired)
{
// ** PROBLEM - how do we now dispose of the SPSite/SPWeb objects we created earlier? **
}
}

private SPList getInstantiatedList()
{
// can't dispose of these objects here if we're returning a list - we'll be attempting to use
// objects which have already been disposed of..
SPSite site = new SPSite("http://cob.blog.dev");
SPWeb web = site.OpenWeb("/MyWeb");
SPList list = web.Lists["MyList"];

return list;
}

private SPList getListFromContext()
{
SPContext currentContext = SPContext.Current;
SPList list = null;

if (currentContext != null)
{
list = currentContext.Site.AllWebs["MyWeb"].Lists["MyList"];
}

return list;
}

When the code starts to become more complex like this, we start to really want a simple way of disposing the objects we created elsewhere. I’d probably caution against this, but what happens if you created the SPSite/SPWeb objects not only in another method, but another class? All of a sudden disposing of objects correctly is a lot more complex than the examples given in the white papers. 

Now, the savvy SharePoint developer might think "Hold on, the SPList has a ParentWeb object (and SPListItem has a ParentList object!), what happens if I go up the hierarchy and dispose that way?" – code to this would look like:

if (bDisposalsRequired)
{
list.ParentWeb.Dispose();
list.ParentWeb.Site.Dispose();
}

Now how would we know the correct objects have been disposed? Are these objects the same objects we were using earlier? Or has SharePoint populated these properties with references to different objects, meaning we’ve left our original objects alive and could run into scalability problems?

Well, the good news is these are the same objects, so disposing in this way will dispose of the objects you created. We can tell this by comparing the objects using System.Object’s GetHashCode() method:

private void compareObjects()
{
SPSite site = new SPSite("http://cob.blog.dev");
SPWeb web = site.OpenWeb("/MyWeb");
SPList list = web.Lists["MyList"];

SPWeb parentWeb = list.ParentWeb;



// no message displayed..
Debug.Assert(parentWeb.GetHashCode() != web.GetHashCode(), "Objects are not same!");


// create a *new* SPWeb object and compare it to our other reference..
web = site.OpenWeb("/MyWeb");

// message IS displayed..
Debug.Assert(parentWeb.GetHashCode() != web.GetHashCode(), "Objects are not same!");
}

What we see from this is that the SPWeb object stored in the list.ParentWeb property has the same hash code as the original SPWeb object we created, but if we create a new instance of the same web, it has a different hash code. This indicates the objects in the first comparison are the same – so disposing of them through the ParentWeb reference will indeed dispose of the objects we created. So happy days – we’ve found we can factor our code in a way which may be more logical and readable.

The caveat

So why might this not be mentioned in any of the published guidance? Well, before using this technique throughout your code you should consider that in writing code in this way we are effectively relying on an internal implementation detail of the current SharePoint API. If Microsoft were to change this in a service pack (or the next version of SharePoint), our code may no longer dispose of objects correctly. I’d recommend giving this your own consideration (not least because you might conclude differently to me), but I’d suggest this change would be fairly unlikely. The reason for this is the SharePoint team themselves need to eliminate any unnecessary SPSite/SPWeb objects, so of course it makes sense that the the same SPWeb instance used to obtain the SPList is used to populate the ParentWeb variable. Creating an extra instance would make the API less efficient. On that basis, I personally am happy to write code in this way. If you think I’ve got this wrong (I can’t guarantee I haven’t), of course I’d love to hear from you.

In an upcoming article, I’ll also discuss the idea of going beyond the MSDN examples, and trying to write SharePoint data access code in a more ‘enterprise’ or ‘patterns’ kind of way. Stay tuned for more on this.

Disposing SharePoint objects – required reading

Code/Feature samples for common SharePoint development tasks

There are lots of tools available to help developers, but I always think the single most useful thing is actually seeing a good example of the thing I’m trying to develop. This is why SharePoint blogs are so popular right? As well as checking my favorite blogs or past articles, I’m always going back to previous work – perhaps to see what values I used in a certain SharePoint file or to grab a chunk of code I wrote which I can amend to fit my current requirement. So today I want to point you in the direction of some useful samples which could be good reference material if you’re a SharePoint developer – in actual fact I’m not releasing anything new here, but highlighting that my recent ‘Config Store’ solution provides a great "one-stop shop" of downloadable examples of several common SharePoint development tasks. Examples of the following dev tasks can all be found in the solution:

  • Deploy site columns, content types as Feature
  • Deploy a list (with default list items) as a Feature
  • Using Feature receivers
  • Using SPWebConfigModification to make web.config changes in code/as part of a Feature
  • Using SPQuery to efficiently find items in a list
  • Build a Solution package (.wsp) without WSP Builder
  • Write a Solution deployment script to retract Solution, deactivate Feature, redeploy Solution, activate Feature etc.
  • Using event receivers
  • As a .Net bonus, how to use the HttpRuntime class to work with ASP.Net caching outside of a web context

The entire set of code/Feature files can be downloaded from Codeplex, making it a great reference project containing samples of all the tasks listed above. If this is useful to you, the list below details the file to open to find the code sample, and some notes which might help you to get to grips with the task:

Deploy site columns, content types as Feature:

<table class="MsoNormalTable" style="margin-left: -0.75pt; width: 298pt; border-top-style: none; border-right-style: none;
border-left-style: none; border-collapse: collapse; border-bottom-style: none” cellspacing=”0″ cellpadding=”0″ width=”650″ border=”1″>

File

Notes

ConfigStoreElements.xml          

This is accomplished by use of the Field/ContentType Feature elements. Note these should be before any ListTemplate/ListInstance items in the same elements file – if using both sets of elements in different files, ensure the field/content type file is listed first in manifest.xml.

Deploy a list (with default list items) as a Feature:

File

Notes

ConfigStoreElements.xml          

This is accomplished by use of the ListTemplate/ListInstance Feature elements.

Feature receivers:

File

Notes

ConfigStoreFeatureReceiver.cs

In my case, the Feature receiver makes modifications to web.config and programmatically adds list event receivers to a list.

web.config modifications

File

Notes

ConfigStoreFeatureReceiver.cs

This is accomplished by use of the SPWebConfigModification class. I add three entries to the web.config to support my solution.

 

Using SPQuery to find items in a list:

File

Notes

ConfigStore.cs                                 

I have two methods in the ConfigStore class which perform queries in this way (also known as a CAML query).

 

Build a Solution package (.wsp) without WSPBuilder:

File

Notes

makecab.ddf/                                  
manifest.xml                          

Whilst WSPBuilder offers a great automated way to build Solutions, occasionally there is a need to step outside the tool to build the package ‘manually’. In my case, I had a problem where WSPBuilder was adding elements in the ‘wrong’ order to the generated manifest.xml file, and this was causing the Solution deployment to fail.

 

Solution deployment script to retract Solution, deactivate Feature etc:

File

Notes

COB.SharePoint.Utilities.ConfigStore_Install.bat      

This is the script I’ve often used to deploy my Solutions. It has error checking at each stage so I can easily tell at what point the install fails for simpler problem diagnosis.

 

Using event receivers:

File

Notes

ConfigStoreListEventReceiver.cs

In the Config Store solution I use a list event receiver to add items to the cache when the list contents change. This code shows using the ItemAdded and ItemUpdated events on the list.

 

Using the HttpRuntime class to work with ASP.Net caching outside of ASP.Net:

File

Notes

ConfigStoreListEventReceiver.cs

Something to remember with list event receivers is that we actually don’t have access to HttpContext.Current or SPContext.Current – both are null. If it wasn’t for the HttpRuntime class, this would be a problem for my Config Store code since I wanted to add/remove items from the cache in the list event receiver (i.e. manage the cache as soon as list items are changed). Fortunately, HttpRuntime allows us access to the ASP.Net framework from any code location (e.g. console app).

 

The link to download all this stuff is at http://www.codeplex.com/SPConfigStore. Hope you find the samples useful, feel free to leave a comment if anything doesn’t make sense.

Considerations when referencing assemblies in your page layouts

This one came up in the office last week, and I thought it worthy of discussion. On one project we have, the page layouts use the ASP.Net Register directive to reference assemblies containing our controls. So we have several directives in the code similar to:

<%@ Register TagPrefix="psw" Namespace="OurCompany.OurClient.SharePoint.WebControls" Assembly="Parity.SharePoint.WebControls, Version=1.0.0.0, Culture=neutral, PublicKeyToken=00000000000000" %>

These serve the purpose of telling the page layout about our controls assembly, and ensuring the designer can supply us with Intellisense for the controls and their members.

The question which then arises is "what happens when we update the shared assembly and we want to increment the version number? Surely we don’t have to update and republish all our page layouts?" There are a couple of answers to this:

  • we could use assembly redirects in application config to avoid changing the page layouts. These entries would then tell .Net to load (for example) version 2.0.0.0 of our assembly whenever version 1.0.0.0 is requested. This would work, but would mean that we don’t get Intellisense for any new/changed members added in subsequent assembly versions.
  • we could also avoid referencing common assemblies in this way completely, and centralize the reference in web.config instead

So instead of having an entry in each page layout, the ‘Pages’ section of web.config (in .Net 2.0 and upwards) allows us to reference assemblies containing controls in one central place, meaning any changes to the assembly name are simple to implement:

<pages>
<controls>
<add tagPrefix="psw" namespace="OurCompany.OurClient.SharePoint.WebControls"
assembly="OurCompany.OurClient.SharePoint.WebControls, Version=1.0.0.0, Culture=neutral, PublicKeyToken=00000000000000" />
<!-- also works with short assembly name -->
<!--
<add tagPrefix="psw" namespace="OurCompany.OurClient.SharePoint.WebControls"
assembly="OurCompany.OurClient.SharePoint.WebControls" />
-->
</controls>
</pages>

[Side note] – as I specify in the comment, it is possible to reference the assembly with the short name, but shops with a defined versioning strategy will want to avoid this since it’s then not possible to have side-by-side versions of assemblies, one of the fundamental forward steps introduced with .Net.

However, it’s not all clear cut. Unfortunately SPD isn’t clever enough to resolve the assembly reference in web.config to provide Intellisense. I initially thought it was, but alas closing and re-opening shows that in fact something was cached. Visual Studio is sufficiently aware (if we were in a pure .Net scenario), but not SPD.

So it’s a trade-off as far as I can see – either have the @Register directive in your page layout (with version numbers in it), or reference in web.config but lose Intellisense.

I wondered if it was possible to use the short assembly name in the @Register directive, but supply a 4 part assembly name in web.config for use at runtime. Unfortunately this doesn’t work since .Net sees it as an ambiguous registration. So if you want to keep assembly version numbers out of your page layouts but keep Intellisense, one strategy could be to remove the @Register directives as part of your release process.

Or as an alternative final thought – which is more important, Intellisense during development or minimising running into this problem at release time? For my money, losing the inconvenience of Intellisense in SPD for controls in is a minor hassle, so referencing assemblies in web.config is a better approach.

Great controls to be aware of when building SharePoint sites

Something I’ve been meaning to do for a while is discuss some of the controls I’ve found useful when putting together SharePoint sites. Obviously before building a custom control for a specific behaviour, it’s a good idea to check if SharePoint comes with anything that will do the job. It sometimes surprising what you find! Certainly those from a Content Management Server background will be familiar with the idea of having lots of reusable controls (which used the CMS API) within the team, but in MOSS some of the equivalent controls come for free. The most obvious is the Content by Query web part (though we won’t mention that the output HTML isn’t accessible [WCAG AA-compliant] without extra work), but some of the smaller controls deserve some attention too. So here’s a rundown of some handy items for the toolbox:

SPSecurityTrimmedControl

This control can be used to selectively display content or controls depending on the current user’s SharePoint permissions. Whatever is the inner content of the control will therefore not be shown if the user doesn’t have the specified permissions. An example would be:

   1: <SharepointWebControls:SPSecurityTrimmedControl runat="server" Permissions="ManageWeb">
   2:     This is only visible to users who can manage current web..    
   3: </SharepointWebControls:SPSecurityTrimmedControl> 

In addition to the Permissions property shown above (enum), the PermissionsString property can be used to specify a comma-separated list of required permissions. This can be used in conjunction with the PermissionsMode property which takes ‘All’ or ‘Any’, and also the PermissionContext property which specifies what SharePoint object (e.g. ‘CurrentList’, ‘CurrentFolder’, ‘CurrentItem’, ‘CurrentSite’, ‘RootSite’) the specified permission applies to, so it’s flexible. Examples of the permissions which can be specified include ‘ManageLists’, ‘AddListItems’, ‘ViewPages’, ‘ManagePermissions’ and so on. These are members of the SPBasePermission enum and Zac Smith has a full list. Notably many of these permissions are probably more useful in a collab scenario rather than WCM. [UPDATE: Text in this section updated following comment below/further testing:] For WCM folks the control does look like it does something useful in being able to filter on authentication type – unfortunately this doesn’t work as advertised. For reference (in case MS fix it), the usage is:

   1: <SharepointWebControls:SPSecurityTrimmedControl runat="server" AuthenticationRestrictions="AnonymousUsersOnly">
   2:     This *should* be visible only to anonymous users but it doesn't work..    
   3: </SharepointWebControls:SPSecurityTrimmedControl> 

As you’d expect, valid values here are ‘AnonymousUsersOnly’, ‘AuthenticatedUsersOnly’ and ‘AllUsers’ – however as noted in the comments below ‘AuthenticatedUsersOnly’ is the only flag which seems to work properly.

There are some other interesting properties too:

  • PagesMode – valid options are ‘Design’, ‘All’ and ‘Normal’
  • QueryStringParametersToInclude
  • RequiredFeatures

These look like extra properties to filter on, but alas I couldn’t get these to work either. My investigative powers let me down here as firing up Reflector onto both SPSecurityTrimmedControl and the related RightsSensitiveVisibilityHelper class didn’t give anything away in terms of usage, nor did Google or searching the 12 hive. Certainly it looks like specifying values is all that would be needed for latter two parameters, but I had no joy. A final consideration for SPSecurityTrimmedControl however, is the idea of further possibilities opened by deriving from this control in the same way SPLinkButton does.

EditModePanel

Where the previous control examined the user’s permissions to establish whether content should be shown, the EditModePanel looks at whether the current page is in display or edit mode. This can be incredibly useful in the WCM world for displaying help messages or other content to users as they edit a page. However there are other uses – hiding navigation, adding inline CSS override classes to use different formatting (particularly useful) and emitting debug information in the HTML output are all examples. The declaration for this control would look like:

   1: <PublishingWebControls:EditModePanel SuppressTag="false" GroupingText="Title help" 
   2:         PageDisplayMode="Edit" runat="server" id="EditModePanel1">
   3:     Page titles should be concise
   4: </PublishingWebControls:EditModePanel>

At run time in edit mode, this would look like:

EditModePanel

A prettier usage might be to include the field controls within each respective EditModelPanel, meaning the input control is shown within the borders of the box. To do this we’d also need an extra EditModePanel set to PageDisplayMode="Display" which also contained the field control, since the original will display in edit mode only. Note this control will output a surrounding <div> unless the SuppressTag property is set to ‘True’.

ListItemProperty

One control you might use in conjunction with the previous control is the ListItemProperty control. This simple little fella allows you to write out a particular field’s value for a list item. The field which is rendered is specified with the Property attribute, where we specify the field’s internal name. By default, the current list item (i.e. for the page we’re viewing) is used, but this can be overridden by specifying the List property – to do this we specify the list GUID with braces:

   1: <SharePointWebControls:ListItemProperty runat="server" id="ListItemProperty1" 
   2:     List="{C110296D-2BE9-4818-80EE-A06BD018DE2F}" ListItemID="1" Property="Title"/>

I think of this control as doing a .ToString() on the contents of the chosen field. This can sometimes be useful in WCM where you want the value to appear in a different location onthe page to where it is edited – for example within a <title> tag. Note we can also specify the version of the list item we want to retrieve the value for with the ListItemVersion property.

ListProperty

Much the same as ListItemProperty except on a list itself. These two may well be used together as the code in a list’s DispForm.aspx does:

   1: <SharePoint:ListProperty Property="LinkTitle" runat="server" id="ID_LinkTitle"/>
   2: : 
   3: <SharePoint:ListItemProperty id="ID_ItemProperty" MaxLength="40" runat="server"/>

This will show the current list’s title, followed by a colon, followed by the current list item’s title.

ProjectProperty

Slightly confusing name, but this control allows us to write out some properties of the current site/web. The MSDN documentation shows the valid list to be:

  • BlogCategoryTitle – Category of the current post item
  • BlogPostTitle – Title of the current post item
  • Description – Description of the current Web site
  • RecycleBinEnabled – 1 if the recycle bin is enabled; otherwise, 0
  • SiteOwnerName – User name of the owner for the current site collection
  • SiteUrl – Full URL of the current site collection
  • Title – Title of the current Web site
  • Url – Full URL of the current Web site

So the following would write out the title of the current web:

   1: <SharePointWebControls:ProjectProperty runat="server" id="ProjectProperty1" 
   2:     Property="Title" />

DelegateControl

This control provides an architecture where an ‘outer’ control is added to the page layout/master page, but the ‘inner’ control is determined by a Feature, thus allowing the control to be swapped out without having to modify the original template. This is used widely in Microsoft’s master pages. This is a great control, I discuss it more fully in my Using the DelegateControl post.

AssetUrlSelector

To finish on something slightly different, this control can be invaluable when developing custom web parts and field controls. It provides the ‘file picker’ experience authors are used to when working with SharePoint, and fits well when you have a control which requires a file to be selected. So in several of my controls I allow the user to override the XSL file used for formatting, and the AssetUrlSelector is added to the user control I use for the edit view of the control. This provides me with the ‘browse’ button which will launch the picker when clicked:

AssetBrowseButton 
Clicking the button then shows:

AssetUrlSelector 
The AssetUrlSelector provides a fairly good user experience, including opening the picker in the location of the currently-selected file if one exists, or defaulting to a specified default location if not. Along similar lines if you need to provide a slick experience to select an image is the RichImageSelector – this is the picker used by the RichImageField control.

Summary

So that’s some of the controls I’ve found useful. Obviously there are stacks more and the ones I did highlight can be used in lots of interesting ways which aren’t discussed here. But hopefully this does serve to remind you to check what’s in the box before spending time writing your own!

New version of Content Deployment Wizard released

I’m happy to announce that development/testing of the next version of the SharePoint Content Deployment Wizard is now complete, and the new release has been uploaded to Codeplex. If you’re not aware of the tool then Introducing the SharePoint Content Deployment Wizard is a good place to start. I’m calling this release "beta 2", which indicates the tool is by no means complete, but is significantly more mature than the original version. I’d encourage all of the 1400+ people (and counting) who downloaded the original in the past few weeks to get the latest version – the link is at the bottom of this post.

So let’s run through the improvements:

  • Better support for large sites:

    The treeview of the site is no longer built in one hit – instead, portions of the site are retrieved as webs/lists are expanded. This can make a huge difference when working with large sites.
  • Support for WSS-only sites:

    The dependency on the MOSS assembly (Microsoft.SharePoint.Publishing) has now been removed, so imports/exports to WSS sites are now possible. The MOSS-specific functionality is only used if we are indeed in a MOSS environment – if you’re interested, this is just related to honoring any publishing schedules (i.e. page expiry) from the source site on the target site.

  • Everything in one .exe:

    No separate WizardBase.dll any more.

  • Easier access to log files after import/export:

    Since it’s highly important to analyze the log file for errors/warnings after an import/export, these now open automatically in notepad at the end of the operation. There’s also a link in the tool to open the log file, so no more manual digging around the filesystem.
  • Auto-discovery of sites in ‘Site URL’ box:

    No need to re-enter the site URL each time – the sites available for selection in your SharePoint environment are automatically shown in the auto-complete entries for the textbox:

     AutoSiteDiscovery

  • Pretty icons in treeview 🙂

    Easier visual distinction between objects in the tree:
     TreeViewImages
  • Logging via System.Diagnostics.Trace:

    In addition to the log file generated by the actual import/export operation, general operations in the tool are now also logged (separately) via System.Diagnostics. So if you have a problem using the tool, it should be easier to pinpoint the actual error – enable by adding a .config file with a TraceSwitch entry for "COB.SharePoint.Utilities.ContentDeploymentWizard" and appropriate TraceListener entries.

  • More reliability:

    Miscellaneous minor bug fixes, including removal of the somewhat psychedelic flickering as the confirmation screen is built.

In terms of issues reported in the first release, it seems that 95% of them have been related to the underlying Content Migration API provided by SharePoint rather than the actual Wizard – this means the issue would also likely surface if out-of-the-box Content Deployment (via Central Admin) or STSADM -export was used to deploy the content instead. This is in line with what I was expecting since in many ways the Wizard doesn’t actually do that much – it just presents a nice UI (with the ability to cherry-pick exactly which items should be deployed) onto existing SharePoint code. Since there are some gotchas with the MS bits however, it does mean that understanding how their API works can help when using the Wizard. My recommended reading list for those wanting to know more would be:

The future

I’ve had great feedback so far (ranging from feature requests to "why the hell don’t you charge for this thing?!" to offers of collaboration). In terms of features, it looks like incremental deployment is the most popular vote so far so I’m hoping to add this fairly soon. Another possibility is making the Wizard scriptable by refactoring the code into something more tiered, thus providing features like the ability to save configurations for import/export jobs and also allowing them to be hooked up to a task scheduler in some way. 

However, the more feedback the better and if you’d like to see particular functionality or have run into problems trying to use the tool I’m definitely keen to hear. Either leave a comment on the blog or over at the Codeplex site.

The download link
 

Go to www.codeplex.com/SPDeploymentWizard and go to the releases tab to download the latest version. Hope you find it useful.

More workflow issues and resolutions

What I wanted to do today is run through is wrap up my series of workflow posts, by running through some of the issues I hit during development. If you’re into workflow, you may have seen Rob Bogue’s excellent 10 issues (and resolutions) for SharePoint+Workflow – this post is something of an extension to Rob’s. It probably makes for rather dry reading, but will hopefully be useful to those Googling for specific problems.

  1. Issue:- Error message “System.Xml.XmlAttribute is not serializable” shown in workflow history list and workflow will complete before it is supposed to. In SharePoint log, will see “End of Stream encountered before parsing was completed” error.

    More information:- This is caused by trying to store a class which cannot be serialized as a member (e.g. private variable) of the workflow class. Furthermore, I also encountered this error when storing the class which represents the InfoPath form – interestingly this class will always definitely be serializable, but the error was happening nevertheless.

    Resolution:- Mark member as NonSerializable (so workflow does not attempt to serialize the class, and copy primitive data (e.g. strings, dates etc.) out of form class and into individual field-level variables. It might also be possible to mark the ‘AnyAttr’ member of the generated class as NonSerializable instead.

  2. Issue:- InfoPath forms do not get updated upon solution deployment – changes are not reflected.

    More information:- This happened when I initially started development. I noticed that if I separately deactivated/uninstalled the workflow Feature, the InfoPath forms would then update successfully. This pointed me to an issue with the PostBuildActions.bat script.

    Resolution:- Modify PostBuildActions.bat to use the -force parameter when deactivating/uninstalling the Feature.

  3. Issue:- Error message “The e-mail message cannot be sent. Make sure the outgoing e-mail settings for the server are configured correctly.” is shown in the workflow history list. E-mail configuration is known to be good (e.g. alerts are working).

    More information:- I suspect there are several reasons why this may happen, but for me the problem was how I was using the SendEmail activity. When I was setting properties on the SendEmail variable the workflow designer had added to my class for me, I had the error.

    Resolution:- The best way I found to deal with this was to set properties on the object which is passed to the SendEmail’s ‘MethodInvoking’ code. You’ll need to cast to SendEmail before you can do this.

  4. Issue:- The TaskID for a CreateTask activity does not get populated. The value defaults to -1 instead.

    More information:- This will be an issue in several scenarios, one example is when building the URL for a task (e.g. to include in an e-mail to a user) with code similar to Paul Hunt’s code shown at http://suguk.org/forums/thread/4978.aspx.

    Resolution:- The TaskID will only be populated if there is a place to store the value. Ensure the TaskID property on your CreateTask activity is bound to a variable.

  5. Issue:- Error messageCorrelation value on declaration ‘x’ is already initialized” message shown in SharePoint log when a task executes for the second time in a workflow.

    More information:- This scenario can happen when a CreateTask runs for a 2nd time due to an approver rejecting something in the workflow.

    Resolution:- The ‘OwnerActivtyName’ of the task’s correlation token is incorrectly set. In a state-machine workflow, it should refer to the particular state, not the overall workflow.

  6. Issue:- Unexplained errors when some part of an otherwise functioning workflow is commented out in the designer (e.g. ‘Failed on start’).

    More information:- You’ve commented out one or more activities in your workflow (using right-click > Disable), but observe strange behaviour and/or errors. This is actually because, in contrast to commenting out source code, the code is actually still compiled into the assembly. Hence if there is an error with say, a correlation token being incorrectly set, this will still cause a problem in your workflow.

    Resolution:- Fix problem or remove faulting activity/activities completely from workflow.

For completeness, some other common issues which are well documented in other sources are:

  • ‘Failed on start’ – this often caused by a problem loading the workflow assembly. Check the PostBuildActions.bat script (or whatever you are using) has properly deployed the assembly to the GAC, and all the assembly attributes (version, public key token etc.) match those specifed in the Feature files.
  • ‘SPException: This task is currently locked by a running workflow and cannot be edited’ message. This is documented by Rob Bogue and others, but I’d also add this can occur as a knock-on effect of another problem. If an earlier exception has occurred whilst trying to process a task, the workflow will have locked the task on the first run, meaning that subsequent attempts by a user to respond to the task will result in this error. Other than fixing the problem at source, I haven’t found an elegant way of dealing with this.

So hopefully that’s of some use to workflow developers. If you haven’t seen it already, there’s a lot more goodness (sample code, tips and tricks etc.) in my workflow resources pack from the workflow deep-dive presentation I did at the UK SharePoint user group earlier this year.

Happy coding!

InvokeWorkflow – child workflows not supported in SharePoint?

Our client through an interesting curve ball at us midway through my workflow project. We were probably 70% code complete, when it was realised the design had no provision for a certain scenario. The workflow is triggered by a user completing an InfoPath form – this has many input controls to collect data, and several of them are dropdowns which dictate the particular path through the workflow which occurs. One such dropdown looks something like:

Type:

  • type 1
  • type 2
  • both

The value selected is evaluated by IfElse conditions in the workflow – if "type 1" is chosen, we go down a particular branch, if "type 2", a different branch and so on. Now for this particular dropdown, we realised we hadn’t specified what should happen for the "both" condition (that’s agile for you!). The new requirement which arose was that notifications and tasks should go to to two teams (i.e. both of them), and after further discussion we decided it would be appropriate to kick off two separate workflows, one for each team.

Most workflow frameworks can deal with this by using ‘child’ workflows, and Workflow Foundation is no different. After a couple of hours refactoring the implementation to use this approach, my proof-of-concept looked like:

ParentWorkflow

So whilst the ‘InvokeWorkflow’ activities are closed (i.e. the child workflows aren’t expanded), you can hopefully see that if we go down the branch on the left, we launch one child workflow, but in the branch on the right (for "both"), we launch two child workflows in parallel.

The problem

I tried a couple of things, the image above shows using a sequential parent workflow (though the child workflow is state-machine), but couldn’t get this to work. Using the debugger I could see that the child workflows were being initialized – I could step through InitializeComponent(), but breakpoints in ‘OnWorkflowActivated’ were never hit – the event wasn’t raised. The SharePoint log showed:

System.Workflow.Activities.EventDeliveryFailedException: Event "OnWorkflowActivated" on interface type "Microsoft.SharePoint.Workflow.ISharePointService" for instance id "67c4929a-9280-4b8b-9a4c-a1b85b04267c" cannot be delivered. —> System.InvalidOperationException: Event Queue operation failed with MessageQueueErrorCode QueueNotFound for queue ‘Message Properties Interface Type:Microsoft.SharePoint.Workflow.ISharePointService Method Name:OnWorkflowActivated

So I became suspicious of whether it’s possible to use InvokeWorkflow in a SharePoint workflow. Microsoft haven’t yet got back to me on this, and the most detailed SharePoint workflow book I’m aware of (Workflow in the 2007 Microsoft Office System by David Mann) does not discuss InvokeWorkflow.

Time for some lateral thinking.

The solution

We know that we will definitely have two workflows if two completed InfoPath forms are added to the document library, so I asked the client what they thought of getting the system to split a "both" request into two separate items – this would mean having two separate InfoPath forms in the list. The answer was "actually that’s probably better for the users come to think of it", so the solution I came up with was:

  • a list event receiver which executes before the workflow – this does the following:
    – opens the binary contents of the InfoPath form as a string, and modifies the dropdown value from "both" to "type 1"
    – creates a duplicate of the original file in the same library, but modifies the dropdown value here from "both" to "type 2"
  • running some ad-hoc code to reorder the event receivers on the list – we need to split the item before workflow starts (so we get two workflows) but SharePoint adds the SPWorkflowAutostartEventReceiver at sequence of 1 so it executes first. This code switches the event receiver sequence so our pre-processing happens first.

On the last point, I considered that the SharePoint team may have set workflow to execute first for a reason, but then again it’s just another event receiver and SharePoint does not prevent this reordering (in the same way it does with some other changes, such as deleting the welcome page from a web for example). In any case, we’ve tested pretty extensively and the solution hasn’t missed a beat.

In case it’s useful, the following code can be used to duplicate and modify a file – this could be used in a list receiver (as in my case) or in ad-hoc code. I used simple string manipulation to perform the modification since InfoPath forms are just XML, and also because using XML-handling methods caused issues with the ‘proprietary’ InfoPath declarations. Simply replace the string replacement line with your own processing.

   1: private string getFileContentsAsString(SPFile file)
   2: {
   3:     byte[] aFileBytes = file.OpenBinary();
   4:     string sContents = Encoding.UTF8.GetString(aFileBytes);
   5:  
   6:     return sContents;
   7: }
   8:  
   9: private void duplicateFile(SPListItem originalListItem, string sNewItemUrl, 
  10:     bool bDeleteOriginal)
  11: {
  12:     SPFile originalFile = originalListItem.File;
  13:     string sOriginalFileContents = getFileContentsAsString(originalFile);
  14:  
  15:     // TODO: replace this line with your own processing..
  16:     string sNewFileContents = sOriginalFileContents.Replace("oldValue", "newValue");
  17:     
  18:     byte[] aNewFileBytes = Encoding.UTF8.GetBytes(sNewFileContents);
  19:  
  20:     // we effectively need to add a new file for this operation, and optionally delete 
  21:     // the original..
  22:     SPFile newFile = originalListItem.ParentList.RootFolder.Files.Add(sNewItemUrl, aNewFileBytes,
  23:                                                                       originalFile.Properties);
  24:     newFile.Update();
  25:     
  26:     if (bDeleteOriginal)
  27:     {
  28:         originalListItem.Delete();
  29:     }
  30: }

P.S. If anyone is using InvokeWorkflow in a SharePoint workflow, leave a comment 🙂