Personal announcements from me

I know not all of my blog readers follow me on Twitter (though if you are a SharePoint Twitteree, I’m @ChrisO_Brien), so I just wanted to post something here to say that last week I became a proud Dad, to twin boys! They came early by emergency caesarean – personally I knew Suzanne shouldn’t have had that last piece of chocolate cake. They initially had low blood sugar levels (not related to the cake, I should add!) so we had to stay in hospital a day or so longer than normal whilst things stabilized, but in general everyone is healthy and happy. It’s funny to wonder if SharePoint will exist and how it will be when they’re older – I’m just not sure I want to bring them into a world with CAML quite frankly.

Can’t remember what was to the left of this photo, but it was clearly very interesting 😉

whatsoverthere

They’re pretty tiny. Going out is very amusing so far, on our local high street (Main St for U.S. folks!) we seem to get stopped every few hundred meters. Whilst sat in a cafe yesterday, I overheard someone say “Oh my God look at the size of that baby, he’s holding it in one hand!”.

chrisandkai 

So, we’re very proud of our tiny Yodas – thanks for all the messages on Twitter, it was great to read through them.

SharePoint Conference 2009

Away from personal stuff, we’re closing in on SPC 09 in Las Vegas. Here you can listen to Microsoft’s own and the best in the field, and gain an unparalleled insight ahead of your first SharePoint 2010 implementation. The odds (!) are currently good that I’ll be there – this  actually involves an elaborate plan to fly Suzanne’s Mum over from the U.S. to London to “cover me” for the week, so if everything falls into place I’ll see you there! I’m really excited to be able to start talking more publicly about all the new platform features – hope you are too.

Ill be at SPC

Remember – you can follow me at @ChrisO_Brien for SharePoint-dev goodness if you’re on Twitter..

Minor updates to Content Deployment Wizard and Config Store

I’m a firm believer that if you build community tools which become important to folks, you’ve got to do your best to maintain them! So this is a brief post for users of my SharePoint Content Deployment Wizard and Config Store utilities. Both of these have recently been updated on Codeplex, and depending on circumstances you may want to ensure the latest versions are what you have in your toolbox. I’ll explain the changes I noted in the readme.txt of the respective updates here:

Content Deployment Wizard (2.6 beta)

Release notes:

This release is primarily to fix a regression bug in the 2.5 release (the first item in the list below) and provides:

  • Fix to issue where lists cannot be exported with ‘All descendents’ option because menu item is missing
  • Separate installers for x86 and x64 to ensure the STSADMCOMMANDS.COB.SPDEPLOYMENTWIZARD.XML file gets properly installed to 12\config directory

Also note the main new features of the 2.5 beta (included in this release):

  • Support for incremental deployments, so only content which has changed since the last export gets deployed (select ‘ExportChanges’ from the dropdown to use this feature)
  • Support for "no compression" deployments via the ‘Disable compression’ checkbox – this is useful for large sites where memory use is an issue during Content Deployment
  • Support for allowing the root web of a site collection to be deployed on it’s own – extra options in context menus now allow selection of ‘site’ or ‘root web’. Previously it was not possible to deploy only a root web.

Should you update?

A definite yes if you downloaded the 2.5 beta release, due to the bug in that version.  If you have an earlier version e.g. 1.1 or 2.0, then most likely it’s still a good idea to upgrade to the 2.6 beta as it’s fairly well tested and has some functionality you might need some day. The beta label will likely come off in a few weeks, as I know this matters to some folks.

Download the Content Deployment Wizard 2.6 beta

Config Store (2.1.0.2)

Release notes:

Changes in this release:

  • Changed internal field names back to not contain spaces – caused problem under some circumstances (an issue reported by Waldek when Config Store used with used with one of his tools)
  • Changed ApplyWebConfigModifications Feature property back to True, since seems most installations use single site collection mode
  • Tidied schema.xml based on recommendations made in http://mo.notono.us/2009/02/moss-dreaded-schemaxml.html

Should you update?

In this case, if you’re already using the Config Store in your site(s) it’s recommended to only consider upgrading to this version if you know you have the same problem reported by Waldek – because the field names changed, an upgrade will require some work on your part (e.g. a utility to migrate from the old schema). However, for greenfield installations, this version should definitely be used.

Download the Config Store 2.1.0.2

N.B. These Config Store changes will be ported to the Language Store framework (sister project) over the next few days.

A note on SharePoint 2010

At this stage I’m expecting my Shareutilities to continue to be useful in the 2010 world.

Incremental deployment support added to CD Wizard

Back now from a short break from blogging for a holiday and some preparation for SharePoint 2010. One of the things I’ve been working on recently is the latest release of my SharePoint Content Deployment Wizard – this has just been uploaded to Codeplex. In this release (2.5 beta) I focused on adding support for a couple of things that were common feature requests, and which should hopefully make the tool more useful for users which have these requirements. As you’ll have guessed from the title of this post, probably the biggest one is incremental deployments, but there are a couple of other useful things in there too.

Support for incremental deployments

This is something that some folks have asked for for a long time. If you are doing regular updates using the Wizard for a set of content (perhaps using the command-line support added in release 2.0), it’s often preferable to only deploy the content which has actually changed. This is now possible, and is done by selecting ‘ExportChanges’ from the ‘Export method’ dropdown on the export settings screen:

ExportChanges  
Confusingly you might have noticed the option was in the dropdown in previous releases, but it now does what it should.

How it works

Incremental deployment relies on SharePoint’s change log – the queryable store which several pieces of SharePoint use to know what content changed when. A change token can be used as a “point in time reference” which is used to identify changes after certain point, and the Wizard now stores a change token each time you perform an incremental export. When you next perform an incremental export for the same scope, the Wizard retrieves the last token for this content which is then used to ensure only changed content is deployed. Some key points here are:

  • The last change token is stored as an SPPersistedObject in the respective content database
  • Since the Wizard allows selection of multiple items to export, the change token of the ‘largest’ scope object which is selected for export is used

Support for disabling compression

Another new feature is the option for turning off the file compression which happens by default in SharePoint content deployment. When exporting a large amount of content, you will have seen that a series of files are generated, each of which is no larger than 25GB (note this is actually a configurable number, though I’ve not yet seen the requirement to expose this in the Wizard). Most of the time, you might not care about this, but the situation where it does become important is where the server doesn’t have enough memory to deal with the compression process. This could be because an extremely large site collection is being exported, or because the server just doesn’t have a lot of available memory.

To disable file compression, use the new checkbox shown below:

DisableCompression

Support for exporting the root web only

Previous versions of the Wizard actually didn’t support the scenario where one wants to export only the root web of a site collection. This is because the treeview only gave one option for the root node – ‘Export entire site’. This meant that when you selected this but really only wanted the root web, what you ended up getting was the entire site collection – which unsurprisingly has confused people in the past. This has now been fixed, the context menu now has an extra level where it’s possible to select just the root web:

ExportRootWebOnly

Conclusion

The new features of this release are:

  • Support for incremental deployments
  • Support for disabling file compression
  • Support for exporting the root web only

The latest 2.5 beta of the Content Deployment Wizard can be downloaded from Content Deployment Wizard Release 2.5 beta. As soon as I feel enough people have used this version without issues I’ll remove the beta label, as I know some folks care about this even for Codeplex utilities. As always, if you run into any problems leave me a message on the Codeplex site and I’ll do my best to help.

Renewed as SharePoint MVP

Like several others, I heard the good news from Microsoft last week that I’m a SharePoint MVP for another year. I’m really pleased to get the award again, as it’s nice to be recognized by Microsoft for what you’ve done. I find it useful (to me anyway!) to have a bit of reflection on what I’ve been up to for the last year. The big things were probably:

A busy year to be sure. I just want to say thanks to everybody that went to the trouble of passing on their thanks or feedback on some of this stuff – I really appreciate it..

My checklist for optimizing SharePoint sites

Optimization is probably one of my favorite SharePoint topics – it seems there’s always a new trick to learn, and I love the fact that a minor tweak can have a dramatic impact on how well your site works. There are some great resources out there now for information on how to boost performance, but it strikes me that there isn’t one single paper or article which contains all the aspects I personally consider or have been relevant to me on past projects. Hence this article is essentially an aggregation of some of these sources. The aim isn’t for it to be “complete” (if there is such a thing) or authoritative in any way, and I know I wouldn’t be the first person to blog on this subject – it really is just a reminder list for me to refer to, but if it helps you also that’s great.

Key resources:

Before I break into the list, remember that different optimization measures will have different effects depending on your circumstances – having the best checklist in the world is really no substitute for the SharePoint architect’s ability to identify what is likely to be the bottleneck for a given implementation. Additionally my list is quite “developer-focused”, but if the infrastructure aspects of the implementation is the limiting factor (e.g. latency between web servers and SQL, amount of usable RAM etc.) you can probably optimize code until you’re blue in the face without solving your performance problem.

My list is broken down into sections, and some of the items have links to some useful resource or other which you might find helpful if the topic is new to you.

Configuration

Items in this section typically have  a big impact – in particular output caching, which is the first thing to consider on WCM sites for example.

Code/development

  • Test with SPDisposeCheck to ensure no memory leaks
  • Measure page payload weight (again, YSlow useful here) – aim to reduce as far as possible
  • Eliminate unnecessary ViewState! A good dev technique may be to turn off ViewState in web.config and then override it in controls/pages which really do need it (haven’t tried this, but keep meaning to).
  • Ensure not downloading core.js etc for anonymous users/delay-loading for authenticated.
  • Ensure image sizes are small/images are optimized for web (still comes up!)
  • Always ensure height/width attributes specified on images
  • Ensure custom CSS is factored correctly
  • Don’t forget client-side code efficiency e.g. CSS, jQuery selectors
  • Consider using a code-profiling tool such as ANTS profiler or dotTrace to identify sub-optimal code
  • Ensure general good coding practice e.g. your optimization work may well be undone if your code is badly-written. Accidentally doing unnecessary processing in a loop (in server-side OR client-side code is one example I’ve seen many times)
  • Be wary of performance impact of custom HTTP modules – I frequently see huge perf gains when running load tests with such modules enabled/disabled. Removing core.js as a post-processing step within a HTTP module is NOT the right approach kids! (No really, I’ve seen it done..)

Code/development – advanced

These tips might only really be necessary if you’re implementing a high-traffic site, but are worth considering:

  • Consider factoring custom JS/CSS into fewer files – fewer HTTP requests is better
  • Consider “minifying” JS
  • Consider using CSS to cluster background images (all images stitched into one for fewer HTTP requests) – AC might have mentioned somewhere an online service to automate this..

Architecture

You should consider this section absolutely incomplete – hopefully there’s enough here to dissuade anybody of the notion that it’s only about code though! The first two in particular are key.

  • 64-bit/sufficient available RAM etc.
  • Latency between web and SQL – MS suggest a ping response of less than 1 millisecond
  • Consider a CDN solution (e.g. Akamai, Limelight etc.) if users are geographically far away from servers (and y0ur client can afford it!)
  • Ensure not using web garden with BLOB cache (known BLOB cache issue)
  • Are app pool settings correct – check for excessive app pool recycling by enabling logging to the Windows event log using – cscript adsutil.vbs Set w3svc/AppPools/ <YourAppPoolName> /LogEventOnRecycle 255. Once or twice per day is probably the maximum you’re hoping for.
  • Is storage correctly architected e.g. are OS, data, logs on different disk spindles?
  • Is storage correctly configured (e.g. SAN configuration) – are your LUNS giving you sufficient IOPS?!


Using YSlow to help optimize websites

One particular tool worthy of discussion when we’re talking about optimizing websites is YSlow – if you haven’t come across it yet, this is a Firefox plugin which extends Firebug) developed by Yahoo! developers which uses their optimization ruleset (customizable) to report on and help further streamline your site. The tool focuses on communication between the server and browser – clearly a browser tool like this can’t identify any back-end issues, but it’s always interesting to run a site through the checks.

The first tab gives a ‘grade’ for your site based on the ruleset, helping you identify areas for improvement (I’ve highlighted where the page stops and the YSlow console starts with a red box):

YSlow_Grading 
The next tab provides a breakdown of files downloaded by the request, and allows me to see their size and headers etc. In particular I can see the expiry date and Etag and so on which the files are being tagged with so they can be cached locally:

YSlow_Components

The statistics tab provides some good analysis on the page weight and also illustrates the difference between a first visit and subsequent visits where appropriately tagged files will be served from the browser cache:

YSLow_Stats

Finally the Net tab in Firebug is also interesting, as this shows me how files were downloaded (sequential or in parallel) – the most recent browsers do a much better job of this, with IE8 and FF3 being able to open 6 channels per URL domain to download files in parallel, but note that IE7 could only open 2.

Firebug_NetAnalysis

From this, I also see the http://sharepoint.microsoft.com site (great site btw) I’ve been analyzing also displays the BLOB cache issue I discussed recently (now confirmed as a bug) where incorrect headers are added to files stored in the Style Library, causing lots of unnecessary HTTP 304s to go across the wire. So YSlow does give a great insight – however I do agree with Jeff Atwood’s reminder that some of the reported “issues” may not be the biggest concern for your site. As always, apply common sense.

Summary

Optimization is a deep topic, and this checklist is simply my reminder of some of the things to consider, though in reality a solid understanding of the nuts and bolts is required to really architect and develop high-performing sites. Tools like YSlow can also help with some aspects of optimization.

So which optimization nuggets did I miss which you consider? Feel free to leave a comment and add to this list..

More on optimization, HTTP 304s etc. – a solution?

In my last post Optimization, BLOB caching and HTTP 304s, I did a fairly lengthy walk-through on an issue I’d experienced with SharePoint publishing sites. A few people commented, mainly saying they’d noticed the same thing, but there have been further developments and findings I wanted to share!

Quick recap

Under certain circumstances some files in SharePoint are always re-requested by the browser despite being present in the browser cache (“Temporary internet files”). Specifically this is observed for files stored in the Style Library and Master Page Gallery, for anonymous users. Although SharePoint responds with a HTTP 304 to say the cached file can indeed be used (as opposed to sending the file itself again), we effectively have an unnecessary round-trip to the server for each file – and there could be many such files when all the page’s images/CSS/JS files are considered. This extra network traffic can have a tangible impact on site performance, and this is magnified if the user is geographically far away from the server.

A solution?

Waldek and I have been tossing a few development matters around recently over e-mail, and he was curious enough to investigate this issue for himself. After reproducing it and playing around for some time, Waldek discovered that flushing the disk-based cache seems to cause a change in behaviour – or in layman’s terms, fixes everything. To be more specific, we’re assuming it’s a flush of the BLOB cache which is having the affect – in both Waldek’s test and my subsequent validation, the object cache was also flushed as well:

FlushDiskCache

After the OK button is hit on this page, the problem seems to go away completely, so now when the page is accessed now for the first time as an anonymous user, the correct ‘max-age’ header is added to the files (as per the BLOB cache declaration in web.config) – contrast the ‘max-age=86400’ header on the Style Library files with what I documented in my last post:

AnonymousCorrectHeadersAfterFlushCache

This means that on subsequent requests, the Style Library files are served directly from the browser cache with no 304 round-trip:

SecondRequestNo304s

This is great news, as it means the issue I described is essentially a non-issue, and there is therefore no performance penalty for storing files in the publishing Style Library.

So what gives?

I’m now wondering if this is just a ‘gotcha’ with BLOB caching and publishing sites. I know other people have run into the original issue due to the comments on my previous post, and interestingly enough one poster said they use reverse proxy techniques specifically to deal with this issue. Could it really be that everybody who sees this behaviour just didn’t flush the BLOB cache somewhere along the way, when it’s actually a required step? Or is the testing that Waldek and I did flawed in some way? Or indeed, was my initial investigation flawed despite the fact others reported the same issue?

I am interested to hear from you on this – if you can reproduce the problem I’ve described with a publishing site you’ve developed, does flushing the BLOB cache solve it for you as described here? Leave a comment and let us know!

Good work Waldek 🙂

Optimization, BLOB caching and HTTP 304s

There’s been an interesting mini-debate going on recently in terms of where to store static assets used by your site – images, CSS, JS files and so on. Broadly the two approaches can be characterized as:

  • Developer-centric – store assets on the filesystem, perhaps in the 12 hive
  • Author-centric – store assets in the content database, perhaps in the Style Library which comes with publishing sites

Needless to say these options offer different pros and cons depending on your requirements – Servé Hermans offers a good analysis in To package or not to package: that is the question. However, I want to throw another point into the debate – performance, specifically for anonymous users. Frequently, this is an audience I care deeply about since some of the WCM sites I work on often have forecast ratios of 80% anonymous vs. 20% authenticated users. Recently I was asked to help optimize an under-performing airline site built on MOSS – as usual the problem was a combination of several things, but one of the high-impact items was this decision to store assets in one location over the other. In this post I’ll explain what the effect on performance is and why you should consider this when building your site.

The problem

Once they’ve been loaded the first time, most of the static files a website uses should be served from the user’s local browser cache ("Temporary internet files") – without this, the internet would be seriously slow. Consider how much slower a web page loads when you do a hard refresh (ctrl+F5) compared to normal – this is because all the images are forced to be re-downloaded rather than served from the browser cache. Unfortunately, for files stored in some common SharePoint libraries/galleries (i.e. the author-centric approach) SharePoint doesn’t deal with this quite right in some scenarios – most of the gain is there, but despite having the image locally, the browser still makes a request for the image – the conversation goes like this (for EACH image on the page!):

Browser: I need this image please – I cached it last time I came at [date/time], but for all I know it’s changed since then.
Server: No need dude, it’s not changed so just use your local copy (in the form of a HTTP 304 – "Not modified")
Browser: Fair enough, cheers.

This essentially happens because the file was not served with a "cacheability" HTTP header to begin with. Needless to say, this adds significant time to the page load when you have 30+ images/CSS/JS files referenced on your page – potentially several seconds in my experience (under some circumstances), which of course is a huge deal. If say, the user is in Europe but the servers are in the U.S., then suddenly this kind of network chatter is something we need to address. Needless to say, in the majority of cases we’re happy to cache these files for a period since they don’t all change too often, and we get better performance as a result.

The Solution (for some SharePoint libraries *)

Mike Hodnick points us to part of the solution in his highly-recommended article Eliminating "304" status codes with SharePoint web folder resources. Essentially, SharePoint’s BLOB caching feature saves the day since it serves the image with a "max-age" value on the HTTP header, meaning the browser knows it can use it’s local copy of the file until this date. This only happens when BLOB caching is enabled and has the max-age attribute like this (here set to 84600 seconds = 24 hours):

<BlobCache location="C:\blobCache" path="\.(gif|jpg|png|css|js|aspx)$" maxSize="10" enabled="true" max-age="86400" />

When we configure the BLOB cache like this we are, in effect, specifying that it’s OK to cache static files for a certain period, so the "cacheable" header gets added. HOWEVER, what Mike doesn’t cover is that this only happens for authenticated users – files served out of common content DB locations such as the Style Library and Master Page Gallery still do not get served correctly to anonymous users. Note this isn’t all SharePoint libraries though – so we need to be clear on exactly when this problem occurs.

* Scope of this problem/solution

Before drilling down any deeper, let’s stop for a moment and consider the scope of what we’re discussing – a site with:

  • Anonymous users
  • Files stored in some libraries – I’m not 100% sure of the pattern but discuss it later – the Style Library and Master Page Gallery are known culprits however. Other OOTB libraries such as SiteCollectionImages do not have the problem.

If you don’t have this combination of circumstances, you likely don’t have the problem. For those who do, we’re now going to look closer at what’s going on, before concluding with how we can work around the issue at the end.

Drilling deeper

For a site which does have the above combination of circumstances, we can see the issue with Fiddler – as an anonymous user browsing to page I’ve already visited, I see a stack of 304s meaning the browser is re-requesting all these files:

BlobCachingDisabled_304s

However, if I’m authenticated and I navigate to the same page, I only see the HTTP 200 for the actual page, no 304s:

BlobCachingEnabled_No304s

Hence we can conclude it works fine for authenticated users but not for anonymous users.

So what can we do for our poor anonymous users (who might be in the majority) if we’re storing files in the problematic libraries? Well, here’s where I draw a blank unfortunately. Optimizing Office SharePoint Server for WAN environments on TechNet has this to say on the matter:

Some lists don’t work by default for anonymous users. If there are anonymous users accessing the site, permissions need to be manually configured for the following lists in order to have items within them cached:

  • Master Page Gallery
  • Style Library

Aha! So we need to change some permissions – fine. This seems to indicate that it is, in fact, possible to get the correct cache headers added to files served from these locations. Unfortunately, I simply cannot find what permissions need to be changed, and nobody on the internet (including the TechNet article) seems to detail what. The only logical setting is the Anonymous Access options for the list – these are all clear by default, but adding the ‘View Items’ permission (as shown below) does not change anything:

AnonPermissions

As a sidenote, the setting above is (I believe) effectively granting read permissions to the identity which is used for anonymous access to the associated IIS site. So in IIS 7.0, I’m fairly sure you’d achieve the same thing by doing this:

AddPermsIUsr

So the problem does not go away when anonymous users are granted the ‘View Items permission, and what I find interesting about this is that a closer look with Fiddler reveals some inconsistencies. The image below shows me browsing to a page anonymously for the first time, and to save you the hassle we can derive the following findings:

  • Files served from the ‘SiteCollectionImages’ library are given the correct max-age header (perhaps expected, since not one of the known ‘problem libraries’ e.g. Style Library)
  • Files served from the ‘_layouts’ folder are given a different max-age header (expected, settings from the IIS site are used here)
  • Some files in the Style Library are in fact given a the correct max-age header! (not expected) 

MixedHeaders_Anonymous

So the 2 questions which strike me here are:

  • Why are some files being served from ‘Style Library’ with the correct header when most aren’t?
  • Why can SharePoint add the ‘max-age’ header to files in the ‘SiteCollectionImages’ library but not the ‘Style Library’?

The first one is a mystery to me – it’s perhaps not too important, but I can’t work it out. The second one might be down to how the libraries are provisioned – the ‘Style Library’ is provisioned by declarative XML in the ‘PublishingResources’ Feature, whereas the ‘SiteCollectionImages’ library is provisioned in code using the same Feature’s activation receiver. Could this be the key factor? I don’t know, but I’d certainly be interested if anyone can put me straight – either on this or the mystery "permissions change" required to make BLOB caching deal with libraries such as the ‘Style Library’.

Conclusion

The key takeaway here is that for sites which want to take advantage of the browser caching for static files (for performance reasons) and have anonymous users, we need to be careful where we put our images/CSS/JS files as per Mike Hodnick’s general message. If we want to use the author-centric approach and store things in SharePoint libraries, we need to consider which libraries (and test) if we will have the 304 problem. Alternatively, we can choose to store these files on the filesystem (the developer-centric approach) and use a virtual directory with the appropriate cacheability settings to suit our needs. My suggestion would be to use a custom virtual directory for full control of this, since the default settings on the ‘_layouts’ directory ("cache for 1 year") are unlikely to be appropriate.

Fix to my Language Store framework for multi-lingual sites

In my last post, I talked about a fix to my Config Store framework for an issue which manifested itself on certain SharePoint builds, with Windows 2008 and a recent cumulative update seeming to be the trigger. Some of you may know that I produced a sister project to this one called the ‘Language Store‘, which is designed to help build multi-lingual SharePoint sites – since this framework is built off the same underlying XML and plumbing, this solution was also affected.  So this post is just a short one to say that the fix has now been applied to the Language Store framework, and the new version is now available on Codeplex at http://splanguagestore.codeplex.com.

Problem/solution

The problem was effectively that items in the SharePoint list could no longer be edited – well, in fact they could be updated using code, but the list form .aspx pages were not showing the fields correctly so items couldn’t be edited in the UI. Since it kind of defeats the point of SharePoint to have to write code to update list items (!), this was a big issue on affected builds. Interestingly some users reported working around the issue by removing/re-adding the content type from the list in the browser, but happily this is no longer necessary since the root issue has now been resolved. The problem was traced to some incorrect XML in my FieldRef elements – see the last post Fix to my Config Store framework and list provisioning tips for the full info.

General recap – the Language Store

If you’re still reading, I figure some folks would welcome a reminder/intro on what the Language Store actually does – it’s not about replacing SharePoint’s variations functionality which is commonly used on multi-lingual sites. I noticed Spence gave it a better name in an e-mail recently where he described it as a ‘term store‘ for multi-lingual sites – this actually captures what it does far better than my name for it. Effectively the idea is to provide a framework for the many small strings of text which are not part of authored page content which need to be translated and displayed in the appropriate language. As an example, here is a page from the BBC site where I’ve highlighted all the strings which may need to be translated but which don’t belong to a particular page:

BBCExample

There are many of these in a typical multi-lingual site, and to help deal with this requirement the Language Store framework provides the following:

  • SharePoint list/content type/site columns etc.
  • API to retrieve items with a single line of code
  • Granular caching for high-performance
  • Packaged as a .wsp for simple deployment
  • All source code/XML freely available

If you want to find out more, see Building multi-lingual SharePoint sites – introducing the Language Store. The solution can be downloaded from the Codeplex site at http://splanguagestore.codeplex.com.

Apologies to existing users who were affected by the issue.

Fix to my Config Store framework and list provisioning tips

Had a couple of reports recently of an issue with my Config Store solution, which provides a framework for using a SharePoint list to store configuration values. If you’re using the Config Store this article will definitely be of interest to you, but I’ve also picked up a couple of general tips on list provisioning which I want to pass on. I have to thank Richard Browne (no blog) of my old company cScape, as the fix and several of the tips have come from him – as well as alerting me to the problem, he also managed to fix it before I did, so many thanks and much kudos mate 🙂

Config Store problem

Under some circumstances, fields in the Config Store list were not editable because they no longer appeared on the list edit form (EditForm.aspx). So instead of having 4 editable fields, only the ‘Config name’ field shows in the form:

ConfigStoreMissingFields

I’ve not fully worked out the pattern, but I think the problem may only appear if you provision the list on a server which has the October or December Cumulative Update installed – either that or it’s a difference between Windows 2003 and Windows 2008 environments (which would be even more bizarre). Either way, it seems something changed in the way the provisioning XML was handled somewhere. This is why the problem was undetected in the earlier releases.

I had seen this problem before – but only when the list was moved using Content Deployment (e.g. using the Content Deployment Wizard) – the original ‘source’ list was always fine. We managed to work around this by writing some code which ‘re-added’ the fields to the list from the content type, since they were always actually present on the content type and the data was still corrected stored. Having to run this code every time we deployed the list was an irritation rather than critical, but something I wanted to get to the bottom of – however, on finding some folks were running into this in ‘normal’ use meant that it became a bigger issue.

The cause

I always knew the problem would be down to a mistake in the provisioning XML, but since I’d looked for it on previous occasions I knew it was something I was seeing but not seeing. In my case, Richard spotted that I was using the wrong value in my FieldRef elements under the ContentType element – I was mistakenly thinking that the ‘Name’ attribute needed to match up with the ”StaticName’ attribute given to the field; the documentation says this attribute contains the internal name of the field. So my FieldRefs looked like this:

<ContentType ID="0x0100E3438B2389F84cc3965600BC16BF32E7" Name="Config item" 
Group="Config Store content types" Description="Represents an item in the config store." Version="0">
<FieldRefs>
<FieldRef ID="{33F5C8B4-A6BB-41a4-AB24-69F2152974C5}" Name="ConfigCategory" Required="TRUE" />
<FieldRef ID="{BD413479-48AB-41f5-8040-918F32EBBCC5}" Name="ConfigValue" Required="TRUE" />
<FieldRef ID="{84D42C64-D0BD-4c76-8ED3-0A9E0D261111}" Name="ConfigItemDescription" />
</FieldRefs>
</ContentType>

..to match up with fields which looked like this:

<Field ID="{33F5C8B4-A6BB-41a4-AB24-69F2152974C5}"
Name="Config category"
DisplayName="Config category"

StaticName="ConfigCategory"
....
....
/>

The CORRECTED version looks like this (note the change in value for the Name attribute of FieldRefs):

<ContentType ID="0x0100E3438B2389F84cc3965600BC16BF32E7" Name="Config item"
Group="Config Store content types" Description="Represents an item in the config store." Version="0">
<FieldRefs>
<FieldRef ID="{33F5C8B4-A6BB-41a4-AB24-69F2152974C5}" Name="Config category" Required="TRUE" />
<FieldRef ID="{BD413479-48AB-41f5-8040-918F32EBBCC5}" Name="Config value" Required="TRUE" />
<FieldRef ID="{84D42C64-D0BD-4c76-8ED3-0A9E0D261111}" Name="Config item description" />
</FieldRefs>
</ContentType>

So, the main learning I got from this is to remember that the ‘Name’ of the FieldRef attribute needs to match the ‘Name’ of the Field attribute – that simple. Why did it work before? No idea unfortunately.

However, I also picked up a few more things I didn’t know about, partly from Richard (this guy needs a blog!) and partly from some other reading/experimenting..

Some handy things to know about list provisioning

  • To make a field mandatory on a list, the ‘Required’ attribute must be ‘TRUE’. Not ‘True’ or ‘true’ – this is one of the cases where the provisioning framework is pernickety about that 6-choice boolean 😉
  • FieldRefs need an ID and Name as a minimum (which must match the values in the ‘Field’ declaration), but you can override certain other things here like the DisplayName – this mirrors what is possible in the UI.
  • You don’t have to include the list .aspx files (DispForm.aspx, EditForm.aspx and NewForm.aspx) in your Feature if you use the ‘SetupPath’ attribute in the ‘Form’ element in schema.xml (assuming you don’t need to associate custom list forms).
  • You can use the ‘ContentTypeRef’ element to associate your content type with the list (specify just content type ID), rather than using the ‘ContentType’ element which needs to redeclare all the FieldRefs.
  • It’s safe to remove all the default ‘system’ fields from the ‘Fields’ section of schema.xml

Going further than these tips, the best thing I found on this is Oskar Austegard’s MOSS: The dreaded schema.xml which shows how you can strip a ton of stuff out of schema.xml. I’ve not tried it yet, but I’m sure that will be my starting point for the next list I provision declaratively. If you’re interested in the nuts and bolts of list provisioning, I highly recommend you read it.

Happy XML’ing..

Slide deck from my deployment talk at Best Practices Conference

Had a great time presenting at the European SharePoint Best Practices Conference last week. I’ve been trying to put my finger on what made it such a good conference and I’m actually not sure, but I notice that other speakers and attendees have also been full of praise, so it’s not just me. The event itself was extremely well-organized with excellent content, and Steve Smith and his team did a great job of looking after us speakers.

Highlights for me on the dev track were sessions from AC, Todd Bleeker, Eric (or "Uncle Eric" as I like to think of him, with his wise words on high-performance coding :-)) and Andrew Woody, but whenever I did stray from developer content I seemed to run into a great session like Mike Watson‘s on SQL Server in relation to SharePoint. Similarly I heard good things about speakers like Dan McPherson doing innovative sessions on the Information Worker track which I was disappointed to miss.

Another highlight was being on the two dev panel sessions we did, and having an interesting debate in one of them with Todd on approaches for provisioning – declarative (Features) vs. programmatic (code/PowerShell etc.). This was probably a good lead-in to my talk the next day, and some folks came up to say they really liked this conversation and that we covered it from angles they hadn’t considered, which was good to hear.

So all in all, a top conference, and fantastic to catch up with so many friends. Here’s the link for my deck:

Slide deck – Approaches and best practices for deploying SharePoint sites through multiple environments (dev, QA, UAT, production)

SBP