Extending the web part framework – part 2

In part 1, I showed how we implemented a ‘toolbox’ of page templates and functionality modules wrapped up in a governance framework, to fulfil our client’s requirement of a flexible WCM platform for building 80-100 internet sites with varying requirements. In this post, I want to detail some of the issues we ran into and the resolutions we found, focusing primarily on the ‘module framework’ we developed which is heavily-oriented around SharePoint web parts. 

Quick recap

The client is a large multi-national enterprise, and the idea is that content authoring teams in 80-100 countries will take what we’ve delivered on MOSS to create their country’s internet presence e.g. .com, .co.uk, .fr, .es etc., replacing the existing mish-mash of sites on different technologies with inconsistent branding/look and feel.

In terms of the module framework, the cornerstones of our implementation were (see part 1 for more complete details on these):

  1. Module matrix – rules for which module can be used where, to guide authors away from building a user experience which doesn’t  ‘make sense’
  2. SmartPart-like approach, but with web part properties – web parts wrapping user controls but also supporting web part properties exposed in custom tool parts
  3. Base web part/base tool part class – responsible for ‘framework’ behaviour such as checking if the current web part can be added (according to the module matrix)
  4. Combine interface of publishing field controls with web part storage – since publishing field controls (e.g. RichHtmlField) must be added in a ‘static’ manner at design-time but our authors can add controls dynamically at run-time, we developed custom controls which combine the rich functionality of the publishing HTML editor with web part storage
  5. Control adapter for WebPartZone for accessibility compliance – to get round the problem of all the HTML tables generated by SharePoint’s web part framework, which will prevent a site validating for AA
  6. Present only our web parts in the web part picker – since standard SharePoint web parts are not used anywhere in these sites
  7. Remove unnecessary options when editing web part properties (tool parts) – to avoid confusing the authors

Issues and resolutions

I think that many of the challenges we faced are worth sharing as they came about through general web part development, rather than anything specific to what we did. Before I detail the actual gotchas, take note of some key development characteristics of our project:

  • Solutions and features used to deploy artifacts such as page layouts, content types etc.
  • Kivati Studio used for some other deployment aspects
  • Main functionality implemented in user controls – web parts were effectively thin wrappers around the .ascx files using LoadControl()
  • Web parts which are ‘mandatory’ are added to pages using the AllUsersWebPart element in a feature (though as the points probably illustrate, we looked at numerous ways of dealing with this)

Finding #1 – web parts outside of web part zones cannot be edited

The reason we wanted to have web parts outside of zones (perfectly possible by dragging a web part directly into page layout markup in SharePoint Designer) is for ‘fixed’ page modules which could not be removed by the content author. When we placed web parts outside of web part zones, we found the web parts would run fine in presentation mode but unfortunately cannot be edited (e.g. to edit web part properties) – the edit menu for the web part simply does not appear. I speculate this is because it is web part zones which are linked to web part storage, and thus web part properties cannot be persisted without a zone (the values in the markup will always be used). Hence, if you want editable web parts, you need web part zones.

Resolution – ensure all web parts (even ones which cannot be removed) live in a web part zone.

Finding #2 – embedding web parts into user control markup appears to be problematic

We tested various permutations of using web parts in/out of web part zones, and also with the HTML markup directly in the page layout .aspx or in a child .ascx file. After establishing that web part zones were required, we also found that whether the markup was in the .aspx or .ascx appeared to make a difference. This was unexpected, but the net effect seems to be that if you insert the web part markup into a web part zone which is in a user control rather than directly in the page layout .aspx (i.e. by refactoring the HTML markup for the web part zone and it’s contents into a user control), again the edit menu will not display. I’m not sure why this is, but it could be related to the page execution lifecycle.

Resolution – accept that if web part zones will have web parts added to them at design-time by markup, the web part zone declaration cannot be in a user control.

Finding #3 – when using AllUsersWebPart element, duplicate web parts appear if the feature containing your page layouts is reactivated

Having decided our ‘fixed’ web parts would be added to pages using the AllUsersWebPart feature element (N.B. using this approach, ‘default’ web parts are associated with page layouts in the feature which deploys them. Web part zones are left empty on the page layout, and SharePoint provisions the web part into the zone at the time of creating a page from the layout). The issue we had with this is that all the web parts in all the zones in existing pages would be duplicated if the page layout feature was reactivated – this is because this XML is used both when the feature is activated (in the same way as say, provisioning for content types happens on activation) but also when new pages are created from a page layout.

Resolution – write a script (a Kivati task in our case) to remove duplicate web parts across all sites

[UPDATE – Waldek has an elegant solution to this problem in ‘Preventing provisioning duplicate Web Part instances on Feature reactivation’, as well as sample code similar to what we wrote for our script. DOH!]

Finding #4 – duplicate web parts can also appear when the page layout is customized (ghosted)

I’m not exactly clear on the reasons why customized files would ever cause duplicate web parts to appear, but that’s certainly what we seemed to find. What happened is that we would deploy our master pages/page layouts using a feature to our QA environment, but immediately these files would be provisioned in that site as customized (i.e. the content in the content database), instead of being uncustomized and referenced on the filesystem. After further investigation, we traced the cause of this unexpected behaviour to the use of these attributes SPD adds to page layouts:

meta:progid=”SharePoint.WebPartPage.Document” meta:webpartpageexpansion=”full”

Resolution – ensure the version of the file does not contain these attributes. We actually switched to running uncustomized master page/page layouts even in our development farm. This means that we deployed the files using a feature and thereafter never opened them in SPD (editing only the source-controlled feature file instead).

Finding #5 – avoid setting default properties in the web part definition file (.webpart)

A final lesson we learnt is that, when working with web parts it’s often better to avoid using the .webpart definition file extensively for setting default property values. There’s nothing wrong with the mechanism – effectively these values are read whenever the web part is provisioned on a page, and your instance will set it’s properties to these values. The problem, of course, is when you realize a property value you defined in the .webpart file needs to be updated because something changed. What happens to all the existing instances on pages around your site? As you might guess, the answer is nothing – unless you take steps to update those also, which generally means writing some kind of script to use SPLimitedWebPartManager. This can be pretty inconvenient when all you wanted to do was quickly change a default value.

Resolution – consider ensuring .webpart files are stripped to the bare minimum (assembly name etc.) and configuration comes from somewhere else. We typically rolled these config items into our use of the Config Store.

Summary

We ran into a few unexpected gotchas when building on the web part framework, but steps can be taken to minimise their impact. Hope you find these useful if you do web part development. Special thanks to Karoly Szalkary for helping to refresh my memory on some of these!

P.S. After 2 years writing about it, I’ve decided I no longer need to capitalize the ‘f’ in ‘feature’ – I think we’re all on the same page on that one now 😉

Extending the web part framework – part 1

Today I want to show some of the interesting things we’ve been doing with web parts for one of our clients. There’s quite a lot to talk about so it will be over two articles:

  • Part 1 – background and implementation
  • Part 2 – issues and resolutions

There are a couple of things in particular which I think are quite cool, as we’ve effectively combined classic WCM (publishing) site functionality with a customized implementation of the web part framework. The context is a fairly large roll-out to an enterprise client, but what we’re rolling out is a centralized platform for 80-100 internet sites. The idea is that content authoring teams in 80-100 countries will take what we’ve delivered on MOSS to create their own sites – replacing the existing mish-mash of sites on different technologies with inconsistent branding/look and feel.

Clearly a key challenge here is satisfying the diverse needs of so many stakeholders. So a cornerstone of the platform is that sites can be tailored somewhat, so each country has some flexibility to communicate with their audience in the way they think is best. We effectively give the authors a set of page templates and building blocks, and a system which governs how the blocks can fit together so that the user experience will still ‘make sense’. Needless to say, a lot of analysis and consideration has gone into this – both in terms of what functionality was needed but also user journeys and navigation through the site, and the experience architects on our side (LBi) played a vital role here. There are many aspects to the project I could zone in on, but since I want to focus on the implementation details here, I’ll briefly list some of these building block requirements before showing how we did it.

Key requirements/challenges:

In order to create the different page types, we needed around 15 page layouts, including these:

  • Home page
  • Channel hub/Alternative hub/Sub home – these are different template options for ‘2nd and 3rd level’ pages 
  • Content page
  • Product page
  • Media release
  • List – provides links to a series of related pages
  • Etc.

And whilst some aspects of page functionality was ‘fixed’ on the template, there were many other items which were optional – these were to be added to pages by the authors, either in a ‘web part’ kind of way or perhaps something else. Some examples of these optional ‘page modules’ were:

  • ‘Hero’ feature – used to highlight something on prominent pages with an image/flash/text
  • Right-hand promo
  • Content editor module – allows an author to enter arbitrary content, but for reasons which will become clearer we developed an interesting custom control which is kind of a cross between a publishing HtmlField and a Content Editor web part (covered later)
  • Generic content module – rolls-up formatted content/links to a selected page
  • List/tabbed list – provides links to a series of related pages
  • Dynamic share price – displays latest stock price based on web service call
  • Product selector – using AJAX cascading dropdowns to filter products
  • Etc.

Although there were lots of other challenges (such as multi-lingual content, packaging/documenting every deployment aspect so the hosting company could deploy etc.!), I felt that building the ‘framework’ could be more challenging than individual functionality bits. To help frame what you’ll read next, some initial questions we had for the implementation were:

  • How do these optional bits of functionality get added to the page? As web parts, or something else?
  • How do we get accessibility-compliance if web parts?
  • How do we provide configuration if not web parts?
  • How do we restrict which modules can be used where (as per the specification)?
  • Since we’re in a ‘flexible’ publishing site, how do we determine which fields are needed on the content types? Does each content type need to have all the possible fields the author might choose to add?
  • If we are working with publishing controls, how would we bind the dynamically added control to the ‘back-end’ publishing field on the content type?

The implementation

As well as the optional page modules, most of the templates had a classic set of publishing fields. After looking at custom approaches, we concluded the web part framework had a lot going for it for the optional stuff – clearly we could avoid building a user interface to pick the module from a list/add to page/allow configuration of properties specific to the module, and also get drag and drop (amongst other things) as an added bonus. The concept of web part zones – as a container where one or more modules could be placed – was also important to our page structure.

Another challenge for the optional modules was where to store the data. If they were publishing fields, we would need every possible module to have a corresponding field on every possible content type, and this was pretty impractical when looking at the spec. Web parts, of course, use a different model and the framework takes care of data storage regardless of how many controls are on the page.

On the downside, a key thing to remember with web parts in publishing pages is that web part data is (by definition) not stored in publishing fields, and therefore isn’t versioned in the same way. After discussing with the client, in our case this proved to not have as big an impact as we initially thought, due to the split and nature of what content would be stored in publishing fields vs. what would be web parts. So, having the client’s acceptance of this trade-off, we went with web parts and came up with these solution elements:

  1. Module matrix

    This comprised two SharePoint lists which contained the ‘rules matrix’, to enforce the design team’s specification of what functionality could be used on which page type. Effectively the data provides the mapping of modules and page layouts. Being list data, it meant that it could be easily updated by the central team if a policy change was required. This data was consumed by our base web part (point 4).

  2. SmartPart-like approach, but with web part properties

    We wanted the actual functionality of our web parts to be implemented in user controls, for the typical reason of avoiding building HTML in C# code (wrong on so many levels!). This is obviously what the SmartPart does using LoadControl(), but we had the additional requirement of needing to pass web part property values to our user controls – this meant we could use the familiar ‘tool part’ interface (i.e. setting web part properties in the right-hand pane) for control configuration. 

    In our model, each user control has a corresponding ‘wrapper’ web part/tool part which understands which properties are required and how to build the properties UI. In the web part’s OnInit() method, values are passed from the web part properties to the user control so that the latter is initialized ready to do it’s processing.

  3. Base web part/base tool part

    All our web parts/tool parts were derived from our custom classes which abstracted some responsibilities. Since we couldn’t easily change the web part picker screen to only display appropriate web parts for the zone the author had selected, we built the check into the base web part – if an ‘invalid’ web part was added, the web part renders nothing in presentation mode but in edit mode we display a message to the author like this:

    ModuleNotValidMessage 

    Adding too many web parts to a zone (count determined in the module matrix data) would have a similar effect.

  4. Combine interface of publishing field controls with web part storage

    Having decided to use web parts for our control architecture, we had one requirement for something similar to the standard Content Editor web part (CEWP). However, this control is pretty lame compared to the MOSS publishing HtmlField, and we quickly established our client needed more than the basic CEWP. So we combined the bits we wanted from both – the front end control used by the publishing field type (the RichHtmlField control), but the backing store of web part storage rather than a publishing field. This meant authors could add multiple instances of this optional module to their page (and get the nice editing experience), but because it’s a web part we didn’t need to worry about having a corresponding set of fields on each possible content type. In code/integration terms it’s the same approach, but in the end we actually swapped the standard MOSS control for the control which fronts Telerik’s RADEditor field since the client wanted to move to this: 

    CustomContentEditorWebPart  

    Also note use of another control typically used with publishing fields here, the AssetUrlSelector – this provides the ‘Browse…’ button shown above, and can be used to provide a friendly way for an author to browse to a file.

  5. Control adapter for WebPartZone for accessibility compliance

    Since web parts normally render with a stack of nested HTML tables which won’t validate against AA, action needs to be taken to remedy this if accessibility is a design goal. However this isn’t necessarily a big deal – the approach is that you ‘correct’ the HTML for the WebPartZone control in presentation mode only, thus leaving the tables intact in edit mode for all the web part editing framework stuff which needs to happen. You do lose the client-side Web Part Services Component (WPSC) API doing this, but we had no requirement for it anyway (I rarely see it used). I initially assumed I’d have to write a control adapter to do this, but I found that David Schneider has already done the job – this works fine. It’s also possible the latest version of the AKS has one, can’t remember if I checked.
  6. Present only our web parts in the web part picker

    Since this is a highly bespoke WCM platform rather than a standard collaboration environment, we don’t want to see any of the standard web parts in the picker for these sites. Two steps to this one:

    – delete all the .webpart files from the web part galleries in the sites (N.B. we used Kivati for rolling out such changes across all the site collections – more on this in the future). However, doing this will still leave you with ListView web parts for all the lists/libraries in your site, so you also need to..
    – ensure all your WebPartZone declarations have the little documented ‘QuickAdd-ShowListsAndLibraries’ property set to false:

    <WebPartPages:WebPartZone id="g_AB07678E486C46bc962DFC8446A6CD13" runat="server" title="Zone 1" QuickAdd-ShowListsAndLibraries="false" />

    Authors are then not confused by any standard web parts which aren’t appropriate for our scenario:

    StrippedWebPartPicker

  7. Remove unnecessary options when editing web part properties (tool parts)

    Finally, we do a bit of work with the accompanying tool parts (for properties editing) for our web parts to avoid confusing our authors with options which won’t take effect. As an example, for a web part which looks like this in presentation mode:

    ProductSelectorModule 

  8. The tool part looks like this:

    ProductSelectorToolPart

    In case you’re wondering what to look at, it’s that we’ve removed the standard options SharePoint would normally provide for every web part (such as chrome style etc.), since we want to control these to ensure proper formatting. Normally we’d have these sections at the bottom of the tool part:

    RemovedToolPartOptions

Summary

There are many ways SharePoint’s web part framework can be extended, and here I’m only showing the path we followed. For a requirement such as our client’s, web parts provided a great starting point, perhaps showing there can sometimes be a place for web parts in an accessible publishing site so long as the trade-offs are understood and accepted.

In part 2 of this series we’ll look at issues encountered and their resolutions.