System.AddIn Scenarios

I started writing an entry about the System.AddIn pipeline and realized that post was not helpful without having a clue why you might be interested in System.AddIn. So, let me start here.


System.AddIn is designed to isolate portions of your application. It was designed for the scenario of AddIns. Despite the naming confusion, this has nothing to do with Visual Studio. System.AddIn is all about extending your application.


 Let’s say you have a commercial application and some clients want to do extra stuff. Giving them back channel access to your database should give you an anxiety attack. With System.AddIn, you just build interfaces with very specific access to your business layer in a way you control. You create an API, and System.AddIn manages it for you.


Understand – with System.Addin and your API, later programmers do not alter your code. They do not even recompile or have the source code. They access your application on channels you define. And you can sandbox them as much as you like.


Don’t yawn. System.AddIn manages application domain boundaries so poorly behaved add-ins don’t crash your application making you look bad. That sand boxing thing is pretty important if there are third parties involved. And System.AddIn assumes your application and the add-in will independently version. You can version in a way that won’t break applications clients or third parties wrote for your application.


Actually, yes, you could have done all this yourself. .NET 3.5 sits on top of 2.0 so I know of nothing changed in the CLR that suddenly opens a new day. You could have written HashSet as well. Heck, there’s a lot in the framework you could have done yourself. But why would you want to. System.AddIn has Microsoft’s backing and commitment so it’s the way to provide application extensibility to your application going forward.


I believe System.AddIn is important beyond retail applications. Here are a few key scenarios:


1.       You need to let your clients or third parties add functionality


2.       Internally, you need to logically branch your application for multiple client’s business logic


3.       Different clients need different WPF user interfaces


4.       You need to unload code so it can be changed without closing the application


5.       Different segments of your application will evolve at different rates


6.       Interacting with a semantic based organization wide service bus


The second and third are similar, but I wanted to call out the fact that the WinForms model does not play well with System.AddIn. There is a workaround on the team blog involving hosting WinForm pieces in WPF on both sides. I haven’t done that yet but it looks like an interesting approach.


The fourth one is what drew me to System.AddIn. The new version of my code generation harness will have the best possible support for VB 9 XML Literal Code Generation. This means templates will be in assemblies and you’ll want to load the harness, run, change templates, run again. Stopping the harness for a small change would be a major pain. I need to unload after each harness run.


That’s not the only time I’ve wanted to unload an assembly. The last time was when we rejected doing customization through satellite assemblies because we could not update them without shutting down the application – and users would be updating localization strings on the fly. Honestly, we’d probably have a tough time anyway because satellite assemblies aren’t setup to run through System.AddIn and you would not want the hit of crossing the isolation boundaries. But we didn’t explore further because updating at runtime of any assembly required app domain isolation which brings in a slew of issues that the CLR team solved with System.AddIn.


The fifth will probably be the most important role of System.AddIn to most programmers within the next few years. One of the side benefits of services is that they allow independent evolution of different segments of your applications, with the corresponding versioning challenges. But what if you are working with an app that does not need service boundaries for deployment, or needs independent evolution within a single service boundary? Why use WCF if what you really need is versioning support? The concept of what an application is has changed in service based world, System.AddIn allows that same shift in thinking without service boundaries – it’s a different kind of API boundary.


I’m going to postpone discussing the last until I can do a really good job of it, and let my friend David McComb look over it. Dave expanded my view of coding in the most amazing way because of his insights on the range of the ecosystem from assembly bytes to organizations. I’d darn near work for free if I get another chance to work for Semantic Arts. My fingers are just itching to keep going on this…but I really need to wait and do it right.


These are some good reasons to keep an eye on System.Addin. There may be more. Prior to the PipelineBuilder release, System.AddIn was such a pain to use that I couldn’t recommend it. The beauty is that System.AddIn went from an absolute pain to downright easy in one day. It’s a tool you should understand for your arsenal, especially if you claim to be anything resembling an architect.


Look for more in upcoming posts…


 

Losing Trust and Moving On

I have been through a difficult week.


If you know me, you know I’m an optimist and a trusting person. I trust the world so much that I refused to teach my children to be afraid of it, staring down a pre-school teacher who was horrified that Peter approached a strange elder at the mall on an outing (and of course made both of their days).


Thus, when the world slaps me with its unfairness I do not take it well. The automobile dealership put a transmission with a cracked housing into my car in October. On the two occasions I took the car back in for a fluid leak they failed to tell the transmission housing was cracked, although I specifically asked if the leak was a sign of a larger problem, and they had just attempted to repair the damage with silicon. I have always had a hard time with mechanics because my experience has been negative, so it was even more difficult that I trusted them and they so betrayed that trust.


I have a more realistic view now that the being a consumer leaves you very few rights since the system is built and maintained by people with the money and power to keep it running for their own benefit. I do not suggest that any mechanic would knowingly put in a broken part – I think it more likely they simply failed to inspect it after they assured me they would. However, auto dealerships (at least in Colorado) have built a system so completely to their benefit that they make more money if they put a bad part in your automobile than if it is good. They charge you labor twice. If I have another mechanic look at their work, I have no right to charge them for the inspection even if it turns out the work was done improperly.


Before you have work done, find out what is warrantied and what the conditions of the warranty are. It is likely that on a failure you will pay a second labor charge, and possible that if the first part is broken and replaced, the warranty still ends relative to the first repair date. It is probable that you have no rights for anyone but the original shop to do any warranty work.


When the world slaps me, I get emotional. I cry; I eat too much; I am not pleasant to be around. Part of healing is seeing beyond – it’s just a car, and in the end I am only out about $250 due to their actions.  I know I do not want to do business with them again. This is a valuable lesson, perhaps worth $250. Since October I have taken my other two vehicles to a different mechanic and gained respect for him. He treats me like I am not an idiot, and excitedly shows me how bad a part is that he pulled off. A long time ago I worked on cars. I know what a spark plug is supposed to look like. This mechanic inspected the transmission when I suspected the dealership was trying to cover up its earlier work. Going from having two repair shops I trust to one is a survivable change. Going from one to none would have been, well, I’m not sure how I would have worked it out.


And far more than that, – the really important thing is that despite being difficult to be around, I had the support of an amazing array of people. My children’s father went far beyond what anyone could have expected in interacting with the dealership when I did not feel I could without screaming hysterically at them, and he eventually negotiated the best settlement we could get. My new mechanic has been awesome spending much time on the phone with me, my children’s father, and the dealership. My boyfriend happens to be a forensic scientist and photographer specializing in automobiles and recorded the evidence. When I had to take the old transmission to the dealership, my son (not a morning person) drove it up there so I would not have to interact with them. Other friends also offered emotional support. I was not alone. I was instead supported by the right people with the right skills willing to help me get to the other side of this, get my car back, and move on.


Moving on is about finding the good in a bad experience. It’s about silver linings and lessons. It’s about feeling the love and support you’re offered, alongside the negative. It’s also about forgiveness. I will not forget what happened, nor have I completely let go of clever (and legal) ideas for revenge. But I’m not holding this in my heart. I’m moving on.


And oh my goodness! If in all the troubles of the world, I say “I’ve been through a difficult week” because of anything to do with a car, let me be immensely grateful.


Thanks for listening. I needed to share.

 (If you live in the Fort Collins area and need the name of either the dealership or my new mechanic, feel free to ask me in email. ) 

 

First Experience with Pipeline Builder

Wow!


System.AddIn the easy way


Build and interface, press a button, build an add in, build a host.


Wow!


So, the pipeline builder tool really works.


You have to do a couple of things not mentioned in the documentation I used. Actually the documentation needs a little work, but its there and not too bad.


Follow the directions for the VS tool


Also


Copy the Blank project to a place to work


Rename the project as desired, or test first then rename later (I am mortified by the idea of millions of “Blank Pipeline Project” assemblies floating aorund)


Add an interface to the Contracts file. It must have the AddInContract attribute and inherit from IContract. If you want to borrow mine:

using System;using System.Collections.Generic;using System.Linq;using System.Text;using System.AddIn.Contract;using PipelineHints;using System.AddIn.Pipeline;  namespace KadGen.FirstPipelineBuilderTest.Contracts{   [AddInContract()]   public interface GenerateFiles : IContract   {      string GenerateForObject(string objectName,  ref string fileName);      string GenerateForStoredProc(string objectName, ref string fileName);      string GenerateForDataTable(string objectName, ref string fileName);   }} 

If you leave off the IContract you’ll get projects but no generated cs files. Ask me how I know!


The only downside is that a lot of you will never experience the pain of creating them by hand and will be therefore so jaded that you don’t realize that the release of this tool is cause for world-wide celebrations!


I’ll blog about using this with VB after I’ve tested that more.


 


 

Pipeline Builder is Released

Yippee! I love it when someone else does my work for me.Part way through building a code generator for the otherwise nightmarish task of creating add-ins for use with System.Addin, a little bird told me that the CLR team was slaying the same dragon. Not one to stand in the way of their being heroes, and aware that they could do things in a far better way because of their knowledge, I shelved that project for a week or so.So, the pipeline builder tool is up. If you don’t know what I’m talking about, you might want to check this blog entry. It will take a while longer to catch on, but many of you have System.Addin lurking in your very near future. Pipeline Builder makes that practical.

 

Taping .NET Rocks TV

Taping my first episode of .NET Rocks TV was fun. Carl’s got some serious editing to do, and when he’s done I think the show will be great.


Predictably I didn’t have time to finish with what I wanted to show you about VB 9 and C# 3.0. So, Carl agreed to let me tape Part 2 of the show next week.


Part1 covers nullable operators in VB and how they differ from the C# operators, ternary and coalesce operators in VB and C#, extension methods, hashsets, partial classes and inferred typing. That’s going to be great groundwork for Part 2 which will cover initializers, lambda expressions, anonymous types and LINQ. Many of the compiler features were introduced to support LINQ, but they are valuable by themselves. I’ll look at core LINQ – sometimes called LINQ to Objects (IEnumerable actually).


Both shows dig into some grimy details while keeping the overall tone focused to why you care about these new features.


 

.NET Rocks and .NET Rocks TV

If you want to watch my first attempt at a webcast under Carl Franklin’s supervision – check out the .NET Rocks TV episode I’ll be taping next Wednesday. It should be posted as week or two after that.


At least the content will be fun! Carl’s letting me show off .NET 3.5 and Visual Studio 2008. I know, I know you’ve seen all the cool tricks. But what you probably have not seen is why these new features matter (and which ones probably don’t matter very much). You’ll see tricks on how to actually use the new tools in a way that improves yoru development.


 Also, the .NET Rocks radio episode we taped before Christmas was posted yesterday. Check it out and let me know what you think.

Why You Care About System.AddIn

When I was fighting with AppDomains to support XML Linq code generation in my new Workflow based code generator, Bill McCarthy said “Hey did you look at System.AddIn” and I said “No, silly I’m not writing add-ins.”


Well, a few months later, I’m still trying to make it work, and have come to think it’s worth the trouble. So, first, what System.AddIn namespace offers, then why it’s so painful, then what I’m doing to fix your pain.


Simply put, System.AddIn provides abstracted/isolated app domain access. AppDomains are the boundary at which some security stuff happens, and the unit which must be unloaded as a group. You can load individual assemblies into an app domain, but to unload them, you need to unload the entire app domain.


There are a few scenarios where this is important – sandboxing code you’re running as a plug-in to your application being the one the designers had in mind. I want to use it so I can load my code generator and have it recognize changes in .NET assemblies that are generating code. With my first tool, I never solved this problem because I didn’t think brute force code generation prior to XML literals in Visual Basic made very much sense. You had a lot of the problems with XSLT (whitespace) and a nearly complete inability to search your templates (since we cannot search separately in quoted text. XML literal code generation is the best way yet to generate code – as powerful as XSLT and as easy as CodeSmith. Anyway, I can get carried away on that – it’s why I was willing to invest heavily in System.AddIn.


Along the way, I gained great respect for the complex model that supplies isolation/abstraction. If you’ve ever played  with plug-ins you know that the first version of your app and its plug-ins is OK, but keeping things in sync while multiple synergistic applications evolve is nearly impossible. The isolation model means the host only speaks to an adapter, and the add-in only needs to speak to an adapter. The adapter’s functionality and the contract can change in whatever manner is needed. This model, combined with the app domain management may lead System.AddIn to have an important role in your application if your application needs to provide variants for individual clients.


Hopefully you have a good idea what sorts of things clients are going to want to customize, and you place this into an API you hit via the add-in model. If you got it 75% correct out of the chute, it would be a miracle, so the capacity for change built into the isolation model is what actually makes this work


Literally, you load code on the fly, with whatever security limitations you want, with the ability to unload at your convenience, and pick the correct code from what’s available in a specific directory location. Cool huh!


In WinForms, the WinForms threading model prohibits UI’s in the add-in. I understand this is fixed in WPF, although I haven’t yet written a WPF add-in user interface.


So, now that you have some idea why System.AddIn is worth the trouble, why is it so painful. How could I have possibly spent so long getting it running in a sample (I just output a single quoted string right now). To provide the isolation there is a minimum of seven projects/assemblies involved. These must be deployed in a very specific directory structure for the AddIn system to find the pieces it needs when it needs it. Then there is the error reporting problem – I’ve blogged about a particularly nasty “The target application domain has been unloaded” error.  So, once you hold your mouth just right, and all your code is perfect, it’s cool. But how many of you right perfect code? And what’s this about an easy maintenance model if you have to change SEVEN assemblies to alter the API.


I’m working on an article and tool for my column in Visual Studio Magazine that will take either metadata for the API, or the interface and build the simple pass through model. This gets you started. Later when you have interface changes, the isolation model pays for itself, but at that point you understand what’s happening.


It’s going to be a pretty cool example of the “overwrite until edited” mode that my tool supports. Before I’ve used this for editable files that were pretty much empty. Now, I want to separate changes due to metadata changes – that could be significant – from those for actual mapping you did in the adapters. With luck partial methods will lead to a pretty robust set of code you can alter as you need, while still generating the main API stream.


I find it very cool to see so many fragments coming together.


 

System.Addin “The target application domain has been unloaded”

I have just escaped an Alice in Wonderland type maze with System.AddIn namespace. It’s a bit of a nightmare to create the plumbing for an add-in so stay tuned in Visual Studio Magazine for a column on how to generate this so you only do the logical work


a)      Define the add-in API


b)      Write the add-in


If you have created one of these in .NET 3.5 you know how tremendously painful it can be. The particular gotcha that I spent numerous days spread over the last three months is that the host adapter much set the contract handle:

    Private _handle As System.AddIn.Pipeline.ContractHandle    Public Sub New(ByVal contract As ILinqGenerator)      Me.contract = contract      _handle = New System.AddIn.Pipeline.ContractHandle(contract)   End Sub 

Now, you might be like me and look at that line of code and say “Hey, no one anywhere is using the _handle variable, so why not just drop it.” That would be consistent with the fact the same sample has numerous Friend methods that are never used. Heck, FxCop is even going to tell me to get rid of this line of code.


However, it’s required. Without this line of code, you get the error “The target application domain has been unloaded.”


But I said I spent days on this, over months which meant I kept coming back its fair to ask why? But that’s the subject for another post.