Category Archives: 12386

Strong-typed Metadata

Your code is code and your code is data.

Metaprogramming opens up worlds where you care very much that your code is data. Editor enhancements open up worlds where you care very much that your code is data. Visualizations open up worlds where you care very much that your code is data. And I think that’s only the beginning.

There’s nothing really new about thinking of code as data. Your compiler does it, metaprogramming techniques do it, and delegates and functional programming do it.

So, let’s make your code data. Living breathing strongly-typed data. Strong typing means describing the code in terms of the underlying problem and providing this view as a first class citizen rather than a passing convenience.

Describing the Underlying Problem

I’ll use logging as an example, because the simpler problem of PropertyChanged just happens to have an underlying problem of classes and properties, making it nearly impossible to think about with appropriate abstractions. Class/property/method is only interesting if the underlying problem is about classes, properties and methods.

The logging problem is not class/method – it’s log/log event. When you strongly type the metadata to classes that describe the problem being solved you can reason about code in a much more effective manner. Alternate examples would be classes that express a service, a UI, a stream or an input device like a machine.

I use EventSource for logging, but my metadata describes the problem in a more generalized way – it describes it as a SemanticLog. A SemanticLog looks like a class, and once you create metadata from it, you can create any logging system you want.

Your application has a handful of conceptual groups like this. Each conceptual group has a finite appropriate types of customization. Your application problem also has a small number of truly unique classes.

Treating Metadata as a First Class Citizen

In the past, metadata has been a messy affair. The actual metadata description of the underlying patterns of your application have been sufficiently difficult to extract that you’ve had no reason to care,. Thus, tools like the compiler that treated your code as data simply created the data view it needed and tossed in out as rubbish when it was done.

The .NET Compiler Platform, Roslyn, stops throwing away its data view. It exposes it for us to play with.

Usage Examples

I’m interested in strongly typed metadata to write templates for metaprogramming. I want these template to be independent of how you are running them – whether they are part of code generation, metaprogramming, a code refactoring or whatever. I also want these templates to be independent of how the metadata is loaded.

Strongly typed metadata works today in T4 templates. My CodeFirstMetadata project has examples.

I’m starting work on expansion first templates and there are many other ways to use strong-typed metadata – both for other metaprogramming techniques and completely different uses. One of the reasons I’m so excited about this project is to see what interesting things people do, once their code is in a strong-typed form. At the very least, I think it will be an approach to visualizations and ensuring your code follows expected patterns. It will be better at ensuring large scale patterns than code analysis rules. Whew! So much fun work to do!!!

Strong-typed Metadata in a T4 Template

Here’s a sample of strong typing in a T4 template

 
image
 



There’s some gunk at the top to add some assemblies and some using statements for the template itself. The important piece at the top is that the class created by this template is a generic type with a type argument – CodeFirstSemanticLog – that is a strong-typed metadata class. Thus the Meta property of the CodeFirstT4CSharpBase class is a SemanticLog class and understands concepts specific to the SemanticLog, like IncludesInterface. I’ve removed a few variable declarations that are specific to the included T4 files.

Killer Feature for VNext – Language Embedded DSL

VS 2010 is nearly out the door, so it’s time to start fantasizing about killer features in the next version of .NET and Visual Studio.


The feature I want to see is “embedded DSL.”


Like many killer features, success comes from doing in a great way something you can already do halfway. So, I’ll demonstrate this feature in relation to T4 and a MEF scenario where it’s useful today. But before I get there…


I want to see DSL embedded in a language – maybe VB since it oddly enough has become the platform for experimentation, and since we’ve already started embedding other stuff in the language (XML). DSL implies particular syntax issues, but ultimately its metadata for code generation. I want to do code generation that is under my control in a fragmented way within the scope of normal code. Here’s a potential syntactic example:


Public Sub New()
‘ Normal Code
End Sub
{PropertyPattern}
{Property Name=”DisplayName” Type=String}
{Property Name=”DataName” Type=String}
{End PropertyPattern}


‘ More normal code


In this simplistic DSL, the Property metadata is just a pretty syntax for filling data into an interface. Wait, wait, you can’t stuff data into an interface. Right. You need an implementation which you can discover in real time (MEF anyone?) and that implementation can provide defaults for all the other values! And an include pattern here should let you reuse metadata defined once in your application.


PropertyPattern refers to a generation template – like an extended version of T4 – that can output real code in response to the metadata that’s passed. It’s an extended version because it’s strongly typed to the metadata defined via the metadata/DSL interface. The pattern is also discovered (MEF anyone?), which, well just trust me on this, allows a full governance model (customizing and governance on templates is one of my specialties, but let’s not geek out on that right in the middle of a hot fantasy). In simple terms, the governance model means a fallback mechanism through project (assembly), to group, to organization, to defaults/in the box.


Before I go further, let me say this is not a replacement for application generation. Application generation and code generation aren’t the same thing. Application generation is either a closely linked set of templates where the interrelations are as important as the templates, or application generation is architecture generation which is a new and emerging field.


Back, to the fantasy, because it’s an important part of a bigger picture of changing how we write applications…


Is this fantasy just Kathleen on too much MEF?


Actually no MEF required at all this morning… let me show you how to do this today – no compiler changes, no new dependencies, you can do this right now in VS 2008 (if you download the DSL Toolkit so you have T4, which you already did anyway, right?).


Well, OK, just a little MEF to get our morning started… the scenario is a MEF scenario, although there is no MEF in the solution. When you work with MEF and you want to create a MEF friendly interface for later discovery, you create an interface that a part will later fulfill, a separate interface of composition metadata, and a custom export attribute, which allows you to define a part just by implementing the first interface and tossing on the attribute. Whew! This means there is an annoying tedious (but technically elegant and necessary) pattern that I’ve implemented perhaps a billion times. What’s worse, the pattern obscures information about the interface, which of course I should also include in XML Documentation so I’m not in the code in the first place checking out the info – but see, more internal redundancy. And oh, by the way, if you screw up the pattern, the bugs can be very hard to track. Let’s fix it…


[NOTE: The implementation jumped back and forth between VB and C#. In VB, I’d implement this as XML literals which makes a more concise syntax with way less code in the included template, but T4 barfed at the XML literals and I didn’t feel like bothering with it today. Without XML literals, C# has better syntax because of the collection initializers which don’t make it into VB until 2010. There is no relation between the syntax of the DSL and the output syntax. I happen to be outputting VB.]


I created a T4 template. If you’re in VS2010, this is a normal Text Template, not a preprocessed one. If you’re in 2008, create a text file and give it a .tt extension. Also copy in the MefInterfaceDsl.t4 file that I’ve attached into your project. (I tested in VS 2008, in VS 2010, you may need to change the extension to .tx) Yes, your DSL will be written in T4 and yes, your DSL T4 template will have a different extension than the supporting one. That’s because Visual Studio outputs code for a file with a .tt extension and does not for one with a .t4 extension. You want output only from your DSL T4.


Here’s the template, then more talk:


<#@ template debug=”false” hostspecific=”false” language=”C#v3.5″ #>
<#@ assembly name=”System.Core” #>
<#@ output extension=”.vb” #>
<#@ include file=”MefInterfaceDsl.t4″ #>
<#=
         new Interface()
         {
            Name = “ISearchModelBase”,
            Scope = Scope.Public,
            CompositionInfo =
            {
               new Property() {Name=”TargetType”, PropertyType=”Type”}
            },
            Members =
            {
               new Property() {Name=”DisplayName”, PropertyType=”string”},
               new Property() {Name=”DataName”, PropertyType=”string”}
            }
         }.Output()
#>


Wow, is that really a DSL? Yes, but I won’t claim it’s a very good one. It’s a hack to allow the concept to work in VS 2008 and VS 2010, in hopes that we can get an elegant syntax in VS 20Next.


The initial four lines are a necessary T4 distraction. I’ve stated that the template language is C# with 3.5 extensions (not necessary in VS 2010). I’ve included core because it makes 3.5 work. I’ve stated that output will be in .vb. The actual syntax for the output is in the include file. Remember this is a working sample, if you wish to use this, you’ll want to enhance the MefInterfaceDsl.t4, including rewriting it to output C# if that’s your current flavor preference.


The include file has a class named Interface, Property and a few others. Initializers and collection initializers build the graph for the ISearchModelBase interface which has two properties, and one composition metadata property. The Interface class has an Output method that returns a string with the code output. Visual Studio places this output in a dependent file (select Show All Files to see this in VB). I included it below so you don’t have to run a project just to see the output.


Since the artifact is generated, I don’t have to remember the pattern and I’ll never be bit by forgetting to set AllowMultiple=false. Since the DSL/metadata is in the project beside the artifact, I can find an work with it (this would not be appropriate for application generation DSL/metadata whose artifacts spanned many projects, solution, and platforms).


So why is a normal T4 template a DSL? Because a DSL is a way to define generation information (metadata) in a way that is friendly to the human, followed by artifact generation.


I want this extended to be a true embedded part of the language to avoid these limitations and supply these features (and probably a bunch more stuff I haven’t thought of):


  • - Can be any part of any normal code file
    • Not limited to entire files (although the T4 output can be partials)
    • The DSL/metadata is a holistic part of the code
  • - Artifact patterns (templates) are discoverable (can be anywhere by anyone)
  • - DSL/metadata patterns are discoverable (default values and pattern extensions
  • - Intellisense on the DSL
  • - Better syntax (drop the new and other class residuals)
  • - No reliance on file location (the T4 support file must be in relation to your project)
  • - No ugly opening stuff about T4 unrelated to the task at hand
  • - Standard extensible metadata/DSL patterns provided
  • - Standard extensible artifact patterns provided

 


Here’s the output:


Option Strict On
Option Explicit On
Option Infer On


Imports System
Imports System.Collections.Generic
Imports System.Linq
Imports System.ComponentModel.Composition


public Interface ISearchModelBase
   Property DisplayName As string
   Property DataName As string
End Interface
public Interface ISearchModelBaseComposition
   Readonly Property TargetType As Type
End Interface


< MetadataAttribute() > _
< AttributeUsage(AttributeTargets.Class, AllowMultiple:=False) > _
public Class SearchModelBaseAttribute
   Inherits ExportAttribute
   Implements ISearchModelBaseComposition


   Public Sub New( ByVal targetType As Type)
      MyBase.New(GetType(ISearchModelBase))
      _targetType = targetType
   End Sub


   Private _targetType As Type
   Public Readonly Property TargetType As Type Implements ISearchModelBaseComposition.TargetType
      Get
         Return _targetType
      End Get
   End Property


End Class