Killer Feature for VNext – Language Embedded DSL

VS 2010 is nearly out the door, so it’s time to start fantasizing about killer features in the next version of .NET and Visual Studio.


The feature I want to see is “embedded DSL.”


Like many killer features, success comes from doing in a great way something you can already do halfway. So, I’ll demonstrate this feature in relation to T4 and a MEF scenario where it’s useful today. But before I get there…


I want to see DSL embedded in a language – maybe VB since it oddly enough has become the platform for experimentation, and since we’ve already started embedding other stuff in the language (XML). DSL implies particular syntax issues, but ultimately its metadata for code generation. I want to do code generation that is under my control in a fragmented way within the scope of normal code. Here’s a potential syntactic example:


Public Sub New()
‘ Normal Code
End Sub
{PropertyPattern}
{Property Name=”DisplayName” Type=String}
{Property Name=”DataName” Type=String}
{End PropertyPattern}


‘ More normal code


In this simplistic DSL, the Property metadata is just a pretty syntax for filling data into an interface. Wait, wait, you can’t stuff data into an interface. Right. You need an implementation which you can discover in real time (MEF anyone?) and that implementation can provide defaults for all the other values! And an include pattern here should let you reuse metadata defined once in your application.


PropertyPattern refers to a generation template – like an extended version of T4 – that can output real code in response to the metadata that’s passed. It’s an extended version because it’s strongly typed to the metadata defined via the metadata/DSL interface. The pattern is also discovered (MEF anyone?), which, well just trust me on this, allows a full governance model (customizing and governance on templates is one of my specialties, but let’s not geek out on that right in the middle of a hot fantasy). In simple terms, the governance model means a fallback mechanism through project (assembly), to group, to organization, to defaults/in the box.


Before I go further, let me say this is not a replacement for application generation. Application generation and code generation aren’t the same thing. Application generation is either a closely linked set of templates where the interrelations are as important as the templates, or application generation is architecture generation which is a new and emerging field.


Back, to the fantasy, because it’s an important part of a bigger picture of changing how we write applications…


Is this fantasy just Kathleen on too much MEF?


Actually no MEF required at all this morning… let me show you how to do this today – no compiler changes, no new dependencies, you can do this right now in VS 2008 (if you download the DSL Toolkit so you have T4, which you already did anyway, right?).


Well, OK, just a little MEF to get our morning started… the scenario is a MEF scenario, although there is no MEF in the solution. When you work with MEF and you want to create a MEF friendly interface for later discovery, you create an interface that a part will later fulfill, a separate interface of composition metadata, and a custom export attribute, which allows you to define a part just by implementing the first interface and tossing on the attribute. Whew! This means there is an annoying tedious (but technically elegant and necessary) pattern that I’ve implemented perhaps a billion times. What’s worse, the pattern obscures information about the interface, which of course I should also include in XML Documentation so I’m not in the code in the first place checking out the info – but see, more internal redundancy. And oh, by the way, if you screw up the pattern, the bugs can be very hard to track. Let’s fix it…


[NOTE: The implementation jumped back and forth between VB and C#. In VB, I’d implement this as XML literals which makes a more concise syntax with way less code in the included template, but T4 barfed at the XML literals and I didn’t feel like bothering with it today. Without XML literals, C# has better syntax because of the collection initializers which don’t make it into VB until 2010. There is no relation between the syntax of the DSL and the output syntax. I happen to be outputting VB.]


I created a T4 template. If you’re in VS2010, this is a normal Text Template, not a preprocessed one. If you’re in 2008, create a text file and give it a .tt extension. Also copy in the MefInterfaceDsl.t4 file that I’ve attached into your project. (I tested in VS 2008, in VS 2010, you may need to change the extension to .tx) Yes, your DSL will be written in T4 and yes, your DSL T4 template will have a different extension than the supporting one. That’s because Visual Studio outputs code for a file with a .tt extension and does not for one with a .t4 extension. You want output only from your DSL T4.


Here’s the template, then more talk:


<#@ template debug=”false” hostspecific=”false” language=”C#v3.5″ #>
<#@ assembly name=”System.Core” #>
<#@ output extension=”.vb” #>
<#@ include file=”MefInterfaceDsl.t4″ #>
<#=
         new Interface()
         {
            Name = “ISearchModelBase”,
            Scope = Scope.Public,
            CompositionInfo =
            {
               new Property() {Name=”TargetType”, PropertyType=”Type”}
            },
            Members =
            {
               new Property() {Name=”DisplayName”, PropertyType=”string”},
               new Property() {Name=”DataName”, PropertyType=”string”}
            }
         }.Output()
#>


Wow, is that really a DSL? Yes, but I won’t claim it’s a very good one. It’s a hack to allow the concept to work in VS 2008 and VS 2010, in hopes that we can get an elegant syntax in VS 20Next.


The initial four lines are a necessary T4 distraction. I’ve stated that the template language is C# with 3.5 extensions (not necessary in VS 2010). I’ve included core because it makes 3.5 work. I’ve stated that output will be in .vb. The actual syntax for the output is in the include file. Remember this is a working sample, if you wish to use this, you’ll want to enhance the MefInterfaceDsl.t4, including rewriting it to output C# if that’s your current flavor preference.


The include file has a class named Interface, Property and a few others. Initializers and collection initializers build the graph for the ISearchModelBase interface which has two properties, and one composition metadata property. The Interface class has an Output method that returns a string with the code output. Visual Studio places this output in a dependent file (select Show All Files to see this in VB). I included it below so you don’t have to run a project just to see the output.


Since the artifact is generated, I don’t have to remember the pattern and I’ll never be bit by forgetting to set AllowMultiple=false. Since the DSL/metadata is in the project beside the artifact, I can find an work with it (this would not be appropriate for application generation DSL/metadata whose artifacts spanned many projects, solution, and platforms).


So why is a normal T4 template a DSL? Because a DSL is a way to define generation information (metadata) in a way that is friendly to the human, followed by artifact generation.


I want this extended to be a true embedded part of the language to avoid these limitations and supply these features (and probably a bunch more stuff I haven’t thought of):


  • - Can be any part of any normal code file
    • Not limited to entire files (although the T4 output can be partials)
    • The DSL/metadata is a holistic part of the code
  • - Artifact patterns (templates) are discoverable (can be anywhere by anyone)
  • - DSL/metadata patterns are discoverable (default values and pattern extensions
  • - Intellisense on the DSL
  • - Better syntax (drop the new and other class residuals)
  • - No reliance on file location (the T4 support file must be in relation to your project)
  • - No ugly opening stuff about T4 unrelated to the task at hand
  • - Standard extensible metadata/DSL patterns provided
  • - Standard extensible artifact patterns provided

 


Here’s the output:


Option Strict On
Option Explicit On
Option Infer On


Imports System
Imports System.Collections.Generic
Imports System.Linq
Imports System.ComponentModel.Composition


public Interface ISearchModelBase
   Property DisplayName As string
   Property DataName As string
End Interface
public Interface ISearchModelBaseComposition
   Readonly Property TargetType As Type
End Interface


< MetadataAttribute() > _
< AttributeUsage(AttributeTargets.Class, AllowMultiple:=False) > _
public Class SearchModelBaseAttribute
   Inherits ExportAttribute
   Implements ISearchModelBaseComposition


   Public Sub New( ByVal targetType As Type)
      MyBase.New(GetType(ISearchModelBase))
      _targetType = targetType
   End Sub


   Private _targetType As Type
   Public Readonly Property TargetType As Type Implements ISearchModelBaseComposition.TargetType
      Get
         Return _targetType
      End Get
   End Property


End Class

9 thoughts on “Killer Feature for VNext – Language Embedded DSL”

  1. Here’s an idea. Xml literals will never be added to C#. But Xml, can be thought of as a dsl (albeit not a very good one). If you add embedded DSL to the platform, the VB’s Xml support can be implemented in terms of the more abstract Dsl support and C# devs can have Dsl support on a per code file basis if they so desire (naturally .NET would ship with an Xml Dsl out of the box).

    Next, let Dsl’s be defined in MGrammar and create deep integration between the CLR type system and MGrammar. So you could have something like this:

    ..define MGrammar named XmlGrammar somewhere..
    and in C# something perhaps like this

    var xml =
    begin XmlGrammar
    …some xml
    end XmlGrammar

    use the MGrammar files to provide syntax highlighting and statement completion of coarse. MEF could be used to provide the interpreter of the given Grammar which could dynamically create code, documents, whatever…

    There are some cool things you could do. On the other hand, I’m not sure that simply adding metaprogramming to .NET wouldn’t give us what we want in a simpler way :)

  2. I agree with you that M is a good approach because it is the textual DSL in the stable. I just wanted to keep this post clear of magic looking semantics or a dependency on any tool.

    To me, this is a route to remain very strongly typed at design time and to keep some limits that metaprogramming wouldn’t have. Sort of wasing instead of jumping in to the deep end.

    I’m not one that buys the “C# will never have XML literals” But I think one reason to delay it was the absence of a clear general purpose “we’re dropping into something else now” syntax. XML would be one such syntax, but it also points out a challenge. XML and C# have fundamentally different syntatic bases, most clearly seen in the handling of nulls. An XML DSL would be trickier to write well than a .NET code producing XML.

    Thanks for your thoughts!

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>