Strategies migrating from Visual Studio add-ins to packages

July 1, 2014

Since the next Visual Studio version (“14”) will remove add-ins, the time has come for those of us still providing our products as add-ins (such as my MZ-Tools productivity add-in) to move to Visual Studio packages. In my view, there are three strategies:

In the first one, you change the minimal number of things in your add-in to convert it to a package. For example, some areas that need to be changed are:

  • Creation of commands (remember that commands are not UI items).
  • Creation of UI items (toolbars, menus, context menus, buttons, etc.).
  • Creation of toolwindows.

Although they are tricky, the Visual Studio package wizard will get you started about those areas. But there is another thing that your add-in surely does: to respond to events to perform some actions. And for that it uses events provided by the automation model (EnvDTE). Furthermore, add-ins are totally based on the automation objects (EnvDTE.DTE, EnvDTE.Solution, EnvDTE.Project and so on). Although you can get an instance of the root DTE object as explained in HOWTO: Get an EnvDTE.DTE instance from a Visual Studio package, there is a another way.

In the second strategy, you get rid of the Visual Studio automation (EnvDTE). The automation model was provided initially in Visual Studio .NET 2002 to support add-ins, macros and wizards. Macros were removed in Visual Studio 2012. And add-ins will be removed in Visual Studio “14”. Microsoft packages don’t use very much EnvDTE. So, which will be the future of EnvDTE? Not a brilliant one, I guess. Likely in some Visual Studio version it will be deprecated and in some other version it will be removed and then you will have to change again your package because of its heavy dependency on EnvDTE. So, you can leverage the migration from a Visual Studio add-in to a package to do things in the natural way of packages, using Visual Studio services and interfaces. They are not easy because 1) there are tons of services and are difficult to discover (in the automation model you can use the Object Browser to discover classes, methods, etc.) and 2) they still show its C++/COM nature in some cases (not a friendly API).

In the third strategy you realize that your Visual Studio extension has a strong dependency on the Visual Studio assemblies, either EnvDTE or the ones used by packages (Microsoft.VisualStudio.*). Furthermore, they can change on each Visual Studio version. To avoid this, my approach is to split my “plug-in” in two assemblies:

  • One assembly has the features of my product and depends only on abstractions such as IHost, ISolution, IProject, IProjectConfiguration, etc. with the methods, properties and events that I define. This assembly only references some assemblies of the .NET Framework. In my case (for the future MZ-Tools 8.0), this assembly has 80% of the code and has only five references: System, System.Data, System.Drawing, System.Windows.Forms and System.Xml.
  • The other assembly is an “adapter” for a Visual Studio version that provides the implementation of the interfaces IHost, ISolution, etc. In my case it has 20% of the code and I have adapters not only for Visual Studio (all versions as add-in), but for the VBA editor (32-bit & 64-bit) of Office, VB 6.0 and VB 5.0. I even have a stub adapter to be used in ultra-fast “integration” tests. So, I only have now to create an adapter for Visual Studio as a package. But the core of my product (80%) is not affected.

To get you the idea, this is the structure of the solution:

MZTools8SolutionExplorer

And my object model has 30 interfaces approx.:

ObjectModel

Needless to say, my implementation of those interfaces in the adapter for VS as package will not use the automation model EnvDTE. In the next posts and articles of the MZ-Tools Articles Series I will explain how to do things in the native way of packages, as I learn about it.


Visual Studio "14" CTP: add-ins are gone

June 5, 2014

I blogged some time ago about Add-ins officially deprecated in Visual Studio 2013 and given that MZ-Tools is still an add-in, after the announcement of Visual Studio “14” CTP I was curious yesterday if it would still support add-ins. I had a very busy day at the office so I couldn’t check it until night at home. In the past it would mean to download the .iso file, setup a virtual machine on my computer, install it, etc. but now with Windows Azure I was already at bed about to sleep when I grasped my iPad, connected to my Windows Azure account, created a virtual machine with the VS 14 CTP from the gallery, installed the Microsoft Remote Desktop client app for iPad, connected to the machine and launched Visual Studio (amazed that all that was possible with a tablet and 10 minutes of time, really cool). Then I clicked the Tools menu and I saw that the Add-In Manager is gone, which means that Visual Studio “14” won’t support add-ins at all and you will have to migrate your add-in to a Visual Studio package. Incidentally, Microsoft has just released the Visual Studio 2013 SDK Samples.


Extending Visual Studio Online

May 30, 2014

Since I adopted Visual Studio Online a few weeks ago, I am more and more happy with it. I have also recently adopted a help desk online software called SmarterTrack (from SmarterTools, the same company that provides SmarterMail) to support questions from users of my MZ-Tools family of add-ins. I am now at the point of automating the creation of TFS work items from some help desk tickets (bugs or feature requests). SmarterTrack allows you to include custom menus in the window of a ticket than can make an http get call (although not a http post call with JSON). But I can make an http get call to some page of my server and then make whatever I want once I extract the parameters.

Fortunately Microsoft made Visual Studio Online fully extensible, as recently announced at TechEd North America 2014. There is a new Visual Studio Online Integrate portal with the scenarios that you can use:

  • Calling Visual Studio Online from any other tool through REST APIs with OAuth.
  • Make Visual Studio Online to call any tool when an event happens (without the need of polling).

There is also a lot of 3rd party tools that provide integration with Visual Studio Online, as explained in this TechEd session:

Using Third-Party ALM Solutions along with Visual Studio Online
http://channel9.msdn.com/Events/TechEd/NorthAmerica/2014/DEV-B368

Investigating how to use the REST APIs of Visual Studio Online I found yesterday this upcoming ebook of MVP fellows Jeff Bramwell and Mike Douglas:

TFSVSOExtensibility

Extending Team Foundation Server And Visual Studio Online
http://www.codesmartnothard.com/2014/05/19/ExtendingTFSAndVisualStudioOnlineEBook.aspx

And to buy it:

https://leanpub.com/tfsapibook

Only some chapters are already written but the table of contents is promising and I am sure it will be great:

Part I – The Basics

Chapter 1: Overview of Team Foundation Server
Chapter 2: A History of Team Foundation Server APIs
Chapter 3: A REST Primer

Part II – Using the Team Foundation APIs

Chapter 4: Our First REST API Call
Chapter 5: Work Items
Chapter 6: Team Foundation Version Control
Chapter 7: Git
Chapter 8: Build Definitions
Chapter 9: Builds
Chapter 10: Workspaces
Chapter 11: Team Room Management
Chapter 12: Team Room Activities
Chapter 13: Reporting and Charts
Chapter 14: Projects and Teams
Chapter 15: Service Hooks

Part III – Other Resources

Chapter 16: Using Fiddler
Chapter 17: Using OAuth

Resources

Given the lack of books about Visual Studio Extensibility (add-ins, packages, DSL, etc.) after all these years it is great that a book about Visual Studio Online Extensibility is appearing so early (and so cheap!).


Visual Studio Express edition doesn’t support extensions (despite its Extension Manager)

May 7, 2014

A question that appears from time to time in forums is whether the Visual Studio Express editions support extensions (packages, add-ins,…) that you create, that is, if they can be installed and can be run.

The short answer: no.

The medium answer: only Microsoft extensions are supported since VS 2012, not your own extensions.

The long answer:

Visual Studio Express editions started with Visual Studio 2005 (Visual Studio .NET 2002 and Visual Studio .NET 2003 didn’t offer them) and each version (2005, 2008, 2010, 2012 and 2013) has offered a different feature set. For example, with VS 2005, 2008 and 2010 you had language-oriented Express editions (Visual Basic Express, Visual C# Express, etc.) while Visual Studio 2012 introduced Visual Studio Express 2012 for Web, Visual Studio Express 2012 for Windows (Store), Visual Studio Express 2012 for Windows Desktop and Visual Studio Express 2012 for Windows Phone.

While their capabilities have been enhanced over time, a common theme remains: Microsoft has made the (business) decision to limit them artificially (meaning that the internal code knows the SKU edition and limits the functionality). In particular, Microsoft decided that they were not extensible. You won’t find an Add-In Manager in any of them and you will find the “Extensions and Updates…” menu only in the 2012/2013 versions that while allows you to install 3rd party extensions, they cannot run, only Microsoft’s extensions can run. Why?

In my opinion, because Microsoft has a conflict of interests:

  • On the one hand, it wants to charge for the developer tools, that is, the Visual Studio IDE. So it offers several tiers and prices: Standard edition, Professional edition, Team edition, Premium edition, Ultimate edition and whatever new name. However, the competing tools, that is, Eclipse/Java are free, which means that hobbyists, college students (future developers…),  and even worse, companies (of any size) prefer “open source” tools. Of course they don’t want those tools because they are open source, but because they are free -gratis- (my freeware MZ-Tools 3.0 add-in has zillions of users and less than three (3) users have requested me to open the source…)
  • On the other hand, to make developers to use the Microsoft developer tools (and to make companies to buy Microsoft Windows servers, Microsoft SQL Server, etc.) they have to offer some IDE version for free: the Express editions. But not powerful enough to eliminate the need of the paid versions, at least yet. And specially not enhanced by 3rd party extensions that would actually convert an Express edition into a “professional” edition. But these artificial limitations are relaxed over time. For example, now the Express editions allow unit testing, which initial versions didn’t offer. I guess that the excuse was that students need to learn automated testing, but guess what, Microsoft offers an ALM solution (Team Foundation Server) so it’s good that they can learn automated testing, continuous integration, etc..

I guess that eventually the Microsoft developer tools will be free, powerful enough and fully extensible, at least for the individual developer or small teams, in the same way that most ALM providers offer you free users in small quantities (Visual Studio Online 5 Basic users, Perforce 20 users, etc.). The software world is evolving to paying for the hardware (Apple and Microsoft give you new versions of their desktop operating systems and free apps that formerly were paid-apps) and towards software services where you pay an affordable subscription like you pay the electricity, or more recently the music, TV, online libraries, etc. However, these transitions can take time (depend on the markets, etc.), in the same way that Microsoft embracing “open source” has taken years, and now you have the code for the .NET Framework, the .NET compiler platform, etc.


Adopting Visual Studio Online (formerly TFS Service)

April 30, 2014

Brian Harry has written a post referencing this other post: “How Visual Studio Online won me over in under 90 minutes“. It happens that I also adopted Visual Studio Online less than I week ago and I am thrilled about it since then. In the past I used and investigated source code providers (Perforce, Subversion, TFS, …) installed locally but I was looking for some solution to the repository backup/restore “annoyance” (after a failed restore that I suffered long time ago). While I already played with TFS Service some months ago, it was not until the past Wednesday when I gave it a serious chance. I uploaded a solution of one of my projects that is part of a bigger source folder and after realizing that it was better to make some adjustments in the folder structure before uploading last Friday I uploaded all my Visual Studio solutions. Since then I am enjoying it a lot because of the following new scenarios:

  • I have a backup on the cloud each time that I check-in some file (instead of a making a backup by hand from time to time).
  • I can use more than one computer at home (I have a tiny MacBook Air 11″ and a huge iMac 27″) to develop without copying code through USB drives or having a central “server”.
  • I can use any device (such as a tablet or smartphone) with an internet connection to view the source code of the files, whenever I want.
  • I can see and create/update work items from any device (instead of using Excel or the tasks of an Outlook .pst file as before). Or rather than using a separate application such as Trello.

Now I am playing with gated check-ins, code analysis execution, automated tests, etc.

Visual Studio Online has several plans, one of them (Basic) completely free for up to 5 users:
http://www.visualstudio.com/products/visual-studio-online-user-plans-vs

And some videos to get started are:

Getting Started with Source Control with Visual Studio Online
http://channel9.msdn.com/Shows/Visual-Studio-Toolbox/Getting-Started-with-Source-Control-with-Visual-Studio-Online

Getting Started with Visual Studio Online
http://channel9.msdn.com/Events/Build/2014/2-575

Welcome to Visual Studio Online
http://channel9.msdn.com/Series/Visual-Studio-Online


Using Roslyn (The future of C# Build 2014 session)

April 30, 2014

I recommend to watch the following 1-hour recorded session of the past Build 2014 Developer Conference to learn about Roslyn (a.k.a the “compiler as service”):

The Future of C#
http://channel9.msdn.com/Events/Build/2014/2-577

The title is somewhat misleading, since the session is not so much about the future of C# (the language) but about the C# compiler, that is, Roslyn. It also covers VB.NET, by the way.  In that session Dustin Campbell shows demos about using Roslyn to do powerful things with the code in a (apparently) very easy manner.

One cool thing (for developers of Visual Studio extensions) is that the Visual Studio IDE will use only the same public API that is offered to extensions. That should guarantee that no longer happens what is happening so far: while VS has access to the compiler, your add-in or package has to use the code model (EnvDTE.Project.CodeModel, EnvDTE.ProjectItem.FileCodeModel), which is much less powerful and much more buggy.

It also seems that Roslyn will be very well documented, and furthermore being open source you can examine it, rather than using reverse-engineering.


Roslyn (.NET compilers as services) now open source

April 16, 2014

The greatest announcement for developers of Visual Studio extensions in the past days has been about Roslyn, the .NET compiler platform that will allow you to use the compilers as services, and that will be open source.

Even in closed form Roslyn would be great, because the current automation code model (EnvDTE.Project.CodeModel, EnvDTE.ProjectItem.FileCodeModel), apart from being entirely missing for some .NET languages such as F#:

  • It is very limited, either to get information or to emit code.
  • It has lots of bugs that I have reported on Microsoft Connect (see the “BUG” articles in the “Articles about the code model” section of my web site)
  • Microsoft decided not to improve it long time ago (and not to fix bugs, either).
  • Being native (not managed) can cause OutOfMemoryException due to memory fragmentation.

None of those things will happen with Roslyn for C# and VB.NET (other languages will have to wait, though), because they will be the compilers used by Visual Studio (starting with VS 2013 if you want) for those languages and Microsoft is changing Visual Studio to use only the public APIs of the parsers/compilers/etc. that will be available for Visual Studio extensions. So, any problem / limitation with the compilers and your extension is a problem for Visual Studio too.

And as if that was not enough, you can have the source code too!

Now, this raises some questions also for developer of Visual Studio extensions:

  • What to do with other languages apart from C# and .NET if your extension supports them (not the case of my MZ-Tools add-in)
  • Since Roslyn will be available for VS 2013 and higher, what to do if an extension has to support older versions (VS 2010, 2012 at least)…
  • What to do if you have a huge investment in the previous technologies (automation code model or your own parsers). For example, ReSharper has decided not to use Roslyn

And of course, the next thing that would most benefit developers of Visual Studio extensions is to open the Visual Studio assemblies. I don’t mean to “open source” but simply “reference source” (like the .NET Framework Reference Source) and “step source” (like .NET Framework Step Source) would be huge. Since opening the .NET compilers has taken years for Microsoft, I guess that we will have to wait, but I hope…


Debugging .NET Framework now working

April 16, 2014

Apart from the new .NET Reference Source experience, another thing that Microsoft has improved (well, has made actually work) is the Debugging .NET Framework feature of Visual Studio, which until recently was basically broken. Now, that feature doesn’t break with changes in the .NET Framework binaries due to security patches or hotfixes. Although Microsoft’s post mentions that it works with Microsoft .NET 4.5.1, it actually works with Microsoft .NET 4.0 and Microsoft .NET 4.5, at least if you have Microsoft .NET 4.5.1 installed (since 4.5 and 4.5.1 are in-place .NET Framework updates sharing the same CLR 4.0).

It definitely doesn’t work with .NET Framework 2.0, which, alas, is what I am still using in my MZ-Tools add-in (to support Visual Studio 2005/2008). But I guess that I can switch temporarily to .NET Framework 4.0 when I need to debug some .NET Framework assembly (just yesterday I needed to).

Of course, you can’t debug Visual Studio assemblies with that feature because their symbols are not available, but you can do it with .NET Reflector, as I explained in my article and in their blog as a guest post.


The new .NET Reference Source experience

April 16, 2014

In the last weeks Microsoft has made a series of announcements that will benefit a lot to .NET developers. The first one is the new .NET Reference Source experience. While the http://referencesource.microsoft.com/ site has existed for a while, now it provides new tools to navigate the code like you are used to in Visual Studio (Go To Definition, Find References, etc.). This means that for some scenarios (navigating the code of the .NET Framework assemblies) you don’t need tools such as Reflector .NET or JustDecompile. Of course these tools are still required in many other scenarios, since, for example, the source code of Visual Studio assemblies is not available.

The .NET Reference Source provides also these benefits:

  • You can download the code in a .zip file for off-line browsing.
  • You don’t need a Windows computer to browse the code. A tablet or even a smartphone will work.
  • You have the comments of the source code. Not just the headers, which are
    useless, but the ones inside the methods explaining why some things are
    done in some way (how a bug was fixed, etc.).

For example, last night I used my iPad to browse a couple of things that I was intrigued about:

Years ago I reported a bug in LadyBug (before Microsoft Connect) in the System.Drawing.Color.Equals method. The bug report is no longer available but I remembered that it was related to a comparison of two names that always returned true because the name variable was the same. And yesterday I found it is still there:

public override bool Equals(object obj) {

   if (obj is Color) {
      
      Color right = (Color)obj;

      if (value == right.value
         && state == right.state
         && knownColor == right.knownColor) {

         if (name == right.name) {
            return true;
         }

         if (name == (object) null || right.name == (object) null) {
            return false;
         }

         return name.Equals(name);
      }
   }
   return false;
}

The second one is a tough problem that I was dealing with lately involving COM Reflection (not .NET Reflection). The problem is, given a COM object, how to know its class name (not its interface name). I knew that VB.NET (like VB6 before) has a handy Information.TypeName() method in the Microsoft.VisualBasic assembly, which always seems to return a class name instead of an interface name. Browsing its implementation, I realized that the TypeNameOfCOMObject and LegacyTypeNameOfCOMObject methods are cheating cleaning up the type name:

Friend Function LegacyTypeNameOfCOMObject(ByVal VarName As Object, ByVal bThrowException As Boolean) As String

   Dim Result As String = COMObjectName

   Try
      Call (New SecurityPermission(SecurityPermissionFlag.UnmanagedCode)).Demand()
   Catch ex As StackOverflowException
      Throw ex
   Catch ex As OutOfMemoryException
      Throw ex
   Catch ex As System.Threading.ThreadAbortException
      Throw ex
   Catch e As Exception
      If bThrowException Then
         Throw e
      Else
         GoTo CleanupTypeName
      End If
   End Try

   Dim pTypeInfo As UnsafeNativeMethods.ITypeInfo = Nothing
   Dim hr As Integer
   Dim ClassName As String = Nothing
   Dim DocString As String = Nothing
   Dim HelpContext As Integer
   Dim HelpFile As String = Nothing

   Dim pDispatch As UnsafeNativeMethods.IDispatch = TryCast(VarName, UnsafeNativeMethods.IDispatch)

   If pDispatch IsNot Nothing Then
      hr = pDispatch.GetTypeInfo(0, UnsafeNativeMethods.LCID_US_ENGLISH, pTypeInfo)
      If hr >= 0 Then
         hr = pTypeInfo.GetDocumentation(-1, ClassName, DocString, HelpContext, HelpFile)
         If hr >= 0 Then
            Result = ClassName
         End If
      End If
   End If

CleanupTypeName:

   If Result.Chars(0) = "_"c Then
      Result = Result.Substring(1)
   End If

   Return Result

End Function

That is, if the name starts with “_” (which likely means it is an interface), the class name that implements that interface is the same name without that character, which can be true sometimes but definitely not most of the time.


Syncfusion 28 free e-books

March 13, 2014

Syncfusion (a well known company that provides enterprise-class software components and tools for the Microsoft .NET platform) has published on its Technology Resource Portal a collection of 28 free e-books about several technologies, languages and frameworks. All of them include the “Succinctly” term in the title, which called my attention because it means that they are larger that an article or blog post, but smaller than the typical book of 400-500 pages. Interestingly enough, one of them is about Visual Studio add-ins:

http://www.syncfusion.com/resources/techportal

You only need to provide your name, company, e-mail and phone to download them. Even better, they are provided in .pdf and Kindle formats.