“The active Test Run was aborted because the execution process exited unexpectedly”

I just got this error when running a medium unit test suite (many hundreds of tests).

“The active Test Run was aborted because the execution process exited unexpectedly. To investigate further, enable local crash dumps either at the machine level or for process vstest.executionengine.x86.exe. Go to more details: http://go.microsoft.com/fwlink/?linkid=232477

Anyone want to guess the problem is?

It appeared in the output window, quite suddenly in an otherwise fairly happy and fairly TDD day.

Hmmm.

  • Delete existing TestResults and .SOU (my traditional voodoo).
  • Have a glass of wine (my traditional, calm down and try again).
  • Restart VS.
  • Reboot my machine.

No joy. Hmmm.

Can I tell you how much I do not want to investigate further through local crash dumps?

Let me see if I can run any tests at all. It really looks like MSTest is broken.

Hmmm.

  • Another test ran fine.
  • Let me run a block: yes.
  • Another block: no
  • Another block: yes
  • Another block: yes
  • Another block: yes
  • And so on…until I found just one test.

OK, I’ll debug it.

When debugging, rather than running the test – the app crashes with a stupid error that resulted in a recursive call and a stack overflow error.

Stack overflows can be really funny to diagnose. There’s no more air and the last gasp of the process is to shout something, anything, a final nonsense farewell…

Except if that error can ever physically get to a user, the phrasing could be better. I already submitted a request for a better message.

I’m posting here, because if you encounter this error do some combination of the following:

  • Run tests in debug mode
  • arrow in on one or more offending tests, then run in debug mode
  • There may be other ways this occurs – but it’s been reported at least twice with a stack overflow

Roslyn (.NET Compiler Platform) As Open Source

I’m really happy that Roslyn has been Open Sourced. But that’s because I trust that it will be an open source project where the teams at Microsoft retain enormous control. I have a couple of stories.

The Conference Conversation

At a bar at a conference last fall I tracked down a guy I respect enormously to convince him to join me in what I perceived at the time would be a battle for a specific language feature. Because of past parallel thinking, I thought this would be an easy sell.

I recall having five reasons that I wanted this feature. I walked through the first four, carefully explaining why I was right. In each case, his response could be summarized as “Anders knows best.” In fairness, he did weigh the issue, and to him the balance of benefit didn’t win over having an expert define the language.

I respect Ander Hejlsberg enormously. I also know from experience that “battling” means ensuring different perspectives are explored over and over in his mind until he either thinks the same thing as I do or doesn’t. I suspect I will never win the XML/JSON literals conversation.

I’m only moderately confident that my putting certain things out there over and over cause Anders and the other language designers to think about issues any more than they already are. They seem to think about your languages all the time. They think fast and fluidly, and with you in mind.

The fifth item my list that night in the conference bar was that there was no other way to accomplish this particular goal. That was the only item that swayed my friend.

The punch line: my friend was right.

The battle I was planning to have is entirely appropriate for the first four of my reasons. It’s just not necessary to involve the language at the intricate behavior layer, because it can be accomplished with parser and editor changes that leverage existing features.

(I’ll discuss the specific issue in another blog post.)

Left or Right Gives Me a Headache

This story is happening as I write this post. I’ve tweeted that maybe you don’t want to know how the sausage is made – how language decisions are made.

If GetCustomer() can return null, to avoid doing this:

var temp = GetCustomer();

string name = (temp == null) ? null : temp.name;

It would be nice to do this:

var temp = GetCustomer()?.name;

But, it turns out to have two possible implementations:

left-associative : x = ((GetCustomer1()?.extraRef)?.name)?.Length

right-associative : x = GetCustomer1()?.(extraRef?.(name?.Length))

Over the last six months, thousands of man hours have gone into trying to decide which is the better approach (before jumping to an opinion, you might want to read the public thread).

People at Microsoft, insiders like MVPs and now the community at large have thought a lot about this problem. Sometimes issues are really, really hard. Subtle and hard enough to make your head hurt.

Sometimes the outcome of a really gnarly issue like this is arbitrary. Sometimes it results in cancelling the feature because the result will be non-intuitive. Sometimes at the end of the discussion, one option has to be vetoed. If one option could win, it was probably not a gnarly issue.

It takes people deeply involved in the languages, trained and collaborating to look for alternative implementations, and subtle issues, and non-obvious breaking changes to manage these gnarly issues. Lucian has seen an issue I didn’t see more times than I can count. It’s his job.

Why This Means Microsoft Has to Stay in Control

I’m confident that Microsoft will remain firmly in control of check-ins to your favorite language compilers. If they don’t we’re dead because of situations like the ones I illustrated.

In the first case, the years of language design teams saying “no, no, no, no, no, no, no” keep the languages comparatively clean. There’s a set of things that have been pushed off because they were hard to do in the old compiler and are now being done. But there is a larger set of things that do not belong in the language because they are parser/preparser/editor issues (I discussed parser/behavior/compiler independence here). It is possible that C# XML and C#/VB JSON literals can even be handled with optional extensions.

The second story illustrates a deeper reason you are happy with Microsoft staying in iron fisted control of compiler check-ins. As a long time MVP I get to see the depth of thinking about dozens of such issues in VB and C# every year. Many issues look simple on the surface, but deep collaborative thinking by the team exposes subtleties that drive one implementation or another. Dozens of issues receive the same kind of “Language Notes” features you see here, here, here, and in many parallel posts.

The Cathedral and the Bazaar

The historic way of creating and supporting the .NET compilers could be likened to building a cathedral (Raymond, Eric S. (2000-09-11). "The Cathedral and the Bazaar" for metaphor). A hierarchy of architects and masons worked together to create a complex result.

Open source has been compared to a bazaar with everyone contributing.

I don’t want the hubbub and chaos of a bazaar. I want the stability and consistency in the longevity of the cathedral. I want the current standards of depth of consideration, and the frequency of “no” to be maintained.

I want to push and prod and play and dance on the edges with parser tweaks and using information from syntactic and semantic trees. I also want the rapid releases of open source.

I want to do this still knowing there is quiet serenity, thick walls and buttresses around the subtleties of runtime behavior and performance.

I still want the language designers that got us this far at the helm. I have no reason to think the open source release of Roslyn changes that.

Language as Two Parts: Parser and Behavior

I wrote here about Roslyn as a Black Box. There’s another layer to this that can be remarkably subtle.

This isn’t actually the complete view.

RoslynWithManyInterfaces

One thing that is missing is that the inherent nature of interfaces is anything can be on the other side.

“Of course!” you say. “Anything can be at the top and bottom of your picture”.

That’s true. But anything can also be in the middle.

RoslynWithMultipleCompilers

There’s no reason a composed compiler system can’t support multiple compilers, and variations. MSBuild scripts need a little tweaking, of course.

There are actually already two compilers – for C# and Visual Basic. They differ in a number of small, relatively subtle ways – such as overload resolution. There are also language concepts that do not align, like VB XML literals and null propagation for XML.

Computer Languages as Two Things

Roslyn clarifies that computer languages are made of two parts. Languages relate intimately to a parser – this determines the syntax. Languages are also defined by runtime behavior. There are thousands of aspects of behavior that are determined with excruciating detail by the way the compiler creates IL (Intermediate Language).

In the diagram above, this means the yellow top layer could be different from the black box.

To understand this, imagine using the Visual Basic compiler to build a tree for the C# language compiler. If all features were supported, you would create a functioning application. But, it would not behave the same on several of these thousand little details. For one thing, it would not have the same overload resolution in all cases.

This is why the dream of one core language with a VB/C# dialect displayed per the user’s preference is an elusive goal. It would result in a third language. VB expressed as C# would not have the same behavior.

The same thing is true for incorporating other strongly typed OO languages – like Java. If the concepts can be mapped to the C# Roslyn API, then C# code can be created. It can be created either as IL, or as human readable code.

Whether cross language compilation is useful depends on expectations. I suspect a straight up conversion, after non-aligned concepts were removed, would be only a little worse than an expensive human programmer. It might be better than a programmer new to the target language.

Common Core Features

One of the interesting things to me about the introduction of Roslyn is that we can get a better understating of core language sets (hint if you are looking for an MS or PhD thesis). Obviously VB and C# are very similar and there is a better body of knowledge on their differences in the post-Roslyn world.

But what about Java? What would be the subset of the C# or VB tree that Java could support? How much of the language would be left out? For example, would structured trivia support Java annotations?

And what about JavaScript? Ok. I know it’s not strongly typed, so it’s a big leap, but I doubt we’d actually go to JavaScript when TypeScript already makes that part of the leap. Could C# go to TypeScript and then to JavaScript. The immediate payback is a new way to share blocks between C#/VB and JavaScript.

And what about Haskell? OK. I know that’s a really big leap. Haskell requires incredible type inference, lazy loading, duck typing and other features beyond the current iterations of C#/VB. As I understand it, and I’ve never had even a temporary tattoo of F#, Haskell is better at strict purity and the guarantee of mathematical reasoning. Or maybe we just need F# at the party.

I don’t believe we need to do these things across our entire applications. Sharing UI code with JavaScript might make sense, but doing mathematical reasoning across a user interface doesn’t make sense to me.

We have the PCL to create components whose IL is transferrable across different platforms (.NET, Windows Phone, XBox). It seems possible and maybe interesting to build a similar subset/transferrable component at the language level.

I don’t know whether there are other things we can do with the separation of parsing, compiling and emitting. But the exciting thing about Roslyn, and its OSS release, is that we can explore.

I’m trained as a scientist, a crystallographer. One of the reasons I’m so excited about Roslyn is that opening up easy access to information leads to interesting research. I’m sharing this vision, as a little part of a broader vision we can build together, and hoping that helps academics and others figure out interesting things to work on.

There are brick walls down some avenues. Research isn’t research if it might not fail. But it’s going to be a heck of a ride. I can hardly wait to see what the next decade brings.

Roslyn (.NET Compiler Platform) should be a Black Box

Compilers should be a black box.

We all know this.

We’re just in the moment of giddiness where we must consciously remember gravity and the physical laws of software.

So let’s get real.

Roslyn is an enormous step forward. I’ll say again that I believe the Roslyn release will go down as one of the pivotal moments in the history of computing.

But it still should be a Black Box.

“Can’t it be my favorite color?”

OK, if you don’t like the idea of a black box, make it gold, rainbow or your favorite color. As long as it is opaque.

From a Computer Science perspective it needs to be a black box with results oriented facades.

The world before Roslyn:

RoslynPre

The world after Roslyn

RoslynWithTrees

Why on earth would you care that there are trees in the box? I can talk a lot about syntactic and semantic trees. Some of us should talk a lot about them. Very few of us.

Most of us should interact with the Roslyn information through a series of interfaces, just like any other Computer Science Black Box.

RoslynWithManyInterfaces

“Isn’t that the Roslyn API?”

No. The Roslyn API represents the actual structures. Those structures have a primary purpose of supporting very fast compilation. They have a secondary purpose of supporting UI features. Support for external things is not a primary purpose, not should it be. Roslyn is a compiler released as a service. Roslyn is a compiler.

The interfaces we want are purpose specific. I drew several of them in the schematic. Clearly one is for the UI – probably another layer with the current editor services.

One is for collecting the meaning of your code. This is important for metaprogramming. I tend to get a bit stuck on that, but exactly the same interfaces can be useful to report and understand your system, and to carry forward some of it into another iteration of your application.

Either that interface, or another, will allow better reasoning about unit tests – including statistics on how your testing boundary conditions. And I expect to be amazed at the other purposes the Roslyn API will allow.

“What then?”

With such interfaces available, anyone, anywhere can write code to move from the Roslyn API into the interface. It is clear defined OSS work and can be done once, or a very small number of times. Anyone, anywhere can build on these interfaces to stretch the limits of what we can accomplish. To figuratively, reach for the stars.

We will go further faster working together with this pattern: find the purpose, gather the parties and establish interfaces – purposeful APIs. Look at mechanisms to vary and extend. Then let people go crazy creating the best implementation.

I want to be part of that conversation for metaprogramming.

I hope there’s a parallel conversation for at least one of the UI spaces – refactoring, IntelliSense, formatting, or something.

And I want the .NET Foundation to support those efforts to establish the path for community defined interface/purposeful APIs. It’s not going to be a trivial task to establish initial interface/purposeful APIs, mechanisms for extension and an ecosystem that may not fit well into other OSS process patterns.