Interception in .NET – Part 3: Static Interception

Updated: see the last post in the series here.

Introduction

Part three of a series on interception in the .NET world. See the first part here and the second here.

This time I’m going to talk about static interception using PostSharp as an example. Static interceptors use information provided at code writing time – aspects – to do IL weaving. This basically means that after the .NET assembly is compiled, PostSharp rewrites the produced IL to apply the changes intended by the code developer, such as method interception. This is a good thing, as you can intercept types and methods regardless or not if they are virtual, static, implement some interface, etc.

This won’t be a full coverage of PostSharp, of course – which has far more than just interception – , but I include it in this series so that we see how it compares to other techniques.

Example Using PostSharp

Imagine you want to retry a certain method a number of times, this may be because what it does is sensible to network failures, or any other reason. In PostSharp, you would write a method interceptor aspect as this:

[PSerializable]
public class RetryOnExceptionAttribute : MethodInterceptionAspect
{
    public int MaxRetries { get; set; }

    public override void OnInvoke(MethodInterceptionArgs args)
    {
        int retriesCounter = 0;

        while ( true )
        {
            try
            {
                base.OnInvoke(args);
                return;
            }
            catch (Exception e)
            {
                retriesCounter++;
                if (retriesCounter > this.MaxRetries) throw;

                Console.WriteLine(
                    "Exception during attempt {0} of calling method {1}.{2}: {3}",
                    retriesCounter, args.Method.DeclaringType, args.Method.Name, e.Message);
            }
        }
    }
}

Note that you need to reference the Nuget package PostSharp.Patterns.Diagnostics, or any other that contains the PostSharp core assemblies.

And you could then apply it to some method:

public class CustomerService
{
    [RetryOnException(MaxRetries = 5)]
    public void Save(Customer customer)
    {
        // Database or web-service call.
    }
}

After Visual Studio (or, rather, MSBuild), builds your assembly, PostSharp pops in: it reads definitions present in the PostSharp.Custom.targets file and modifies that assembly to call the RetryOnExceptionAttribute’s OnInvoke method around the Customer.Save method, because the aspect was applied to it.

The MethodInterceptionAspect is not the only aspect class, there’s also:

After PostSharp modifies the assembly, it will no longer resemble your code: if you decompile it, you will see other stuff in there, like, a call to RetryOnExceptionAttribute.OnInvoke. Something like this:

[DebuggerTargetMethod(0x600000d), DebuggerBindingMethod(0x6000010)]
public void Save(Customer customer)
{
    try
    {
        Arguments<Customer> arguments = new Arguments<Customer> {
            Arg0 = customer
        };
        MethodInterceptionArgsImpl args = new MethodInterceptionArgsImpl(this, arguments) {
            DeclarationIdentifier = new DeclarationIdentifier(0x78acef3000030000L),
            Method = <>z__a_1._2,
            TypedBinding = <Save>c__Binding.singleton
        };
        <>z__a_1.a1.OnInvoke(args);
    }
    finally
    {
    }
}

Although hard to understand, rest assured that it does what you expect! Winking smile There are a bunch of additional types generated, but it’s really irrelevant to look at them.

Alternatives to PostSharp

There are some commercial alternatives and then there’s Mono.Cecil. Implemented as part of the Mono project – meaning, free and open source – it allows you to do low level IL weaving, but you should know what you are doing as there are none of the nice wrappers that PostSharp provides.

There are some interesting projects that make Mono.Cecil easier to use, and one of them is Fody. It is basically a wrapper around it, and is also open source. If you want to do things yourself, give it a try!

Conclusion

PostSharp is very powerful, as is IL weaving in general, but sometimes we need more than static interception. Here’s where dynamic interception kicks in. In the next post, I will start describing a framework for dynamic interception with a number of implementations. Stay tuned!

Entity Framework Pitfalls: Skip/Take Position Matters

In LINQ, you normally do paging through the Skip and Take operators. These are the LINQ counterparts to whatever the relational database uses for pagination – ROW_NUMBER(), LIMIT…OFFSET, TOP, ROWNUM, etc.

There are some gotchas to it:

  • When you do paging, Entity Framework demands that you add ordering to your query. This makes sense as without an explicit ORDER BY clause, search results may come in an unexpected order;
  • Skip and Take yield different results if added to different places in the LINQ expression.

Consider these two LINQ queries:

ctx
.MyEntities
.Where(x => x.Name == "")
.OrderBy(x => x.Id)
.Skip(10)
.Take(5)
.ToList();

And:

ctx
.MyEntities
.OrderBy(x => x.Id)
.Skip(10)
.Take(5)
.Where(x => x.Name == "")
.ToList();

The first one will produce this SQL:

SELECT TOP (5)
[Filter1].[Id] AS [Id],
[Filter1].[Name] AS [Name],
[Filter1].[Date] AS [Date]
FROM ( SELECT [Extent1].[Id] AS [Id], [Extent1].[Name] AS [Name], [Extent1].
[Date] AS [Date], row_number() OVER (ORDER BY [Extent1].[Id] ASC) AS [row_number
]
FROM [dbo].[MyEntities] AS [Extent1]
WHERE N'' = [Extent1].[Name]
) AS [Filter1]
WHERE [Filter1].[row_number] > 10
ORDER BY [Filter1].[Id] ASC

Whereas the second will produce this one:

SELECT
[Limit1].[Id] AS [Id],
[Limit1].[Name] AS [Name],
[Limit1].[Date] AS [Date]
FROM ( SELECT TOP (5) [Extent1].[Id] AS [Id], [Extent1].[Name] AS [Name], [Extent1].[Date] AS [Date]
FROM ( SELECT [Extent1].[Id] AS [Id], [Extent1].[Name] AS [Name], [Extent1].[Date] AS [Date], row_number() OVER (ORDER BY [Extent1].[Id] ASC) AS [row_number]
FROM [dbo].[MyEntities] AS [Extent1]
) AS [Extent1]
WHERE [Extent1].[row_number] > 10
ORDER BY [Extent1].[Id] ASC
) AS [Limit1]
WHERE N'' = [Limit1].[Name]
ORDER BY [Limit1].[Id] ASC

Basically, the TOP(5) expression is placed in the wrong place. So, in LINQ, it’s not just the presence of a Skip and Take operator, it’s where you place it that matters!

Visual Studio Tips 4

OK, so this is another round of tips on using Visual Studio! See first collection here, second here and third here.

  1. Create shortcuts: you can create shortcuts to frequently used code. Just click <Ctrl>KH on any line, and you will add a shortcut to it:

    image
    After that, you can jump to any of your bookmarks from the Task view:
    image
    To remove it, click <Ctrl>KH again in the same line. Note that this is not the same as the bookmarks featured introduced in the third set of tips!
  2. Code Maps: this requires Visual Studio Enterprise. Open the Architecture menu and select Generate Code Map for Solution.

    image
    You will get a nice diagram showing all dependencies between assemblies in your project:
    image
  3. Code Metrics: get a static analysis report of your project, only in Enterprise edition. In the Solution Explorer, right click on your project, Analyze, and select Calculate Code Metrics.

    You get an analysis including metrics Maintainability Index, Cyclomatic Complexity, Depth of Inheritance, Class Coupling and Lines of Code, which you can even export to Excel:
    image
  4. Format document on save & remove unused and sort using statements on save: Visual Studio can take care of formatting your documents and removing any unnecessary using declarations for you, when you save your documents; you just need to install the Productivity Power Tools extension and enable these options in ToolsProductivity Power ToolsPowerCommands:
    image
  5. Thread Deadlocks: Visual Studio can show you thread deadlocks while debugging. When debugging a multithreaded app, break it and open DebugWindowsTasks:
    image
    From here, you can flag your code so that it is more clearly visible.
  6. Executing code in Immediate Window: the Immediate Window can be used to run .NET code. While debugging your app, pause it, and select DebugWindowsImmediate:

    image
    You can enter any .NET expressions and variables or even complex expressions and see them evaluated at once.
  7. Pin and export Data Tips: while debugging, break and hover any variable or field in your code. Click the pin icon on the right of the tooltip window. This will create a Data Tip.
    image
    You can have multiple Data Tips, add comments to them:
    image
    Also save Data Tips to files so that they can be loaded in another Visual Studio session, and, of course, load a collection of Data Tips:
    image
  8. Place reusable code in Toolbox: you can add code that you use frequently to the Toolbox. Pin your Toolbox window, highlight your code and drag it to the Toolbox. Even if you are not working in a visual app, the code snippet can be dragged from the Toolbox into your code:
    image
  9. Change C# language version: Visual Studio can use any of the C# versions. For example you may or may not want to use the latest (and greatest, by the way!) features of C# 6. All you have to do is right click on a project, Properties, Build and Advanced. In the Advanced Build Settings window, Language Version, select the one you want to support:
    image
  10. Find tips: some tips related to the Find functionality. First, you have two results windows, 1 and 2, and you can have your search results appear in any of them:
    image
    With this, you can also append to current results or show only the file names that match the search pattern.

Coming Up Next

The following topics have been somewhat neglected… I promised to write follow ups to existing posts, but have been unable to so far. Hopefully I will resume them soon, in some cases I already started working on them:

As always, I’m keen to hear from you, so, do drop me a note if you have any questions or suggestions!

Entity Framework Pitfalls: Null Semantics

Imagine you have a simple class:

public class MyEntity
{
public int Id { get; set; }
public string Name { get; set; }
}

You may want to query it:

string name = ...;

var entities = ctx
.MyEntities
.Where(x => x.Name == name)
.ToList();

Entity Framework generates a query such as this:

SELECT [Extent1].[Id] AS Id, [Extent1].[Name] AS Name
FROM [dbo].[MyEntities] AS [Extent1]
WHERE ([Extent1].[Name] = @p__linq__0)
OR (([Extent1].[Name] IS NULL) AND (@p__linq__0 IS NULL))

Meaning: return all records from the MyEntities table if

  1. The Name column is identical to the name variable or
  2. Both the Name column and the name variable are NULL

Seems excessively complex, doesn’t it? Well, this is to account for the case where the name parameter is null. In  that case, a simple COLUMN = VALUE comparison won’t work, because in SQL, NULL = NULL returns FALSE! The syntax for comparing with NULL is COLUMN IS NULL. By generating this kind of query, Entity Framework is playing safe. Mind you, this only happens if you do not use the literal null, if you do, Entity Framework is smart enough to generate a simpler query:

var entities = ctx
.MyEntities
.Where(x => x.Name == null)
.ToList();

Results in:

SELECT [Extent1].[Id] AS Id, [Extent1].[Name] AS Name
FROM [dbo].[MyEntities] AS [Extent1]
WHERE ([Extent1].[Name] IS NULL)

However, there is a configuration setting, UseDatabaseNullSemantics (the opposite of the old UseCSharpNullComparisonBehavior of ObjectContext), that can be used to control this behavior. If set to true (the opposite of the default):

ctx.Configuration.UseDatabaseNullSemantics = true;

It generates all SQL queries as the unsafe version:

SELECT [Extent1].[Id] AS Id, [Extent1].[Name] AS Name
FROM [dbo].[MyEntities] AS [Extent1]
WHERE ([Extent1].[Name] = @p__linq__0)

The result is, even if you have a record whose Name column is NULL, it won’t be returned: remember that in SQL, by default, NULL <> NULL!

Therefore, if you are not 100% sure that the parameter that you will be using in the LINQ query will never be null, do not mess with UseDatabaseNullSemantics even if it generates somewhat poorer SQL.

Of course, if you are using SQL Server, you can set ANSI_NULLS to OFF, and the problem goes away; in that case, NULL = NULL.

SQLPort em Aveiro

(Portuguese only, sorry!)

No próximo dia 22 de Setembro irá ter lugar em Aveiro, nas instalações da Fusion Cowork (Fusion 2), o LXXXVI evento da Comunidade SQLPort, o primeiro dos quais em Aveiro!

Os oradores são do mais alto nível: Niko Neugebauer (@NikoNeugebauer) e Eduardo Piairo (@EdPiairo), irão falar sobre Performance Tuning e Database Source Control. O programa é o seguinte:

18:30 Abertura e recepção
19:00 “Performance Tuning” – Niko Neugebauer (MVP, Consultant – OH22)
20:10 Intervalo
20:30 “Database source control: migrations vs state” – Eduardo Piairo (DBA – Celfinet)  
21:40 Sorteio
22:00 Jantar livre

Podem ver mais informações aqui e registarem-se aqui. O evento é gratuito mas não dispensa inscrição!

A morada da Fusion 2 é:

Rua Carlos
Aleluia, 4, Edifício Aveiro
Centrum, Loja 3

3810-177 Aveiro

Aguardamo-vos lá! Winking smile

Entity Framework Pitfalls: Queries Over Navigation Properties When There Are Foreign Key Properties Are Ignored

Long title, I know! Smile

Another one for Entity Framework Code First. The problem is: imagine you have an entity with a relation to another entity (one-to-one, many-to-one) where you have both the navigation property and the foreign key property:

public class Source
{
public int Id { get; set; }

[ForeignKey("TargetId")]
public virtual Target Target { get; set; }

public int? TargetId { get; set; }
}

public class Target
{
public int Id { get; set; }
public virtual ICollection<Source> Sources { get; set; }
}

You may want to query all Sources that have a Target set:

var sourcesWithTarget = ctx
.Sources
.Where(source => source.Target != null)
.ToList();

However, you will get this SQL instead:

SELECT [Extent1].[Id] AS [Id], [Extent1].[TargetId] AS [TargetId], 
FROM [dbo].[Source] AS [Extent1]

See? No WHERE clause!

If, instead, we query on the foreign key property:

var sourcesWithTarget = ctx
.Sources
.Where(source => source.TargetId != 0)
.ToList();

Then we get what we want:

SELECT [Extent1].[Id] AS [Id], [Extent1].[TargetId] AS [TargetId], 
FROM [dbo].[Source] AS [Extent1]
WHERE 0 <> [Extent1].[TargetId]

Weird, huh?

Stay tuned for more!

Missing Features in Entity Framework Core 1

Here’s a brief summary of some of the features that were present in previous versions of Entity Framework (6.x) and were excluded (or are not yet implemented) in Entity Framework Core 1:

Feature Description Reason/Workaround
Lazy loading The ability to load entity relations after the root entity was loaded, automatically

Will be available in a future version. For now, we need to use eager loading (Include).

https://github.com/aspnet/EntityFramework/issues/3797

Explicit loading The ability to load entity relations after the root entity was loaded, explicitly

Will be available in a future version. For now, we need to use eager loading (Include).

https://github.com/aspnet/EntityFramework/issues/3797

Support for Group By Specifying GroupBy in a LINQ query

Currently it falls back silently to LINQ to Objects, meaning, brings everything from the database and groups in memory. Will be available in a future version. For now, use plain SQL.

https://github.com/aspnet/EntityFramework/issues/2341

Support for user defined functions

Using static methods as UDFs

The ability to use .NET static methods as User Defined Functions (DbFunctionAttribute) is not supported now. Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/4319

DateTime Operations and common SQL functions Doing DateTime operations and using common SQL functions in LINQ

The ability to do queries involving Date/Time operations (DbFunctions) and common SQL functions (SqlFunctions) in LINQ queries is not supported now. Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/2850

Complex Values Support for properties of complex types (value objects)

The ability to use properties of complex types is not supported yet. Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/246

Many to Many Collections Many-to-many relations without a middle entity/table

Will be available in a future version. For now, we need a middle entity and table.

https://github.com/aspnet/EntityFramework/issues/1368

Table Per Type Inheritance Strategy

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/2266

Table Per Concrete Type Inheritance Strategy

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/3170

Mapping CUD with stored procedures The ability to use stored procedures for doing inserts, updates and deletes seemlesly

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/245

Map database views The ability to map views instead of tables

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/1679

https://github.com/aspnet/EntityFramework/issues/827

Spatial data types The ability to query and use spatial data types

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/1100

Custom conventions The ability to add custom conventions

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/214

Populate non-model types from SQL Turn the results of custom SQL into classes that are not part of the model

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/240

Connection resiliency support The ability to retry connecting and sending queries

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/237

Seeding data in migrations The ability to add data when migrating

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/629

Command and query interception The ability to intercept queries and SQL commands

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/626

https://github.com/aspnet/EntityFramework/issues/4048

https://github.com/aspnet/EntityFramework/issues/737

Visual Studio support for generating/updating entities from the database and viewing the model The ability to generate the model from the database from inside Visual Studio and to view the model graphically

Will be available in a future version.

https://github.com/aspnet/EntityFramework/issues/1186

Database initializers Database initializers Dropped.
Automatic migrations Automatic migrations Dropped.
Pluralization Service Pluralization Service

Dropped.

https://github.com/aspnet/EntityFramework/issues/2506

ObjectContext events SavingChanges and ObjectMaterialized events of ObjectContext

Dropped.

https://github.com/aspnet/EntityFramework/issues/3204

ObjectContext (Entity SQL) Entity SQL Dropped.
Model first approach Model first approach Dropped.
Data Annotations validations The ability to perform data annotations validations before saving changes Dropped.

Please let me know if you think I missed something! Winking smile

For the most up to date roadmap, please consult the EF Core roadmap: https://github.com/aspnet/EntityFramework/wiki/Roadmap.

Also, do check out the Entity Framework issue tracker: https://github.com/aspnet/EntityFramework/issues

Implementing Missing Features in Entity Framework Core – Part 5: Getting the SQL for a Query

This will be the fifth post in a series of posts about bringing the features that were present in Entity Framework pre-Core into EF Core. The others are:

  • Part 1: Introduction, Find, Getting an Entity’s Id Programmatically, Reload, Local, Evict

  • Part 2: Explicit Loading

  • Part 3: Validations

  • Part 4: Conventions

This time I’m going to talk about something that is often requested: how can I get the SQL string for a LINQ query? If you remember, in the pre-Core days you had to do some reflection in order to get the underlying ObjectQuery and then call its ToTraceString method. Now, things are very different, although I may say, still rather tricky!

So, this is all we need:

private static readonly FieldInfo QueryCompilerField = typeof(EntityQueryProvider).GetTypeInfo().DeclaredFields.First(x => x.Name == "_queryCompiler");
private static readonly PropertyInfo NodeTypeProviderField = QueryCompilerTypeInfo.DeclaredProperties.Single(x => x.Name == "NodeTypeProvider");
private static readonly MethodInfo CreateQueryParserMethod = QueryCompilerTypeInfo.DeclaredMethods.First(x => x.Name == "CreateQueryParser");
private static readonly FieldInfo DataBaseField = QueryCompilerTypeInfo.DeclaredFields.Single(x => x.Name == "_database");
private static readonly FieldInfo QueryCompilationContextFactoryField = typeof(Database).GetTypeInfo().DeclaredFields.Single(x => x.Name == "_queryCompilationContextFactory");

public static string ToSql<TEntity>(this IQueryable<TEntity> query) where TEntity : class
{
if (!(query is EntityQueryable<TEntity>) && !(query is InternalDbSet<TEntity>))
{
throw new ArgumentException("Invalid query");
}

var queryCompiler = (IQueryCompiler) QueryCompilerField.GetValue(query.Provider);
var nodeTypeProvider = (INodeTypeProvider) NodeTypeProviderField.GetValue(queryCompiler);
var parser = (IQueryParser) CreateQueryParserMethod.Invoke(queryCompiler, new object[] { nodeTypeProvider });
var queryModel = parser.GetParsedQuery(query.Expression);
var database = DataBaseField.GetValue(queryCompiler);
var queryCompilationContextFactory = (IQueryCompilationContextFactory) QueryCompilationContextFactoryField.GetValue(database);
var queryCompilationContext = queryCompilationContextFactory.Create(false);
var modelVisitor = (RelationalQueryModelVisitor) queryCompilationContext.CreateQueryModelVisitor();
modelVisitor.CreateQueryExecutor<TEntity>(queryModel);
var sql = modelVisitor.Queries.First().ToString();

return sql;
}

You can see that it needs some reflection, meaning, things *may* break on a future version. I cached all of the fields to make the access faster in subsequent calls. For the time being, it works perfectly:

var sql1 = ctx.Blogs.ToSql();

var sql2 = ctx
.Blogs
.Where(b => b.CreationDate.Year == 2016)
.ToSql();

Hope this is useful to you! Winking smile

Unit Testing .NET Core

Introduction

With the recent arrival of .NET Core, some things that we were used to having are no longer available. This includes unit tests – plus Visual Studio integration – and mocking frameworks. Fortunately, they are now becoming available, even if, in some cases, in pre-release form.

In this post, I will cover:

  • Unit testing frameworks
  • Mocking libraries
  • Validation libraries

I won’t explore everything that exists, just present a simple setup that works well for .NET Core.

Unit Testing

So, you want to do unit testing? There are a couple of frameworks for that purpose that work with .NET Core, which include:

MS Test is Microsoft’s own unit testing library, I’d say a much hated one because of its historical tie to Visual Studio. This was the last one that was announced for .NET Core and it’s still not in release form.

They more or less work the same way, so let’s see how to use xUnit, my favorite one So, you need to add two Nuget packages:

  • xunit: this is the actual library that you will be using
  • dotnet-test-xunit: this is the runner and integration with Visual Studio

I sometimes have a base class for tests, which takes care of loading configuration and initializing stuff, but let’s leave that aside. A unit test class for xUnit looks like this:

using Xunit;

public class MyTests
{
[Fact]
public void Test()
{
}
}

Now, you need to configure the runner so that you can run tests in Visual Studio. Using project.json, it goes like this:

{
"version": "1.0.0-*",
"testRunner": "xunit",
"buildOptions": {
"emitEntryPoint": false,
"copyToOutput": ["appsettings.json"]
},

"dependencies": {
"dotnet-test-xunit": "2.2.0-preview2-build1029",
"Microsoft.Extensions.Configuration": "1.0.0",
"Microsoft.Extensions.Configuration.Json": "1.0.0",
"Microsoft.Extensions.PlatformAbstractions": "1.0.0",
"Microsoft.NETCore.App": {
"type": "platform",
"version": "1.0.0"
},
"xunit": "2.2.0-beta2-build3300"
},

"frameworks": {
"netcoreapp1.0": {
"imports": "dnxcore50"
}
}
}

Notice that I am copying to the output folder the appsettings.json file, I am not going to talk about it, but let’s just say that it has some configuration that my tests will use, and it needs to be on the same file as the unit tests assembly.

With the Visual Studio integration working, we get this:

image

So we can run and debug our tests directly from here. Nice to have coherent behavior for all unit test frameworks!

Mocking

As for mocking, there are also some mocking frameworks that work with .NET Core:

I’ll pick Moq for this exercise. Just add the Moq package to your project. Here’s how to do simple mocking of an interface:

using Xunit;
using Moq;

[Fact]
public void Test()
{
var mock = new Mock<IService>();
mock
.Setup(x => x.Get())
.Returns("Hi there!");

var svc = mock.Object;

var result = svc.Get();

Assert.NotNull(result);

Assert.Equal("Hi there!", result);
}

Exactly the same as you’d do in a classic .NET unit test using mocking.

Validation

As for validations, I know of two libraries that work with .NET Core:

I’ll pick FluentAssertions. Here’s a simple example, for the same test:

using FluentAssertions;
using Moq;
using Xunit;

[Fact]
public void Test()
{
var mock = new Mock<IService>();
mock
.Setup(x => x.Get())
.Returns("Hi there!");

var svc = mock.Object;

var result = svc.Get();

result
.Should()
.BeOfType<string>()
.And
.Be("Hi there!");
}

Again, this should be familiar to everyone.

Conclusion

So, it’s no longer unit tests holding us back from .NET Core! Most of what is done today in classic .NET can be done in Core by now. There are still some more complex libraries, for interception, mapping, serialization, etc, that are not quite there yet, but I expect these to come with time.