Another Look at Event Source Performance

EDIT: Added/changed things in the section near the end on Optimization #4 on 2013-11-06

In this post, Muhammad Shujaat Siddiqi talks about high volume ETW performance and the following three optimizations.

Optimization # 1: Use IsEnabled() before WriteEvent()

Optimization # 2: Convert static field fetch to local member fetch

Optimization #3: Minimize the number of EventSources in the application

Optimization # 4: Avoid fallback WriteEvent method with Object Array Parameter

He does a great job explaining each of them, but I disagree with the second optimization and wanted to add some comments on the others based on some rough metrics I ran to check performance.

To start with the summary, EventSource plays very nice with ETW and so maintains amazing, absolutely amazing, performance characteristics.

Optimization # 1: Use IsEnabled() before WriteEvent()

If all you’re doing is calling WriteEvent, there’s no preparatory work, then I haven’t seen a significant performance difference. Basically, the first thing WriteEvent does is check whether it’s enabled, and the one extra method call is awfully hard to measure.

However, if you’re doing preparatory work that has any expense, you should absolutely use this. Preparatory work most often happens in complex calls, where a public method marked with the NonEvent attribute calls a private method that is the actual Event definition. In this case, you need to check whether the class is enabled in that public method, before the preparatory work (I realize this might be obvious, but it’s important).

As Muhammad pointed out, if you have keywords or levels associated with the event, you can check them. But the overarching principle is simplicity, so I think only applies in the far more rare verbose/keyword scenario than in the more common scenario where you use the defaults except for provider name and event id.

That said, if you always use IsEnabled() as soon as you arrive in your EventSource class, you will remember to do it on the rare occasion it matters.

Optimization # 2: Convert static field fetch to local member fetch

Muhammad recommends instantiating an instance of the logger in each class that uses it, instead of accessing a static field in the logger class. I tested instantiating the instance vs. a call to the static method for the reverse reason – because some documentation suggested it was slower to instantiate rather than reuse. I could not measure a difference, so arguably this is a matter of opinion. I stated that disagree with Muhammad, not that he is wrong.

I believe there are two principles that override performance, since the perf difference is somewhere between small and non-existent.

The first is that I want to minimize the greatest possible extent code (textual) dependency between my application classes and my logger. Unfortunately, for some technical reasons EventSource does not easily support interfaces, and therefore does not easily support dependency injection (look for an upcoming blog post on that). So, you’re stuck with the dependency of a direct call, or the actual class name. Since the direct call is easier to fix later if you choose to introduce the complexity of DI with EventSource, I believe it is preferable.

I also do not want multiple copies of the logger in case a future way I am logging requires shared state. So, my preferred way to instantiate blocks creation of multiple instances:

[EventSource(Name = "KadGen-EtwSamples-NormalEventSource")]


public class NormalEventSource : EventSource


{


    private static readonly Lazy<NormalEventSource> Instance = new Lazy<NormalEventSource>(() => new NormalEventSource());


    private NormalEventSource()


    {


    }


    public static NormalEventSource Log


    {


        get { return Instance.Value; }


    }    static public NormalEventSource Log = new NormalEventSource();


 


    [Event(1)]


    public void AccessByPrimaryKey(int PrimaryKey, string TableName)


    {


        if (IsEnabled()) WriteEvent(1, PrimaryKey, TableName);


    }   


 


}


 


 

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }

Please determine the best approach for your team, and be consistent.

Optimization #3: Minimize the number of EventSources in the application

I didn’t actually measure this, because I think performance here is irrelevant.

Each EventSource class maps to a single ETW provider with its name, id and manifest.

Designing your providers is one of the most critical aspects of creating your ETW system. I believe one per component and some sharing (with associated hazards), and Muhammad suggests one per type, which is far more than I would use. Regardless, there is no other aspect – not criticality level, not channel, not opcodes, not keywords – that is in the ballpark of provider names (ids) when it comes to your design.

Later in a consumer, you will almost certainly sort and filter on the provider names. You will collect data based on provider names, and the more you have the more chance that you’ll miss key events.

Create exactly the number of EventSources you need and no more. When in doubt, take the path that results in a simpler strategy.

Optimization # 4: Avoid fallback WriteEvent method with Object Array Parameter

EDIT: Added/changed things in this section 2013-11-6

This is the place you can change the performance characteristic of your ETW tracing – for the good or for the bad.

As Muhammad explains well, the WriteEvent method has a finite number of overloads. If you pass parameters that do not match one of these overloads, a paramarray of objects is used. Now, I didn’t anything close to the 10-20 times proposed in the spec, but it double the time, which is enough to get my attention.

There are two scenarios where you’ll wind up with the object overload. The simplest to fix is passing enums. Accept the enum into your EventSource method as the corresponding enum so that the manifest correctly manages the enum. And then cast the enum to an int in the WriteEvent call like this:

[Event(1)]


public void AccessByPrimaryKey(int PrimaryKey, string TableName, ConnectionState ConnectionState)


{


    if (IsEnabled()) WriteEvent(1, PrimaryKey, TableName, (int)ConnectionState);


}


 



If I check the overloads for WriteEvent, there is an (int, string, int) overload so this call will not hit the object overload. But, it’s quite easy to create a collection of parameters that don’t match one of the overloads. I do not believe that it is appropriate to rearrange parameters to your public method just to match an overload – and the parameters to the WriteEvent call must match the order of parameters passed to your public method.



The fix for this isn’t quite as trivial – it’s easy, but ugly. This is explained in the EventSource specification, and section 5.6.2.2 Creating fast WriteEvent overloads explains how to create the overloads. I always track down the spec as the attachment to this post on Vance Morrison’s blog.



Summary



Muhammad provided a very good, extremely readable, accessible discussion of potential EventSource/ETW performance issues.



· None of these performance tips will be observable in your application if you use a listener other than the blazingly fast ETW.



· Based on the metrics I ran a while back, I believe the only one of these performance tips worth worrying about is avoiding the object overload, and it isn’t as bad as anticipated in the spec.



· There are other considerations, at least consistency, that are important for the other three items.



As you dig deeper into ETW, side issues like this come up. The isolation of all EventSource aspects into the single class – inherent in its design – means these complexities always affect only a single location in your application if you encounter them.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>