C++ Wrappers for Windows Registry APIs

I uploaded on GitHub some C++ code of mine that wraps some Windows registry C-interface APIs, using RAII, STL classes like std::wstring and std::vector, and signals error conditions using exceptions.

Using these high-level C++ wrappers, you can easily access the Windows registry with simple code like this:

// Open a registry key
RegKey key{ 
    HKEY_CURRENT_USER, 
    LR"(SOFTWARE\MyKey\SubKey)" 
};

// Read a DWORD value
DWORD dw = key.GetDwordValue(L"MyValue1");

// Read a string value
wstring s = key.GetStringValue(L"MyValue2");

// Enumerate the values under the given key
auto values = key.EnumValues();

// etc.

On the May issue of MSDN Magazine you’ll find an article describing some of the techniques applied in this code.

EDIT 2017-05-02: Added link to my MSDN Magazine article.

Updates to the ATL/STL Unicode Encoding Conversion Code

I’ve updated my code on GitHub for converting between UTF-8, using STL std::string, and UTF-16, using ATL CStringW.

Now, on errors, the code throws instances of a custom exception class that is derived from std::runtime_error, and is capable of containing more information than a simple CAtlException.

Moreover, I’ve added a couple of overloads for converting from source string views (specified using an STL-style [start, finish) pointer range). This makes it possible to efficiently convert only portions of longer strings, without creating ad hoc CString or std::string instances to store those partial views.

 

Custom C++ String Pool Allocator on GitHub

I’ve uploaded on GitHub some C++ code of mine, implementing a custom string pool allocator.

The basic idea is to allocate big chunks of memory, and then serve single string allocations carving memory from inside those blocks, with a simple fast pointer increase.

There’s also a benchmark comparing this custom allocator vs. STL’s strings.

Custom string pool allocator benchmark results.
Custom string pool allocator benchmark results.

The results clearly show that both allocating strings that way, and sorting them, is faster than using the default std::wstring class.

 

The New C++11 u16string Doesn’t Play Well with Win32 APIs

Someone asked why in this article I used std::wstring instead of the new C++11 std::u16string for Unicode UTF-16 text.

The key point is that Win32 Unicode UTF-16 APIs use wchar_t as their code unit type; wstring is based on wchar_t, so it works fine with those APIs.

On the other hand, u16string is based on the char16_t type, which is a new built-in type introduced in C++11, and is different from wchar_t.

So, if you have a u16string variable and you try to use it with a Win32 Unicode API, e.g.:

// std::u16string s;
SetWindowText(hWnd, s.c_str());

Visual Studio 2015 complains (emphasis mine):

error C2664: ‘BOOL SetWindowTextW(HWND,LPCWSTR)’: cannot convert argument 2 from ‘const char16_t *’ to ‘LPCWSTR’

note: Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast

wchar_t is non-portable in the sense that its size isn’t specified by the standard; but, all in all, if you are invoking Win32 APIs you are already in an area of code that is non-portable (as Win32 APIs are Windows platform specific), so adding wstring (or even CString!) to that mix doesn’t change anything with respect to portability (or lack thereof).

 

Unicode Encoding Conversions Between ATL and STL Strings

At the Windows platform-specific level, when using C++ frameworks like ATL (or MFC), it makes sense to represent strings using CString (CStringW in Unicode builds, which should be the default in modern Windows C++ code).

On the other hand, in cross-platform C++ code, using std::string makes sense as well.

The same Unicode text can be encoded in UTF-16 when stored in CString(W), and in UTF-8 for std::string.

So, there’s a need to convert between those two Unicode representations. I discussed the details of such conversions in my MSDN Magazine article: “Unicode Encoding Conversions with STL Strings and Win32 APIs”However, the C++ code associated to that article used std::wstring for UTF-16 strings.

I’ve created a new repository on GitHub, where I uploaded reusable code (in the form of a convenient header-only module) for converting between UTF-16 and UTF-8, using CStringW for UTF-16, and std::string for UTF-8. Please feel free to check it out!

 

Passing std::vector’s Underlying Array to C APIs

Often, there’s a need to pass some data stored as an array from C++ to C-interface APIs. The “default” first-choice STL container for storing arrays in C++ is std::vector. So, how to pass the array content managed by std::vector to a C-interface API?

The Wrong Way I saw that kind of C++ code:

// v is a std::vector<BYTE>.
// Pass it to a C-interface API: pointer + size in bytes
DoSomethingC( 
  /* Some cast, e.g.: (BYTE*) */ &v, 
  sizeof(v) 
);

That’s wrong, in two ways: for both the pointer and the size. Let’s talk the size first: sizeof(v) represents the size, in bytes, of an instance of std::vector, which is in general different from the size in bytes of the array data managed by the vector. For example, suppose that a std::vector is implemented using three pointers, e.g. to begin of data, to end of data, and to end of reserved capacity; in this case, sizeof(v) would be sizeof(pointer) * 3, which is 8 (pointer size, in bytes, in 64-bit architectures) * 3 = 24 bytes on 64-bit architectures (4*3 = 12 bytes on 32-bit).

But what the author of that piece of code actually wanted was the size in bytes of the array data managed (pointed to) by the std::vector, which you can get multiplying the vector’s element count returned from v.size() by the size in bytes of a single vector element. For a vector<BYTE>, the value returned by v.size() is just fine (in fact, sizeof(BYTE) is one).

Now let’s discuss the address (pointer) problem. “&v” points to the beginning of the std::vector’s internal representation (i.e. the internal “guts” of std::vector), which is implementation-defined, and isn’t interesting at all for the purpose of that piece of code. Actually, misinterpreting the std::vector’s internal implementation details with the array data managed by the vector is dangerous, as in case of write access the called function will end up stomping the vector’s internal state with unrelated bytes. So, on return, the vector object will be in a corrupted and unusable state, and the memory previously owned by the vector will be leaked.

In case of read access, the vector’s internal state will be read, instead of the intended actual std::vector’s array content.

The presence of a cast is also a signal that something may be wrong in the user’s code, and maybe the C++ compiler was actually helping with a warning or an error message, but it was silenced instead.

So, how to fix that? Well, the pointer to the array data managed by std::vector can be retrieved calling the vector::data() method. This method is offered in both a const version for read-only access to the vector’s content, and in a non-const version, for read-write access.

The Correct Way So, the correct code to pass the std::vector’s underlying array data to a C-interface API expecting a pointer and a size would be for the case discussed above:

DoSomethingC(v.data(), v.size());

Or, if you have e.g. a std::vector<double> and the size parameter is expressed in bytes (instead of element count):

DoSomethingC(v.data(), v.size() * sizeof(double));

An alternative syntax to calling vector::data() would be “&v[0]”, although the intent using vector::data() seems clearer to me. Moreover, vector::data() works also for empty vectors, returning nullptr in this case. Instead, “&v[0]” triggers a “vector subscript out of range” debug assertion failure in MSVC when used on an empty vector (in fact, for an empty vector it doesn’t make sense to access the first item at index zero, as the vector is empty and there’s no first item).

&v[0] on an empty vector: debug assertion failure
&v[0] on an empty vector: debug assertion failure

What’s Wrong with My UTF-8 Strings in Visual Studio?

Probably nothing: Maybe you just have to tell Visual Studio it’s an UTF-8-encoded string!

The std::string class can be used to store UTF-8-encoded Unicode text.

For example:

std::string s{"LATIN SMALL LETTER E WITH GRAVE (U+00E8): \xC3\xA8"};

However, in the Locals window, instead of the expected è (Latin small letter e with grave, U+00E8), Visual Studio displays some (apparently) garbage characters, like a Latin capital letter A with tilde, followed by diaeresis.

UTF-8 String Misinterpreted in the Locals Window
UTF-8 String Misinterpreted in the Locals Window

Why is that? Is there a bug in our C++ code? Did we pick the wrong UTF-8 bytes for ‘è’?

No. The UTF-8 encoding for è (U+00E8) is exactly the 2-byte sequence 0xC3 0xA8, so the above C++ code is correct.

The problem is that Visual Studio doesn’t use the UTF-8 encoding to display that string in the Locals window. It turns out that VS is probably using the Windows-1252 code page (a character encoding commonly mislabeled as “ANSI” on Windows…). And, in this character encoding, the first byte 0xC3 is mapped to à (U+00C3: Latin capital letter A with tilde), and the second byte 0xA8 is mapped to ¨ diaeresis (U+00A8).

To display the string content using the correct UTF-8 encoding, you can use the explicit “s8” format specifier. For example, typing in the Command Window:

? &s[0],s8

the correct string is displayed, as this time the bytes in the std::string variable are interpreted as a UTF-8 sequence.

 

The s8 Format Specifier in the Command Window
The s8 Format Specifier in the Command Window

 

Similarly, the s8 format specifier can be used in the Watch window as well.

The s8 Format Specifier in the Watch Window
The s8 Format Specifier in the Watch Window

 

How Small Is Small Enough for the SSO?

Yesterday I wrote about the SSO.

A good question would be: “What is the maximum length threshold to trigger the SSO?” In other words: How long is too much for a string to enable the SSO?

Well, this limit length is not specified by the standard. However, for the sake of curiosity, in VS2015’s implementation, the limit is 15 for “narrow (i.e. char-based) strings, and 7 for “wide (i.e. wchar_t-based) strings.

This means that “meow”, “Connie” and “Commodore” are valid candidates for the SSO, as their lengths are less than 16 chars. On the other hand, considering the corresponding wide strings, L“meow” and L“Connie” are still SSO’ed, as their lengths are less than 8 wchar_ts; however, L“Commodore” breaks the 7 wide characters limit, so it’s not SSO’ed.

Thanks to Mr. STL for his clarification on the SSO length limit in Visual Studio’s implementation.

 

The Small String Optimization

How do “Connie” and “meow” differ from “The Commodore 64 is a great computer”?

(Don’t get me wrong: They are all great strings! 🙂 )

In several implementations, including the Visual C++’s one, the STL string classes are empowered by an interesting optimization: The Small String Optimization (SSO).

What does that mean?

Well, it basically means that small strings get a special treatment. In other words, there’s a difference in how strings like “Connie”, “meow” or “The Commodore 64 is a great computer” are allocated and stored by std::string.

In general, a typical string class allocates the storage for the string’s text dynamically from the heap, using new[]. In Visual Studio’s C/C++ run-time implementation on Windows, new[] calls malloc, which calls HeapAlloc (…which may probably call VirtualAlloc). The bottom line is that dynamically-allocating memory with new[] is a non-trivial task, that does have an overhead, and implies a trip down the Windows memory manager.

So, the std::string class says: “OK, for small strings, instead of taking a trip down the new[]-malloc-HeapAlloc-etc. “memory lane” 🙂 , let’s do something much faster and cooler! I, the std::string class, will reserve a small chunk of memory, a “small buffer” embedded inside std::string objects, and when strings are small enough, they will be kept (deep-copied) in that buffer, without triggering dynamic memory allocations.”

That’s a big saving! For example, for something like:

std::string s{"Connie"};

there’s no memory allocated on the heap! “Connie” is just stack-allocated. No new[], no malloc, no HeapAlloc, no trip down the Windows memory manager.

That’s kind of the equivalent of this C-ish code:

char buffer[ /* some short length */ ];
strcpy_s(buffer, "Connie");

No new[], no HeapAlloc, no virtual memory manager overhead! It’s just a simple snappy stack allocation, followed by a string copy.

But there’s more! In fact, having the string’s text embedded inside the std::string object offers great locality, better than chasing pointers to scattered memory blocks allocated on the heap. This is also very good when strings are stored in a std::vector, as small strings are basically stored in contiguous memory locations, and modern CPUs love blasting contiguous data in memory!

Optimizations similar to the SSO can be applied also to other data structures, for example: to vectors. This year’s CppCon had an interesting session discussing that: “High Performance Code 201: Hybrid Data Structures”.

I’ve prepared some C++ code implementing a simple benchmark to measure the effects of SSO. The results I got for 200,000-small-string vectors clearly show a significant advantage of STL strings for small strings. For example: in 64-bit build on an Intel i7 CPU @3.40GHz: vector::push_back time for ATL (CStringW) is 29 ms, while for STL (wstring) it’s just 14 ms: one half! Moreover, sorting times are 135 ms for ATL vs. 77 ms for the STL: again, a big win for the SSO implemented in the STL!

 

Beginner’s Bug: What’s Wrong with Brace-Init and std::vector?

Question: I have some old C++ code that creates a std::vector containing two NUL wchar_ts:

std::vector<wchar_t> v(2, L'\0');

This code works just fine. I tried to modernize it, using C++11’s brace-initialization syntax:

// Braces {} used instead of parentheses () 
// to initialize v
std::vector<wchar_t> v{2, L'\0'};

However, now I have a bug in my new C++ code! In fact, the brace-initialized vector now contains a wchar_t having code 2, and a NUL. But I wanted two NULs! What’s wrong with my code?

Answer: The problem is that the new C++11 brace initialization syntax (more precisely: direct-list-initialization) in this vector case stomps (prevails) on the initialize-with-N-copies-of-X constructor overload that was used in the original code. So, the vector gets initialized with the content of the braced initialization list: {2, L’\0’}, i.e. a wchar_t having code 2, and a NUL.

I’m sorry: to get the expected original behavior you have to stick with the classical C++98 parentheses initialization syntax.