What Is the Encoding Used by the error_code message String?

std::system_error is an exception class introduced in C++11, that is thrown by various functions that interact with OS-specific APIs. The platform-dependent error code is represented using the std::error_code class (returned by system_error::code). The error_code::message method returns an explanatory string for the error code. So, what is the encoding used to store the text in the returned std::string object? UTF-8? Some other code-page?

To answer this question, I spelunked inside the MSVC STL implementation code, and found this _Winerror_message helper function that is used to get the description of a Windows error code.

STL _Winerror_message helper function
STL _Winerror_message helper function

This function first calls the FormatMessageW API to get the error message encoded in Unicode UTF-16. Then, the returned wchar_t-string is converted to a char-string, that is written to an output buffer allocated by the caller.

The conversion is done invoking the WideCharToMultiByte API, and the CodePage parameter is set to CP_ACP, meaning “the system default Windows ANSI code page” (copy-and-paste’d from the official MSDN documentation).

I think in modern C++ code, in general it’s a good practice to store UTF-8-encoded text in std::strings. The code pages are a source of subtle bugs, as there are many of them, they can change, and you end up getting garbage characters (mojibake) when different char-strings using different code pages are mixed and appended together (e.g.  when written to UTF-8 log files).

So, I’d have preferred using the CP_UTF8 flag with the WideCharToMultiByte call above, getting a char-string containing the error message encoded as a UTF-8 string.

However, this would cause mojibake bugs for C++/Windows code that uses cout or printf to print message strings, as this code assumes CP_ACP by default.

So, my point is still that char-strings should in general use the UTF-8 encoding; but unless the Windows console and cout/printf move to UTF-8 as their default encoding, it sounds like the current usage of CP_ACP in the error message string is understandable.

Anyway, due to the use of CP_ACP in the wchar_t-to-char string conversion discussed above, you should pay attention when writing error_code::message strings to UTF-8-encoded log files. Maybe the best thing would be writing custom code to get the message string from the error code identifier, and encoding it using UTF-8 (basically invoking FormatMessage followed by WideCharToMultiByte with CP_UTF8).

Thanks to Stephan T. Lavavej, Casey Carter and Billy O’Neal for e-mail communication on this issue.

 

Leave a Reply

Your email address will not be published. Required fields are marked *