Unicode UTF-16/UTF-8 Encoding Conversions: Win32 API vs. C++ Standard Library Performance

In this MSDN Magazine article, I showed how to convert Unicode text between UTF-16 and UTF-8 encodings using direct Win32 API calls (in particular, I discussed in details the use of the MultiByteToWideChar API).

In addition to that, the C++ standard library offers some classes to perform such conversions.

For example, you can combine std::codecvt_utf8_utf16 with std::wstring_convert to convert from a UTF-16-encoded std::wstring to a UTF-8 std::string:

#include <codecvt>  // for codecvt_utf8_utf16
#include <locale>   // for wstring_convert
#include <string>   // for string, wstring

...
std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>, wchar_t> conversion;
std::wstring utf16 = L"Some UTF-16 string";
std::string utf8 = conversion.to_bytes(utf16);

I developed some C++ code to convert many strings and measure the time spent for the conversions using the aforementioned C++ standard library classes vs. direct Win32 API calls, and the result is clearly in favor of the latter (code compiled with VS2015 in release build):

STL: 1050 ms

Win32: 379 ms  << — Clearly wins

 

One Reply to “Unicode UTF-16/UTF-8 Encoding Conversions: Win32 API vs. C++ Standard Library Performance”

  1. That’s… interesting. One would think that the Windows STL implementation would hook its conversion facilities into the Win32 API converters if at all possible, unless the ones provided by the STL are noticeably better than the Win32 ones in at least one field.

Leave a Reply

Your email address will not be published. Required fields are marked *