When what you set is not what you get : SetEnvironmentVariable and getenv

Mixing SetEnvironmentVariable and getenv is asking for trouble, as we recently found to our dismay (and exasperation!). It took some serious debugging to figure out the actual problem – let’s see if you can figure it out.

Here’s some C++/CLI code that uses getenv to read the value of an environment variable and print it to the console.

   1: public ref class EnvironmentVariablePrinter


   2: {


   3:     public:


   4:         void PrintEnv(String ^key)


   5:         {


   6:              IntPtr ptr = Marshal::StringToHGlobalAnsi(key);


   7:              const char *ckey = (const char *)ptr.ToPointer();


   8:              const char *cval = getenv(ckey);


   9:              string val = cval == NULL ? "" : string(cval);


  10:  


  11:              cout << "Key is " << ckey << " and value is " << val;


  12:  


  13:              Marshal::FreeHGlobal(ptr);    


  14:         }


  15: };

PrintEnv merely converts the .NET string into an unmanaged one and calls getenv, printing the returned value to the console.

And here’s some C# code that tests the class.

   1: class Test


   2: {


   3:     const string key = "MyKey";


   4:     public static void Main()


   5:     {


   6:         Environment.SetEnvironmentVariable(key, "MyValue");


   7:         PrintValue();


   8:     }


   9:  


  10:     private static void PrintValue()


  11:     {


  12:         EnvironmentVariablePrinter er = new EnvironmentVariablePrinter();


  13:         er.PrintEnv(key);


  14:     }


  15: }

The code uses System.Environment.SetEnvironmentVariable to set the value of the variable and then calls the C++/CLI code to verify that it prints the correct value. And of course, being written in different languages, the two pieces of code must reside in different projects, say CPlusPlusLib.dll and ConsoleApplication.exe, with the latter referencing the former.

No surprises here – this works as expected and prints “Key is MyKey and value is MyValue”.

However, a seemingly harmless change breaks the code big time.

   1: class Test


   2: {


   3:     const string key = "MyKey";


   4:     public static void Main()


   5:     {


   6:         Environment.SetEnvironmentVariable(key, "MyValue");


   7:         EnvironmentVariablePrinter er = new EnvironmentVariablePrinter();


   8:         er.PrintEnv(key);


   9:     }


  10: }

All I’ve done is inlining of PrintValue, yet running this code prints “Key is MyKey and value is” – getenv is now returning NULL instead of “MyValue”.

It gets even more interesting.

   1: class Test


   2: {


   3:     const string key = "MyKey";


   4:     public static void Main()


   5:     {


   6:         Environment.SetEnvironmentVariable(key, "MyValue");


   7:         EnvironmentVariablePrinter er = null;


   8:         PrintValue(); 


   9:     }


  10:  


  11:     private static void PrintValue()


  12:     {


  13:         EnvironmentVariablePrinter er = new EnvironmentVariablePrinter();


  14:         er.PrintEnv(key);


  15:     }


  16: }



This doesn’t work either – the mere declaration of EnvironmentVariablePrinter inside Main makes getenv return NULL for “MyKey”. This can’t be good, can it?



Actually yes, because that is a valuable clue – it means the change in behavior has something to do with JITting. As long as all code that references EnvironmentVariablePrinter is in a separate method, everything works fine (in debug mode, atleast). Things start going south when any such code is in Main itself.



What would the JITter do differently in the two scenarios? Load the assembly containing EnvironmentVariablePrinter (CPlusPlusLib.dll) at different times, of course. When all code referencing EnvironmmentVariablePrinter is inside PrintValue, it will have to load the DLL only when JITting PrintValue, whereas in the other case, it will have to load it when JITting Main. JITting Main obviously occurs much before JITting PrintValue, so DLL load time (relative to other code) is one big difference between the two scenarios that occurs because of JITting.



Why would loading CPlusPlusLib.dll a little early make getenv return NULL?



To understand that, you’ll first have to know how the getenv function works. Windows has APIs to set and get environment variables (SetEnvironmentVariable/GetEnvironmentVariable), and the .NET method P/Invokes into the Windows API to set and get values. getenv, on the other hand, is a CRT function, and does not delegate to the Windows API. Instead, the CRT gets all environment variables and their values when it is starting up (using the GetEnvironmentStrings Windows API), and copies them into its own data structures (MSVCR80!environ). getenv then works on the copied data from then on.



Now do you see the problem? When CPlusPlusLib.dll is loaded early (when JITting Main), the CRT also gets loaded as one of its dependencies, and the startup code that copies environment variables runs right away. At that point, Main hasn’t even been JITted yet, so there’s no way our call to System.Environment.SetEnvironmentVariable could have run by that time. And when it actually runs, it’s too late – the CRT environ block would have been updated much earlier, and calling the Windows API’s SetEnvironmentVariable wouldn’t have any effect on the cached values. When getenv runs, it looks in the cached values and returns NULL.



It’s easy to see why it works in the first case now – CRT loading occurs when JITting PrintValue, and that occurs after our call to SetEnvironmentVariable has executed. Which means that when it calls GetEnvironmentStrings as part of startup, it gets the variable (and its value) that we just set.



Nasty, ain’t it? The actual scenario was a lot more messy – things suddenly stopped working when we linked to a DLL ported to VS2008. We actually figured the problem backwards – we first saw that the CRT load time was different, theorized how getenv works, verified the theory by stepping through the assembly code and looking at the environ block, and once we realized the problem, figured out what was causing early loading of the CRT. Windbg was awesome for debugging this – things would have been very difficult if not for sxe ld:MSVCR90 and x MSVCR80!environ.



The fix was rather simple – in our case, we merely had to move code that set environment variable before CRT load. There’s another twist though; mscorwks.dll, which is the heart of the CLR, loads MSVCR80 when it loads, and you can’t set your environment variables before that, not from managed code anyway. Fortunately, in our case, the getenv call is from a library that links to MSVCR90, so as long as we set the environment variable before that version of the CRT loads, we’re good to go. Until the CLR gets linked to MSVCR90, anyway :).

5 thoughts on “When what you set is not what you get : SetEnvironmentVariable and getenv”

  1. I don’t belong to .Net, but I’m surprised if there’s no equivalent of getenv (ie., setenv) that is available for use?

    In the non .net world, it is a paradigm to use only setenv/getenv to set/get Environment vars and not through Win APIs (just for the exact reason that you discovered).

  2. Honestly, I was surprised when I found that getenv did not call the Windows API.

    I don’t see why this is encouraged – in one instance, there were two versions of the CRT in a process, and each of them had its own copy, and the copies were different.

    The problem goes away if everyone calls the Windows (or equivalent OS) API, so I’m curious why it isn’t recommended.

  3. Possibly, optimization?. I’ve rarely had to set an ENV variable from within my application (for use in my own application).

    On the other hand, reading an ENV variable is a common act – say for temp folders. So I would choose to cache it if possible. This seems more common than the case we are discussing.

    I would also be against this optimization, if there isn’t a way around; but given that setenv takes care of it and that the mandate is to, “Not mix up the CRT and Win APIs for ENVs”, I’m ready to accept this for the benefit.

    Having two CRT in a process itself is strange!! (again, I’m not sure about .net). There are so many other things that will go for a toss if you have two CRTs into the same process — e.g., file descriptors? heap pointers? huh!!

  4. string val = cval == NULL ? “” : string(cval);

    Oh man, you should aviod such a line at all costs. I really enjoyed reading your posts until I got to this little line. Although it’s easy to understand, I would suggest to use brackets for clarity. As a general rule of the thumb I tend to code the simplest and easiest way such that even complete morons with no idea in coding can look at your product and understand some bits of it.

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>