Scheduled task would not run

The scheduled task for running ntbackup on our fileserver would not run anymore. It would stop with a return code of 0x80, and nothing worthwhile in the log file or event log. It ran fine for months, apart from the occasional hickup such as having no more free tapes in the loader, or a tape not being imported correctly.


I tried running the backup batch file manually, and that worked fine. I ran it under the dedicated backup account, and that worked fine too. But it wouldn’t run automatically, and starting the task manually when logged in didn’t work either. I couldn’t find any useful debugging info so I turned to google. I found several people who has similar problems, but most had to do with a bug in the HID service which somehow interfered with ntbackup. I looked around some more and finally I found something that ended up giving me the final clue.


I opened up the task manager and noticed that there were 21 instances of xcopy still running, 23 instances of psexec, and 3 instances of ntbackup. These were probably left over from some tests I had done with the backup script of one of the new servers (a backup domain controller) which uses the 3 aforementioned programs.


After killing these dead processes I tried to run the backup task again, and everything worked fine this time. In my naivety, I had assumed that ending a task would end the processes it had spawned, but that seems to have been a mistake. Ah well, everything is working again, and now that I know the cause, I can prevent this from happening again.


This is why I try to understand a problem, rather than just rebooting the machine. A reboot would have fixed the problem in less time then I needed to figure this out. But then I wouldn’t have known what happened (I hate that) and I wouldn’t be able to prevent this from happening again. I would have been stuck in a regular but unknown problem -> reboot cycle.

Practial ATL: Implementing a non-standard class object

In my previous article in this series, I explained how class objects are associated with COM objects, and how it all fits together.


Armed with that knowledge, I can now demonstrate how to provide a custom class object that will allow us to support parameterized construction.


If you haven’t already read my previous article on class objects, please do so first because I am going to assume you did J


The non-standard class object interface: IStuffCreator


A class object can have any interface you want, just like the COM object we have created earlier. It is just another COM object after all. My custom interface looks like this:


[
      object,
      uuid(6DD69CDB-3128-432b-B335-773A287E6F06),
      oleautomation,
      helpstring(“IStuffCreator Interface”),
      pointer_default(unique)
]
interface IStuffCreator : IUnknown {
  [id(1), helpstring(“method MakeMeAStuff”)]
    HRESULT MakeMeAStuff(
      [in] BSTR name,
      [in] REFIID riid,
      [out, iid_is(riid)] IUnknown** ppStuff);
};


The bits between the square brackets are the attributes that are applied to this interface. ‘Object’ specifies that it is a COM interface. The uuid specifies the GUID that identifies the interface. ‘oleautomation’ specifies that the arguments being used are all automation compatible. The meaning of  ‘helpstring’ is pretty obvious, and pointer_default specifies the assumptions that the marshaller can make about pointers.


If you want to know the details about it, then have a look in MSDN for MIDL attributes.


The interface itself is very simple. Instead of deriving from IDispatch like the IStuff interface, this interface derives from IUnknown. Deriving from IDispatch would only complicate matters without any benefit.


The interface has only one method, which is responsible for handing out new stuffs.


The parameter that will be used for constructing the CStuff instance itself is the ‘name’ parameter. If the CStuff would need multiple parameters, they would have to be part of the parameter list so that they can be used at the time of construction. The riid and ppStuff parameters have the same meaning as those of CoCreateInstance.


The declaration of CStuffCreator


If we want to implement this interface, we add a new C++ class with the name ‘CStuffCreator’ to our project, and implement our IStuffCreator interface


class CStuffCreator:
  public IStuffCreator,
  public CComObjectRootEx<CComGlobalsThreadModel>
{
public:

  CStuffCreator(void){}
  virtual ~CStuffCreator(void){}

  BEGIN_COM_MAP(CStuffCreator)
    COM_INTERFACE_ENTRY(IStuffCreator)
  END_COM_MAP()

  //IStuffCreator methods
  HRESULT STDMETHODCALLTYPE MakeMeAStuff(
    BSTR name,
    REFIID riid,
    IUnknown **ppStuff);

  // helper method
  void SetVoid(void* pv);
  _ATL_CREATORFUNC* m_pfnCreateInstance;
};


As you can see, it is pretty simple. The CComObjectRootEx base class helps with reference counting, and is the same that is used by the other COM objects in our project.


We add IStuffCreator to the interface map using the ATL macros, and then we simply declare the single MakeMeAStuff method that we need to implement.


The SetVoid method and the function pointer are part of the default C++ interface for class objects in ATL, and are explained in my previous article.


The implementation


The implementation of CStuffCreator is even simpler than the declaration:


HRESULT STDMETHODCALLTYPE CStuffCreator::MakeMeAStuff(


  BSTR name,


  REFIID riid,


  IUnknown **ppStuff)


{


  CComPtr<IUnknown> newStuff;


  HRESULT hr = m_pfnCreateInstance(


                 NULL, __uuidof(IUnknown), (void**)&newStuff);


  if(FAILED(hr))


    return hr;


 


  return newStuff->QueryInterface(riid, (void**)ppStuff);


}


 


void CStuffCreator::SetVoid(void* pv)


{


  m_pfnCreateInstance = (_ATL_CREATORFUNC*)pv;


}


Ignore the ‘name’ parameter for now. That will be used when we are going to initialize the new instance. The bare functionality of our class object is to use the function pointer to create a new instance. I could have done this in 1 step instead of 2 (the QueryInterface isn’t necessary at this point) but this way I can slide in some additional functionality later on.


And you can see now that the class object will receive the creator function pointer via the ‘SetVoid’ method. This mechanism was explained in my previous article.


Parameterized construction


The point of using a non-standard class object in this demonstration was to be able to create objects that need some sort of initialization before the client has access to them. Normally, ATL supports only default constructors using, because IClassFactory is a generic interface.


For parameterized constructors, we are going to have to do something special.


There are essentially 2 ways we can do this. The first is to change CStuff to have a parameterized constructor. But ATL can’t handle that, and it would mean we have to abandon the incredible convenience of being able to use the ATL::Creator<T> class, ATL::CComObject class, and other helper classes that take the pain out of COM.


Don Box wrote an article about parameterized construction, many years ago, and this was the approach he took. Of course, ATL was much simpler in those days, and abandoning it was perhaps less of an inconvenience.


The approach I am taking is much simpler: I cheat!


That’s right. I cheat. I am not going to implement parameterized constructors. I am going to fake it. But from the client’s point of view, it’s going to look exactly the same, so he won’t mind. And from the server’s point of view, it will be much simpler to implement, so everybody wins.


The second approach to implementing parameterized construction is to implement an internal interface on CStuff, which will be used for initializing it. The new CStuff instance will not be accessible to anyone until the first interface pointer is handed out by the class object. And we are the ones implementing the class interface, so we can make sure that the new instance is initialized properly before it is released into the wild.


The initialization interface


The interface for initializing a CStuff is pretty simple (I left out the header for clarity):


interface IStuffInit : IUnknown {


  [id(1), helpstring(“method InitStuffInstance”)]


    HRESULT InitStuffInstance(


      [in] BSTR Name);


};


Our CStuff instances are supposed to have a name, so that’s what needs to be in the init interface. If they would also need an address, a hat, and a pair of pants, we would have to put those in the parameter list of the InitStuffInstance method as well.


Implementing the initialization interface


Implementing the interface in CStuff is easy I highlighted the changes that have to be made to the declaration.


// CStuff


 


class ATL_NO_VTABLE CStuff :


  public CComObjectRootEx<CComMultiThreadModel>,


  public CComCoClass<CStuff, &CLSID_Stuff>,


  public IDispatchImpl<IStuff, &IID_IStuff, &LIBID_Stuff_ServerLib, /*wMajor =*/ 1, /*wMinor =*/ 0>,


  public IStuffInit


{


private:


  CComBSTR m_Name;


public:


DECLARE_CLASSFACTORY_EX(CStuffCreator)


 


// removed irrelevant stuff


 


BEGIN_COM_MAP(CStuff)


  COM_INTERFACE_ENTRY(IStuff)


  COM_INTERFACE_ENTRY(IDispatch)


  COM_INTERFACE_ENTRY(IStuffInit)


END_COM_MAP()


 


// removed irrelevant stuff


  HRESULT STDMETHODCALLTYPE InitStuffInstance(BSTR Name);


};


First we add IStuffInit to the inheritance list, we add it to the COM map, and then we add the declaration for InitStuffInstance.


Of course, the name has to be part of the objects state, so we add a CComBSTR variable to hold it.


And finally, we use the DECLARE_CLASSFACTORY_EX macro to change the class object for CStuff from CComClassFactory to CStuffCreator.


The actual implementation of the interface consists of the simple code


STDMETHODIMP CStuff::InitStuffInstance(BSTR Name)


{


  m_Name = Name;


  return S_OK;


}


Note the ease with which CComBSTR allows us to perform simple string operations.


Initializing the new CStuff object


Now that there is a dedicated function to handle the initialization, the changes to CStuffCreator are trivial:


HRESULT STDMETHODCALLTYPE CStuffCreator::MakeMeAStuff(


  BSTR name,


  REFIID riid,


  IUnknown **ppStuff)


{


  CComPtr<IStuffInit> newStuff;


  HRESULT hr = m_pfnCreateInstance(


                  NULL,__uuidof(IStuffInit), (void**)&newStuff);


  if(FAILED(hr))


    return hr;


 


  hr = newStuff->InitStuffInstance(name);


  if(FAILED(hr))


    return hr;


 


  return newStuff->QueryInterface(riid, (void**)ppStuff);


}


What about double initialization


By now you might be wondering: But what if the client application retrieves this interface and performs another initialization?


However, this will not be possible.


It’s true that I put the interface in our IDL file, and that I implemented it in the CStuff object. But I did not put that interface in the library declaration, nor did I put it in the interface list of the coclass.


So for all practical purposes, this interface is not accessible to the client application. If you are really paranoid, you could argue that an attacker might know the interface GUID, and guess the interface definition, and with some trial and error make it work.


Even then, the fix is quite simple. You can add a boolean to your class that indicates if the object was initialized or not. Set it to false in the constructor, and then the InitStuffInstance can either perform initialization if it is false and set it to true, or return E_FAIL if it is already true.


You don’t even have to care about race conditions, because there aren’t any. Only the class object can perform the real initialization. And since no one else has an interface pointer yet, there can’t be a race.


The client side


Creating a new CStuff has changed a little bit, because now we can’t use CoCreateInstance anymore. So construction is now done in 2 stages:


    CComPtr<IStuffCreator> stuffClass;


    hr = CoGetClassObject(


           __uuidof(Stuff), CLSCTX_ALL, NULL,


           __uuidof(IStuffCreator), (void**)&stuffClass);


    if(FAILED(hr))


      return hr;


 


    //make a new stuff   


    CComBSTR stuffName = L“Kato”;


    CComPtr<IStuff> newStuffUnk;


    hr = stuffClass->MakeMeAStuff(


           stuffName.m_str,


           (GUID*)&__uuidof(IStuff),


           (IUnknown**)&newStuffUnk);


    if(FAILED(hr))


      return hr;


It’s still very simple though. First we get the class object, and then we use the IStuffCreator interface to create the new CStuff.


Server lifetime management


Everything works, hip hip hurray… until you destroy the first CStuff instance, and then try to create another one. That is where it all goes pear shaped.


The problem is that we haven’t taken care of managing the lifetime of the server (the exe). The key issue here is the module lock reference count.


Each COM module (exe or dll) has a global reference count to keep track of how many objects are ‘alive’. COM objects exist physically inside a DLL or EXE. So the EXE has to stay alive as long as there are living objects inside. Conversely, as soon as there are no living objects anymore, the EXE should shut down.


What happens in our example is that CStuff raises the module ref count when it is created. And when the CStuff instance destroys itself, it lowers the module ref count. If you remember correctly, the class object itself uses CComObjectNoLock as its COM core. This means that its existence does not influence the module lifetime.


So when the CStuff instance destroys itself and lowers the module ref count, the ref count reaches 0 and the server EXE shuts down.


The client is unaware of this and tries to use the class object again. It still thinks it has a valid interface pointer, but the underlying class object is long gone and there will be a critical error in the client application.


You might say ‘well duh, then don’t use CComObjectNoLock, doofus’ but that has the opposite effect. The COM runtime will always have the interface pointer that was used for registering the class object with CoRegisterClassObject. This means that the class object would not destroy as long as the server lives. And the server would keep living as long as the class object was around. This is clearly not the answer.


IExternalConnection to the rescue


The key to solving this problem is a rarely used standard interface. IExternalConnection is an interface that can be used by an object to track how many external connections there are to itself. This is basically an alternative to the standard object reference count.


There are 2 methods in this interface:


  STDMETHODIMP_(DWORD) AddConnection(


                         DWORD extconn, DWORD dwReserved);


  STDMETHODIMP_(DWORD) ReleaseConnection(


                         DWORD extconn,


                         DWORD dwReserved,


                         BOOL bLastUnlockReleases);


 


If a client requests any class object, the COM runtime will retrieve it. But before the interface pointer is released to the client, it will request some interfaces itself for housekeeping purposes. One of those interfaces is IExternalConnection.


If a client requests an interface pointer to a class object, the marshaller will call AddConnection when it hands out the pointer, and ReleaseConnection when the pointer is destroyed.


So all we have to do is to make sure that a call to AddConnection will increment the module lock count, and a call to ReleaseConnection releases the module lock count.


Implementing IExternalConnection


If you look in MSDN, you won’t find it. It is undocumented, but the ATL headers contain an implementation for IExternalConnection. It’s called IExternalConnectionImpl<T>.


To use it, we only have to change CStuffCreator so that it inherits from it, and add it to the COM interface map.


class CStuffCreator:


  public IStuffCreator,


  public IExternalConnectionImpl<CStuffCreator>,


  public CComObjectRootEx<CComGlobalsThreadModel>


{


public:


 


  //removed irrelevant stuff


  BEGIN_COM_MAP(CStuffCreator)


    COM_INTERFACE_ENTRY(IStuffCreator)


    COM_INTERFACE_ENTRY(IExternalConnection)


  END_COM_MAP()


 


  //IExeternalConnectionImpl<T> extension


  void STDMETHODCALLTYPE OnReleaseConnection(


         bool bThisIsLastUnlock, bool bLastUnlockReleases);


  void STDMETHODCALLTYPE OnAddConnection(


         bool bThisIsFirstLock);


 


  //removed irrelevant stuff


};


That is really all there is to it. The default implementation does not modify the module lock count. But it has 2 extension points that we can override. So that’s what we do to make sure the module lock count is managed properly.


void STDMETHODCALLTYPE CStuffCreator::OnAddConnection(


       bool bThisIsFirstLock)


{


  _pAtlModule->Lock();


}


 


void STDMETHODCALLTYPE CStuffCreator::OnReleaseConnection(


       bool bThisIsLastUnlock, bool bLastUnlockReleases)


{


  _pAtlModule->Unlock();


  IExternalConnectionImpl<CStuffCreator>::OnReleaseConnection(


    bThisIsLastUnlock,  bLastUnlockReleases);


}


The only special thing that still needs mentioning is that OnReleaseConnection delegates back to the default implementation in IExternalConnectionImpl. It does this because there is something that needs to be done when the last external connection is closed.


  void OnReleaseConnection(


       bool bThisIsLastUnlock, bool bLastUnlockReleases)


  {


    if (bThisIsLastUnlock && bLastUnlockReleases)


         CoDisconnectObject(static_cast<T*>(this)->GetUnknown(), 0);


  }


If the last connection closes, the class object needs to call CoDisConnectObject to destroy its marshalling stub. Now I could copy paste this code into my implementation of OnReleaseConnection, but it is simpler to just delegate back to the base class for stuff that needs doing anyway.


The result


Now that we have provided this implementation of IExternalConnection, the server EXE will stay alive until all CStuff instances are gone, and noone is using the class object anymore. Only then will the EXE terminate.


CComClassFactory revisited


If you look at the interface implemented by CComClassFactory, you’ll notice that IExternalConnection is missing. And yet, server lifetime of CComClassFactory is managed correctly.


The reason for this is that the COM runtime treats IClassFactory class objects differently from other class objects. It will check if the class object supports IClassFactory. And if it does, it calls IClassFactory::LockServer to indicate that someone is using the class object. And it unlocks the class object when the class object reference is released again.


In a way, it uses the IClassFactory in almost the same way as it would use IExternalConnection. And in a way it is too bad that this functionality was included in IClassFactory directly, instead of making IClassFactory inherit from IExternalConnection. The reasons for this are lost in the mists of time. But whatever the reasons, this is why IClassFactory doesn’t need IExternalConnection.


Conclusion


As you can see, implementing a non standard class object is not so hard. As long as you have a basic understanding of the underlying COM principles, ATL takes care of all the dirty work.


And by using these techniques, we can implement COM objects that need initialization before they can safely be used by the client. The source code for this article is available under the MIT license as usual.


And again I’d like to add a thank you for Microsoft’s Tim Springfield and Amit Mohindra


 


 

Practical ATL: Understanding the class object

In my previous article in this series, I explained how to create a simple COM server and implement a method that returns an enumerator object.


It was all done very easily, but the ATL wizards hide some of the things that you really ought to know for my next article. For starters: where is the class object?


The class object is the COM object responsible for creating new instances of the CStuff COM object that I demonstrated in a previous article. But we didn’t have to write any code for it, nor is it even visible… so what’s up with that?


Some more on class objects


The class object of a COM server is the object that is actually responsible handing out instances of the COM object. The class object is a COM object itself, but only 1 instance of it normally exists in a module (DLL or EXE) per defined COM object type.


For example, the ‘Stuff’ COM object that I implemented in my previous article uses the standard class factory with IClassFactory as the interface for handing out new CStuff instances.


The most important method in IClassFactory is ‘CreateInstance’ which simply creates a new instance of the COM object and then passes an interface pointer back to the caller.


Normally, this process is invisible to the client application, because ‘CoCreateInstance’ uses IClassFactory behind the scenes so that the client doesn’t have to bother with it.


First it calls ‘CoGetClassObject’ to get the class object for the requested COM object, and then it uses the IClassFactory interface of that class object to create a new instance. That instance is then passed to the caller, as an interface pointer of the type that was requested in the call to ‘CoCreateInstance’.


I think it is safe to say that ‘IClassFactory’ (or one of its relatives) is the interface on the class object of 99.99% of all COM objects that have a class object.


The default class factory


By default, CStuff uses CComClassFactory as its default class object. But looking at the declaration of CStuff, it is not immediately apparent how it does this.


CStuff derives from CCoClass<T>, which has the macro DECLARE_CLASSFACTORY in its implementation.


For EXE servers, this macro expands to DECLARE_CLASSFACTORY_EX(ATL::CComClassFactory), which in turn expands to


typedef ATL::CComCreator< ATL::CComObjectNoLock< ATL::CComClassFactory > > _ClassFactoryCreatorClass;


DLL servers use CComObjectCached instead of CComObjectNoLock, but I don’t cover DLL servers in this article.


As you can remember from the first article that featured CStuff, the C++ COM classes are missing the IUnknown core by default, because the implementation of that core depends on how we want to the object to behave.


This means we have to shove our class into a COM object to make it whole. For class objects, the CComObjectNoLock is most appropriate, because the lifetime of the object should not contribute to the lifetime of the server executable. Otherwise there would be a deadlock.


The server would start, and create the class object. And then the server would continue to live until the class object was destroyed. But since the COM runtime will always have a reference to the class object, the reference count would never reach zero, the class object would never be destroyed, and the server would never be able to shut down.


And finally, this class definition is shoved into a creator object which will help us by creating a properly initialized instance of the class object itself via its static CreateInstance function.


Associating the COM object C++ class with the class factory


So the COM object C++ class has a typedef that identifies the class object type. But how does the executable use this fact?.


This is where things get a little hairy, and possibly start to make ominous squishy sounds J


After the class declaration of CStuff, there is the innocuous line OBJECT_ENTRY_AUTO(__uuidof(Stuff), CStuff)


The OBJECT_ENTRY_AUTO is a macro that expands to this:


#define OBJECT_ENTRY_AUTO(clsid, class) \
      __declspec(selectany) ATL::_ATL_OBJMAP_ENTRY __objMap_##class = {&clsid, class::UpdateRegistry, class::_ClassFactoryCreatorClass::CreateInstance, class::_CreatorClass::CreateInstance, NULL, 0, class::GetObjectDescription, class::GetCategoryMap, class::ObjectMain }; \
      extern “C” __declspec(allocate(“ATL$__m”)) __declspec(selectany) ATL::_ATL_OBJMAP_ENTRY* const __pobjMap_##class = &__objMap_##class; \
      OBJECT_ENTRY_PRAGMA(class)


Don’t worry if the sight of this code makes you nauseous. That’s normal. I highlighted the important parts


What it does is simple. It’s just how it does it that is ugly. It creates a map entry for the ATL innards of our server executable. In this map entry, it associates the static method for creating the class object with the static method for creating an object instance.


Creating the class object


Now that the class object and the COM object are associated with each other, the next bit of code in the ATL headers is where the magic happens:


  _ATL_OBJMAP_ENTRY* pEntry;
  if (m_pObjMap != NULL)
  {
    pEntry = m_pObjMap;
    while (pEntry->pclsid != NULL && hr == S_OK)
    {
      hr = pEntry->RegisterClassObject(dwClsContext, dwFlags);
      pEntry++;
    }
  }


When the ATL module initializes, it will iterate over all elements in the object map and register each one. And this registration process entails the following:


    IUnknown* p = NULL;
    if (pfnGetClassObject == NULL)
      return S_OK;
    HRESULT hRes = pfnGetClassObject(
                         pfnCreateInstance,
                         __uuidof(IUnknown),
                         (LPVOID*) &p);
    if (SUCCEEDED(hRes))
      hRes = CoRegisterClassObject(
               *pclsid, p, dwClsContext, dwFlags, &dwRegister);
    if (p != NULL)
      p->Release();
    return hRes;


If you do a bit of spelunking in the headers, you’ll see that pfnGetClassObject is a function pointer that maps to _ClassFactoryCreatorClass::CreateInstance. So this code simply creates a new class object and registers it.


Again, this is specific to EXE servers. In DLL servers, the class object is created on demand in the DllGetClassObject function, which is called by CoGetClassObject if the COM server is a DLL.


Creating CStuff instances


If you look at the _ClassFactoryCreatorClass::CreateInstance, you’ll see that its first parameter is a void *. And the value that is passed in this parameter by the ATL module is the address of the static method for creating new object instances.


_ClassFactoryCreatorClass::CreateInstance simply creates a new class object ‘p’, and then calls p->SetVoid(pv); to pass the aforementioned function pointer to the class object.


And the SetVoid method is no more complicated then this:


  void SetVoid(void* pv)
  {
    m_pfnCreateInstance = (_ATL_CREATORFUNC*)pv;
  }
  _ATL_CREATORFUNC* m_pfnCreateInstance;


I guess the original developer hadn’t had his morning coffee yet when he came up with this name. It manages to be 100% factually correct, while being 100% opaque at the same time. But it does the job.


When the class object has to create a new instance, it simply does the following:


  STDMETHOD(CreateInstance)(LPUNKNOWN pUnkOuter, REFIID riid, void** ppvObj)
  {
    // snip irrelevant code
        hRes = m_pfnCreateInstance(pUnkOuter, riid, ppvObj);
    // snip irrelevant code
    return hRes;
  }


Providing a new class factory


Now that you understand how the class object plumbing is hooked up, it is easy to see how you can implement your own class object.


Remember how DECLARE_CLASSFACTORY_EX maps a class object to the _ClassFactoryCreatorClass typedef?


In C++, a typedef in a derived class can override a typedef in a base class. So even though the CCoClass<T> base class says that the _ClassFactoryCreatorClass maps to ATL::CComCreator< ATL::CComObjectNoLock< ATL::CComClassFactory > >,


We can override this by simply putting DECLARE_CLASSFACTORY_EX(CStuffCreator) In the declaration for CStuff. That’s really all there is to it. The macro magic takes care of the rest.


Conclusion


If changing the class object for a COM server is as trivial as adding a single line to your COM server C++ class, then why did I just write 4 pages of stuff?


I wrote all this because if I tell someone to use a macro to do something non-trivial, I want him or her to understand what is going on. There are already far too many programmers on this planet who do things without understanding what it is they do, or how it works. And I am not going to add some more by glossing over the details.


Of course, judging by the number of visitors on this corner of the internet that I call ‘home’, I stand little risk of doing so no matter how crappy I would write J


This article was originally part of my article on implementing a custom class object. But after I spent more than 3 pages on the previous explanation, I thought it would be best to put it in a separate article.


A big thank-you goes to Jim Springfield of the Architects team within the developers division (I think) of Microsoft for taking the time to review this article for correctness, and to Amit Mohindra for helping my questions and articles reach the right people.


And you can find the demo project attached to this article under the MIT license.