Dynamically Bound Modules

Taquis - PC Base FFT Spectrum Analyzers, Oscilloscopes, Data Analysis, Data Acquisition, and Application Frameworks

AtOne Application Framework
"A Framework working with you, not against you..."

Home
Features
Programming
License
Downloads
Directions

Dynamically Bound Modules

Have you ever wished to be able to code dynamically bound WIN32 DLL's in C++ without having to code a C wrapper for all of your classes? Have you ever used COM to code C++ DLL's and were dissappointed by the way that COM binds you to the operating system through the registry, complicates your program installation, and looks likes a poor immitation of C++ because of the way in which objects are created? Are you dissapointed by the complex way in which COM handles interface versioning? If you answered yes to any of these questions then this software technology may have the answer you require.

This technology arose out of the need to put common C++ code into a DLL and make it useable in an application without freezing the interface of the DLL, and without have to deal with the non-object oriented model of COM interfaces and the added burden of binding to the Windows registry. The resulting technology is leen and flexible, and like COM, relies on a standard v-table implementation of C++ classes (if cross compiler binary compatibility is an issue to you).

Definitions

A dynamically bound module, henceforth referred to as a module, is a collection of one or more module interfaces contained in a dynamic link library and/or executable. A module interface is merely a collection of related API that can be invoked through a bound context. The collection takes on the form of an abstract C++ base class which defines only pure virtual methods and no other attributes. The context is an object instance or the this pointer.

Reasoning

A module interface does not contain any data attributes to avoid creating dependencies between the client application and the module service provider. The only dependency between client and service provider is the interface methods themselves which must be fixed as a concrete specification once the module interface is published. The only exception to this rule is through the publishing of a revised version through the provided module interface versioning mechanisms. In other words, the introduction of a revision requires that the revised interface be registered with a different version number. Continued implementation of superceded versions is not required although by not providing them, old applications may no longer run correctly. All interface methods must be implemented using a common calling convention for cross compiler binary compatibility to be maintained.

Rules

Interface definitions should be pure abstract class definitions without any data attributes, constructors or destructors. An interface should only contain pure virtual methods of which some typically perform object initialisation in ways similar to non-default class constructors. Once defined and published an interface definition should not be changed. The only exception is in publishing a revised interface with a new version number. Note that changing the order of methods in a published interface is not allowed as it will destroy the late binding mechanism, because the v-table order will have changed. All implementations of interface methods must used the "modcall" calling convention (This evaluates to stdcall on WIN32). Due to an inconsistency in Microsoft Compilers, when constructing v-tables it is necessary to provide two interface declarations if any of the methods that the interface defines are overloaded (have the same name but different argument list). One interface defines the methods in normal order whilst the other defines the same interface but with the overloaded methods in reverse order. These two cases should be conditionally compiled based on the definition of the MS_VTABLE symbol. In other words,

#ifdef MS_VTABLE
class MdlMyClass
{
public:
    //reversed order of overloaded methods
    virtual void modcall initialise(int A, int B) = 0;
    virtual void modcall initialise(double C) = 0;
    virtual bool modcall process() = 0;
};
#else
class MdlMyClass
{
public:
    //normal order of overloaded methods
    virtual void modcall initialise(double C) = 0;
    virtual void modcall initialise(int A, int B) = 0;
    virtual bool modcall process() = 0;
};
#endif

As an explanation, the MS_VTABLE is defined when compiling under Microsoft Compilers and is used to differentiate between Micrsoft v-table generation and other compiler vendors. Typically compilers generate v-table entries in the order in which virtual functions are declared. This is basically true for Microsoft Compilers except in the case of overloaded member functions. For overloaded members the Microsoft Compiler reverses the order of the overloaded members which screws up compatability between compiler platforms. However, this cross compiler incompatibility issue may be worked around by using the following strategy. When MS_VTABLE is defined the overloaded members should appear in reverse order and should always be grouped together. This should ensure binary compatability between Microsoft Compilers and there competitors (which unfortunately seem to be dying out). It is curious to note that all the COM interfaces that Microsoft have ever published (at least the ones that I have seen) never use overloading. Instead they always use paired Get and Set methods for each property. Perhaps this archaic coding convention that is poorly suited to the world of C++ programming is a result of this compiler incompatiblity issue.

How do I build Modules and Module Interfaces?

A module is nothing more that a collection of module interfaces that are implemented within a DLL (or executable). Module interfaces are, in turn, nothing more than classes that contain only pure virtual methods with no attributes (data members) and no constructors and destructors. In essence it is just a v-table. The only other requirement is that module interfaces declare a minimum set of standard methods that are defined in the class MdlInterface.

class MdlInterface
{
public:
virtual void modcall              lock() const = 0;
virtual int modcall              unlock() const = 0;
virtual void modcall              destroy() = 0;
virtual const string modcall      className() const = 0;
virtual int modcall               classVersion() const = 0;
virtual bool modcall              hasClassNameAndVersion(const string pClassName, int nVersion) const;
};

The first two standard methods, lock() and unlock(), are required to implement object reference counting for module interface instances. It should be noted that, although these methods must be implemented, reference counting of objects is optional in there implementation. AtOne provides a range of macros to help in coding these and other standard methods, both with and without reference counting. The destroy() method is equivalent to calling the destructor of the class using the this pointer as the context. Note that we need an explicit destructor because all references to a given module interface must be late bound. The last three methods, className(), classVersion() and hasClassNameAndVersion(), are class / interface identity methods that can be use to identify an interface instance and whether it can be safely cast into another class of object. It is customary for class name to return the string name of the interface : for example, in this case an implementation to this interface would return _T("MdlInterface"). The classVersion() returns a number corresponding to the version nuber of the interface implementation and customarily starts at one. Thus if I were to propose a new extended MdlInterface I should bump the version number to 2 to avoid breaking existing code that relied upon the old version.

The first thing that you need to decide upon is the methods that your interface will contain and the name of that interface. It is customary to give the interface name the prefix Mdl to signify the fact that it is a module interface. Lets say, for argument sake, that we want to create an interface that encapsulates a window function to be applied to a time series in the process of spectrum analysis. This corresponds to one of the many interfaces that AtSpec Spectrum Analyzer defines in the ATSDSP.DLL dynamic link library. The interface is defined as,

class MdlWindowFunction : public MdlInterface
{
public:
virtual bool   modcall                  initialise(int nOrder) = 0;
virtual const MdlFixed32* modcall       data() const = 0;
virtual int    modcall                  size() const = 0;
virtual int    modcall                  order() const = 0;
virtual double modcall                  mean() const = 0;
virtual double modcall                  meanSquare() const = 0;
virtual bool   modcall                  valid(int nOrder) const = 0;
virtual MdlWindowFunctionType modcall   type() const = 0;
virtual const string modcall            windowName() const = 0;

declInterfaceIdentity;
};

Firstly note that MdlWindowFunction inherits from MdlInterface. Secondly note that this interface defines an initialise() method. Remember that interfaces must be created through the default constructor since we cannot pass parameters to the DLL that implements the interface. We get around this limitation by always creating the object instance through the default constructor and then providing an initialisation method to serve the purpose of a non-default constructor initialisation. Finally note that the standard methods have also been declared as implemented through the use of the macro declInterfaceIdentity. At first this may seem to violate the rules of interfaces (ie. they must be pure abstract) but this need be the case. All that is required is that all methods of an interface must be late bound when called through the interface passed from a DLL to an executable. Given that an executable calling into the interface only knows it through the definition of MdlWindowInterface this is guaranteed because the class is pure abstract (an instance of it can never be created). Instead MdlWindowFunction is always created through the implementation of a derived module interface such as,

class MdlBartlettWindow : public MdlWindowFunction
{
public:
virtual bool modcall initialise(int nOrder) = 0;
};

A concrete implementation of all instantiable interfaces are coded and packaged into a DLL. For instance, the MdlBartlettWindow interface is implemented in ATSDSP.DLL through,

class OsiWindowFunction : protected OsiReferenceCount
{
protected:
MdlFixed32*   Data;
int           Size;
int           Order;
double        Mean;
double        MeanSquare;

protected:
bool createBuffer(int nOrder);

public:
OsiWindowFunction();
virtual ~OsiWindowFunction();
};

and,

class OsiBartlettWindow : public MdlBartlettWindow, protected OsiWindowFunction
{
public:
OsiBartlettWindow();
OsiBartlettWindow(int nOrder);
virtual ~OsiBartlettWindow();

virtual bool   modcall                  initialise(int nOrder);
virtual const MdlFixed32* modcall      data() const;
virtual int    modcall                  size() const;
virtual double modcall                  mean() const;
virtual double modcall                  meanSquare() const;
virtual int    modcall                  order() const;
virtual bool   modcall                  valid(int nOrder) const;
virtual MdlWindowFunctionType modcall   type() const;
virtual const string modcall            windowName() const;

declInterfaceStandardMethods;
};

The declInterfaceStandardMethods macro declares the standard methods common to all MdlInterfaces. Likewise the implementations for the standard methods are implemented with either implInterfaceStandardMethodsWithLocking(CTYPE, PCTYPE, NAME, VERSION) or implInterfaceStandardMethodsWithoutLocking(CTYPE, PCTYPE, NAME, VERSION) depending upon whether the interface uses reference counting. In this case reference counting is used and is implemented with,

implInterfaceStandardMethodsWithLocking(OsiBartlettWindow, MdlBartlettWindow, _T("MdlBartlettWindow"), 1);

Refer to module.hpp for definitions for these macros. The only other consideration to keep in mind is the coding of constructors and destructors. As a interface instance is generally bound to a DLL, the DLL life-cycle is in turn bound to the interface life-cycle, or at least needs to be. This is easily achieved by ensuring that each constructor calls the LockModule() macro and each destructor calls the UnlockModule() macro as in,

OsiBartlettWindow::OsiBartlettWindow()
: MdlBartlettWindow(),
OsiWindowFunction()
{
LockModule();
}

OsiBartlettWindow::OsiBartlettWindow(int nOrder)
: MdlBartlettWindow(),
OsiWindowFunction()
{
LockModule();
initialise(nOrder);
}

OsiBartlettWindow::~OsiBartlettWindow()
{
UnlockModule();
}

One question remains. How does the DLL know how to create a particular interface? This will become clearer in the following section but each module has a class registry which is responsible for creating interface instances and passing them back to the client. Therefore any interface implementation must be registered with the registry before we can use it. This is easily achieved through the macro registerModuleClass(). In this particular example we use,

registerModuleClass(MdlBartlettWindow, 1, OsiBartlettWindow);

to register the implementation to the interface. Then its simply a matter of linking the object code into a DLL which also links in MODULE.LIB, COMMON.LIB and CONTAIN.LIB AtOne libraries.

How do I use an Interface in a Module?

Given our newly created module how do we make use of it? To access the module we use an instance of the OsiModule class. The constructor of this class will load the specified module (DLL). You can then use this instance to create instances of any interfaces that are defined in that module by calling the create() method. This will require the casting of an MdlInterface pointer into the appropriate type which can be regarded as error prone. Therefore, to reduce the possibility of error and improve code readability it is customary to define a macro to do the creation and type casting for each interface defined in a given module. For example, the ATSDSP.DLL module defines macro,

#define CreateMdlBartlettWindow(Module) ((MdlBartlettWindow*)Module.create(_T("MdlBartlettWindow"), 1))

in dspmdl.hpp as an aid to creating instances of the MdlBartlettWindow interface. It is also customary to define a corresponding interface identity macro to check the type of an interface, such as,

#define IsMdlBartlettWindow(pObj) ((pObj != 0) && pObj->hasClassNameAndVersion(_T("MdlBartlettWindow"), 1))

The interface declarations along with these macros are all typically all defined in the one header file that is supplied with the module to the developer of a client application, allowing the developer to use the module interfaces. In the case of ATSDSP.DLL these interfaces are all defined in dspmdl.hpp which is included into the files within the AtSpec source that make use of these interfaces. So, for example, if we wanted to create and use an instance of the MdlBartlettWindow interface in a CPP file we would do the following.

//include relevant files.
#include module.hpp
#include dspmdl.hpp
.
.
.

// Create a instance of the module. This need not be global
// but can also be local and you can have multiple instances
// referencing the same module. For the module to be useable
// it must be present in the specified path.
OsiModule DspModule(_T("ATSDSP.DLL"));
.
.
.

SomeClass::someMethod(...)
{
.
.
.
//Create an instance of the MdlBartlettWindow interface.
MdlBartlettWindow* pBartlettWindow = CreateMdlBartlettWindow(DspModule);

if (pBartlettWindow != 0)
{
//We must initialise the interface before using it.
pBartlettWindow->initialise(10);

    //Call methods on the interface as we please.
    .
    .
    .

//We must release the interface when we no longer need it.
MdlReleaseReference(pBartlettWindow);
}
}

It should be clear that this method of dealing with interfaces is more typical of object oriented C++ programming than COM. It is fealt that this approach gives the developer greater flexibility than COM in its usage, and more importantly, provides a framework that is naturally more object oriented than the COM approach of querying interfaces.

Modules In Exectuables

Unlike COM, modules make no attempt to provide a mechanism to share objects between processes. There is no support for creating an interface in an out of process module. Modules are always DLL's. However, client applications that make use of modules can declare and register their own module interfaces which a loaded module can create and use. This unique approach provides a two way communication between client application and server module. Say, for argument sake that the MdlBarlettWindow interface was implemented in our application ATSPEC.EXE and that this application loaded our module ATSDSP.DLL. By referencing the parent module (ie. the module that loaded this module), ATSDSP.DLL could create an instance of MdlBartlettWindow in ATSPEC.EXE. To do so all we do is substitute OsiParentModule for OsiModule giving,

//include relevant files.
#include module.hpp
#include dspmdl.hpp
.
.
.

// Create a instance of the parent module. This need not be global
// but can also be local and you can have multiple instances thereof.
// For parent module to be valid this code must reside inside a module
// (DLL) that has been loaded by an application or another module.
// The parent module is the module which loaded this module.
OsiParentModule ParentModule;
.
.
.

SomeClass::someMethod(...)
{
.
.
.
//Create an instance of the MdlBartlettWindow interface.
MdlBartlettWindow* pBartlettWindow = CreateMdlBartlettWindow(ParentModule);
.
.
.

Typically a better use for this feature is in providing a dynamically bound means of notification between module and executable, similar to control events in a technology such as ActiveX but with much greater flexibility and ease of use.

Driver Development

One aspect of program development for which this type of technology is well suited is in driver developement. Drivers invariably must be late bound and also must be able to be modified over time without breaking compatibility. Both these mechanisms are naturally supported in AtOne modules. COM is also suitable for driver development but is cumbersome to use for this task due to the nature of drivers. In essence a driver interface is a fixed interface that defines the common API used to control and interact with a particular device. All drivers must share the same interface but differ in what they do.

For example, AtSpec Spectrum Analyzer provides a driver interface for the retrieval of data from an abstract data soure that will be analyzed by AtSpec. Drivers are available to read data from a sound card, a wave file and a text file and all have the same interface. If you were to code this in COM you would need to create multiple "class" entries, one for each driver, despite the fact that they all share the same interface. You would also need to manage the process of maintaining and querying a driver database yourself. In AtOne modules direct support for drivers is given making the process of driver development and usage that much easier.

When developing a driver using AtOne modules all we need to do differently is the way in which we register the interface implementation with the module interface registry. Normally interface implementations registered with the registry have a one to one correspondence to the interface definition : that is, one interface implementation to one interface definition. In the case of a driver however, we have many interface implementations to one interface definition. In this case rather than registering with the registerModuleClass(CNAME, CVERSION, CIMPL) macro we instead use the registerModuleClassWithVersionName(CNAME, CVERSION, CIMPL, VNAME) macro. This differs from the other case in the fact that we also specify a version name (typically the device name of the driver - sound card, text file, etc). For example, the drivers embedded in ATSDSP.DLL for AtSpec were registered with,

registerModuleClassWithVersionName(MdlAtSpecInputDeviceDriver,
                                   1,
                                   DrvAtSpecSoundBlasterDeviceDriver,
                                   DrvName_AtSpecSoundBlasterDriver);

registerModuleClassWithVersionName(MdlAtSpecInputDeviceDriver,
                                   1,
                                   DrvAtSpecImpedanceDeviceDriver,
                                   DrvName_AtSpecInputImpedanceDriver);

registerModuleClassWithVersionName(MdlAtSpecInputDeviceDriver,
                                   1,
                                   DrvAtSpecWaveFileDeviceDriver,
                                   DrvName_AtSpecWaveFileDriver);

registerModuleClassWithVersionName(MdlAtSpecInputDeviceDriver,
                                   1,
                                   DrvAtSpecTextFileDeviceDriver,
                                   DrvName_AtSpecTextFileDriver);

given this module four different implementations to the same interface.

Given a module with drivers, how do we use them? It is more than likely that when a module containing drivers is loaded for the first time the client application will not know the names of the drivers available. Typically the application will want to know this so that the user can select the driver that they wish to use. For this very reason each AtOne module provides a standard interface, MdlClassRegistry, to query the contents of the interface registery and is defined as,

typedef bool (modcall *MdlForAllClassesDoCallback)(const MdlClass* pClass, void* pData);

class MdlClassRegistry : public MdlInterface
{
public:
virtual MdlInterface* modcall create(const string sClassName, int nClassVersion) const = 0;
virtual const MdlClass* modcall forAllClassesDo(const string sClassName, MdlForAllClassesDoCallback pCallback, void* pData) const = 0;
virtual const MdlClass* modcall forAllClassesDo(MdlForAllClassesDoCallback pCallback, void* pData) const = 0;
};

So to find out which classes/interfaces are present in a given module create an instance of the MdlClassRegistry interface and call one of the forAllClassesDo() methods. The first version taking the sClassName argument is used to enumerate available drivers for a given driver interface or class. The second version is used to enumerate all instantiable interfaces (excluding drivers). For example, AtSpec queries the available drivers in the method,

bool OsiAtSpecDriverManager::addDriver(const string pClassName, const string pModuleNameAndPath)
{
bool bLoaded = false;

if (pModuleNameAndPath != 0)
{
OsiModule Module(pModuleNameAndPath);

    if (Module.isValid())
    {
      MdlClassRegistry* pRegistry = CreateMdlClassRegistry(Module);

      if (pRegistry != 0)
      {
        LastLoadedCount       = 0;
        LastLoadedPath        = pModuleNameAndPath;
        DriverClassNameToAdd = pClassName;

pRegistry->forAllClassesDo(pClassName, addDriverClassCallback, (void*)this);

        bLoaded = (LastLoadedCount != 0);
      }
    }
}

return (bLoaded);
}

The callback addDriverClassCallback() is passed a pointer to an MdlClass interface instance describing the class. This interface is defined as,

class MdlClass
{
public:
virtual const string modcall        className() const = 0;
virtual int modcall                 classVersion() const = 0;
virtual const string modcall        versionName() const = 0;
virtual bool modcall                isClass(const string sClassName, int nClassVersion) const = 0;
virtual bool modcall                hasVersionName(const string sVersionName) const = 0;
virtual MdlInterface* modcall       create() const = 0;
};

You can create an instance of the given driver by calling the create() method in this interface and then casting the pointer to the appropriate class. These inbuilt features provide a flexible means of supporting drivers in C++ without the problems typically associated with C++ in DLL's. By building this technology seperate from COM your software benefits from not being tied in to the Windows registery and the possible side effects associated with its use.

AtOne Libraries

"We use Zeus for Windows and Watcom C/C++ 11.0 as our development environment of choice..."

Paavo Jumppanen
Creator of AtOne Application Framework