Now where did I put my type?

Today I ran across an interesting problem while trying to export a function from a dll built in vc++ (2003). I’ve been digging into the mozilla source a lot lately, doing the library to embed mozilla and get our much-needed webcontrol working on Mono, and after a successfull browser window invocation from .net (screenies to come soon) (woohoo), I decided to clean up the code a bit and start doing the functions properly.

The first functions of the library were only returning ints, but for the glue to work properly, I needed to return a pointer to the newly created class in C++, and since cross-platform is good, all the function exports got marked with mozilla’s NS_EXPORT_(type). While I had int as a return type, all was well, but when I decided to return void*… boom!

Ooops!

Looking at the exported functions of the dll, all those marked with void* or similar (anything with *, really) were not there. Everything was happily compiled… they just weren’t exported. :p

Cue 2 hours of googling and mucking about with the export syntax and going absolutely nowhere.

Now note, please, exhibit A - the NS_EXPORT_ typedef:

#define NS_EXPORT_(type) type __declspec(dllexport) __stdcall

Note, also, as exhibit B, an example signature that jchambers gave to me on the #mono channel:

__declspec(dllexport) void* STDCALL myfunc();

Notice anything?

Well, apparently, if you put the type before the __declspec, all functions with pointer return types will not be exported. If you put the type between the declspec and the stdcall, they are exported.

.<

I ended up doing this:

#ifdef VSTUDIO
#define NS_EXPORT_(type) __declspec(dllexport) type __stdcall
#endif

sigh

Of course, I could be doing something really wrong here… if anyone can set me right, feel free! I’ll just go back to doing some useful now :)