[DLLs] modules, static and dynamic linking

[chronozphere]

Hi everyone

I want to build a modulair system (for my engine). It should have core DLL which implements managing of resources at a very abstract level. Modules can be hooked and these can implement classes with their own properties. Other modules (or the EXE) can create objects of a class and change their properties (all through this abstract Core DLL).

Besides that, it should be possible for the game (EXE or one of the modules) to call exported functions from different modules. This is a more easy and non-abstract way of accessing module/class data.

So let's say i want to get the vertexcount from a Mesh (All identifiers are pointers).

(CODE BELOW: **this post editor is weird, pascal tags dont work here** )

I want to build this abstract core because my engine will be more modulair, and it will be easier for me to intergrate the game into the editor (when i start building an editor).

And my question is..........

What is the best way to link these DLL's together?

I don't know much about DLL linking, but yesterday i realized, that it might be more complex than i thought, when i was originaly thinking about this system.

So.. Let's think of a scenario where i have my EXE, the Core and one module.
The EXE want's to acces the module both through the abstract core, and by using the Module's exported routines. This will be a combination of Static and dynamic linking.

-- The EXE project will contain a unit with all the exported routines for both the core and the module. (Static linking).
-- The EXE will call a N3D_Module_Hook routine (for each module), to load the Module into the core.
When this is called, the core will call LoadLibrary with the DLL path. (dynamic linking).
After this, communication (MODULE <-> CORE) is possible, by using datastructures with lots
of callbacks, that are provided by the MODULE when it's entry point code is executed.


But when the Module is loaded by the core (by LoadLibrary), It has allready been staticly linked when the EXE initialized. This raises a few (more specific) questions:

-- What happens when i call Loadlibrary from the Core, when the target module allready has been staticly linked by the EXE?? Will it be copied in memory or is the same instance used?

-- When i both staticly and dynamicly load a module, is it's entry point called twice??

If so, this might cause problems. In my current design, my entry point is the place where the
module registers itsself in the Core. So i have to make sure that the core module is allready
loaded (staticly linked) before all other modules. This raises another question:

-- In what order are modules staticly linked??

I hope you all understand my little story. :) If someone knows good articles of DLL linking, please post the URL. If anyone can think of a better way of building such a "Core with lots of modules - Spider", please let me know. ;)

Thanx a lot for reading.


//--Through abstract level
N3D_Class_GetByName('Mesh', MyMeshClassIdentifer ); //gets the Mesh class identifer

N3D_Class_GetPropertyByName( MyMeshClassIdentfier , 'VertexCount', MyPropertyIdentifier );
//Gets the property identfier from the class

//Gets the VertexCount property from an Object which should be a Mesh
//And stores it at the address of VertexCount
N3D_Object_GetProperty( MyObjectIdentifier, MyPropertyIdentfier, @VertexCount);


//--Direct call to an exported function
N3D_Mesh_GetVertexCount( MyMeshIdentifier, Vertexcount );
//Accepts vertexcount as VAR parameter

[Mirage]

Why use dll? Modular system doesn't mean use of dll. Use of units will be enough dor modularity.
Each dll have separate memory manager. And there will be problems. For example, if you allocate something (even a string) in one dll you can't free it within another. So there should be a serious reason to put an engine into dlls. I can't see any.

To integrate my world editor with my engine I used Delphi packages (bpl). Because a typical game project will introduce its own game-specific classes, which the editor should be able to create, edit properties of, etc, and I don't want to have an editor for each project.
A package is similar to dll but share common memory manager and contains units. When you load a package initialization sections of units contained in the package will execute.

But it's only in editor. In a game project is no need in packages or dlls.

[NecroDOME]

If you init the core dll as static in your .exe, you can just call all other (module) dll's trough the core dynamically. If you also link your module dll static at your .exe AND load them dynamically, I don't know what will happen then...

What I do know is when you include Forms.pas in both dll and you .exe, you can access Application.Anything. I don't know if this is shared, or 2 separated classes, but it works when you call Application.Terminate.

[Setharian]

Dynamicly loading a module (by LoadLibrary) if the module is already loaded in a process' address space does nothing more than incrementing module's internal reference count. If the module got loaded because it has been staticly linked with the exe, the reference count is -1 and LoadLibrary doesn't do anything other than returning a module handle.

As for the "Application.xxx" thing, there will be 2 application objects created if Forms.pas is compiled into both of the modules (this holds true for any unit with global data). If you however link with a runtime package (vcl.bpl), only 1 instance will be shared by all the modules which link with the package.

Next the original question. Entry point of the dll should be called when a process attaches to it so it should be called just once. Use LoadLibrary to load libraries which might not be available at runtime and are not crucial for the application to run properly (like plugins) or you have an alternative implementation for such case.

And finally to the code in your post - I think you are overcomplicating things. You are probably using RTTI to dynamicly get/set the values which is slow as hell. Besides such flexibility is of no use here because you do not store any dynamic data. A mesh must always have vertices otherwise it wouldn't be a mesh, right? I believe this can be applied to many object types used in a game engine. If you want efficient inter-module communication you have 3 possibilities. Use dlls + structures (records) in the good old C-way (like your "N3D_Mesh_GetVertexCount") - you cannot use Delphi's built-in types like strings or dynamic arrays nor its register calling convention if it is to be interoperable with other languages like C/C++/C#/etc. so you have to marshal data around. Then you can use delphi's packages which allow you to work like as if units from the package are staticly linked into the exe but they are in a seperate special dll - you can safely use Delphi's built-in types since packages can only be used from Delphi or C++ Builder. And finally you can use dlls + interfaces - you define an interface through which the host will communicate with the dll. For example you define an interface "IMesh" and a class "TN3DMesh = class(TInterfacedObject, IMesh)" and let the dll export a function "CreateMesh(Vertices: PVertexArray; VertexCount: Integer): IMesh" - the host calls the exported function from the dll, obtains an IMesh interface which it can then use to access data and call methods.

I hope this helps. :)

[chronozphere]

Thanx for all your usefull replies :)



Why use dll? Modular system doesn't mean use of dll. Use of units will be enough dor modularity.


I know.. I dont have a good reason to use DLL's. The only thing i can think of is C++ compatibility.
So when i start learning C++ i don't have to throw away my entire game-engine. I can still use the module's which i made in pascal. Besides that, it's good to learn how to work with DLL's, because they are used alot.



Each dll have separate memory manager. And there will be problems. For example, if you allocate something (even a string) in one dll you can't free it within another. So there should be a serious reason to put an engine into dlls. I can't see any.


I know that :) It's always good practice to see what happens with the allocated mem in a DLL scenario. In general, i let every module maintain it's own data. When i want other modules to read that data, i send a pointer to it. When i want modules to write data to others, i usualy let the destination module read from the source module. When it comes to destroying data, i call an exported routine which destroy's it properly. :)

But eh?? does this also apply to pChar's?? :?



And finally to the code in your post - I think you are overcomplicating things. You are probably using RTTI to dynamicly get/set the values which is slow as hell.


No, i will not be using RTTI. Registering Classes + Properties into the Core DLL is done by sending a "Class/Property description Record" to the engine. This record contains a Getter and Setter callback for every property. There is more info present in this record, like e.g the property type and size (when the type is custom) and the readonly/writeonly flags. Pointers are used by the callbacks to read/write data.

This, i think is a very good system. In each module you can write getters and setters for each property. You could also make a property like "Initialize" which is a boolean. When you send true to the initialize property, the object in question will be initialized. You can use false to terminate the object. When you want to get the value of "Initialize", you will get a boolean, which indicates whether the object is initialized or not. Isn't that a nice system??? :razz:

[JSoftware]

I'll second the idea of using interfaces. This will also allow you to use the dll in another language without having to make alot of hoops.

[chronozphere]

I've read some articles about interfaces, and it was quite compicated stuff :(

Is there a good article which explains interfaces in the DLL context???