Threads..

[M109uk]

Hi all,

Is it possible to get my opengl engine to 'render' in a thread?

I have tried using BeginThread(), this seems to work except all i get in the opengl context is a black screen.

Is there something else i need to do?

I can't put my entire engine in to a thread as there are several events going to and from my application.

Thanks

[Nitrogen]

Threading is an immense headache.

The latest version of my Font Studio uses a render thread to do the pixel manipulation and I had endless problems getting the thing to work properly even with copying all necessary data into the thread at creation and copying it back to the main program after processing.

I would recommend this text: http://www.eonclash.com/Tutorials/Multithreading/MartinHarvey1.1/ToC.html

It was quite a help to me while writing my threaded program.

All I can say to you is: brush up on your critical sections and mutexes!

[Chebmaster]

As far as I know, OpenGL is *not* thread-safe. It's a purely single-threaded system. So no, you cannot render in another thread. You can do math, physics, background loading, though.

[jdarling]

You can render in another thread, problem is you can't render in two threads at the same time :).

Take a look at the GLScene source code and try posting this in the GLScene NG and you will get LOTS of replies on how it can be achieved. Lots of smart people working on that project and Eric is always more then willing to help others out.

[JernejL]

As far as I know, OpenGL is *not* thread-safe. It's a purely single-threaded system. So no, you cannot render in another thread. You can do math, physics, background loading, though.

Ugh.. your logic is flawed, opengl is useful within any thread as long as you execute opengl calls only one thread.

[M109uk]

Thanks for all the replys :)

I will have a look at GLScene..

I've managed to get threading working on one example, which is a barebone applications with no forms and an SDL window is created, and then everything is rendered in a thread via a repeat..until,

However, with the same code, except i have a main form, this creates everything in the OnCreate, and this is now where i get a black screen :s...

Im just not sure if windows is being temperamental lol...

Thanks again

[Chebmaster]

your logic is flawed, opengl is useful within any thread as long as you execute opengl calls only one thread.
And...? As far as I know, you get a free crash fest if your window is created not in the main application thread. Same goes for trying to create OpenGL rendering context, and same goes for any DirectX operations.

At least it was such way when I tried to explore the threading aspect of 3d graphics. It was in the times of Windows 98 and Riva TNT, though. Maybe things improved a bit since then.

[Setharian]

OGL and D3D are single-threaded, but it doesn't matter in which thread you use it that is it doesn't have to be the main process thread...

[JSoftware]

Why would you want to go through the hassle of writing a threaded rendering module?

[M109uk]

Its mainly for the use in my editors,

i want to use my engine code so i have all the effects on hand, but it renders in a repeat loop, which renders, ticks and all,
but with that some of the applications messages are not getting through, for example if i wanted to rename an object it would take 1 minute to type the name, sometimes missing out characters. Sometimes it even prevents me from closing Dialogs and such, i could'nt close a simple InputBox once and hard to Ctrl-alt-del it.

I tried adding Application.ProcessMessages in several areas of the rendering, which made a few little differences but still having problems.

If that makes any sense :?

[JSoftware]

Can't you just use OnIdle? I can't imagine that would be that hard. I would under no circumstance(unless paid) try to implement threadbased rendering :P

[Srki_82]

I don't know is this what you want... I wrote this simple program in Lazarus but you could port it to Delphi very easy (only Windows and GL functions are used).

http://www.geocities.com/srki_82/GLThread.zip

[Almindor]

Threading and openGL are big enemies. This is known and future GL versions promise to fix it by changing the whole API from "set_state(x); do_something;" to "set_state(x); do_something(x);"

Right now, threading and openGL results in slowdowns more than anything else. You need to add a hell load of locks to ensure proper functionality and even on multicores I doubt it'd be faster than single-threaded one.

[M109uk]

Thanks for all the replys :)

i have came across sooo many annoying bugs and problems so i have decided not to bother and just use the OnIdle event as suggested and all seems to work fine, a little slow sometimes, and an annoying problem with loosing the contents of the GL rendering context everytime i resize :( but i think thats an SDL issue.

again thanks for the replys.

[Chebmaster]

Create your window manually and you'll never lose the context. Still, you can lose some "less significant" pieces, like z-buffer, when you switch from 16-bit to 32-bit video mode.

I wonder if the Doom3 way (require restart each time user changes screen resolution) is the only absolutely safe one.

On the other hand, maintaining your window manually is *such a pain*...

[M109uk]

Yeah i looked into creating the windows manually, however i had several problems with initializing the OpenGL extensions, for some reason it never let me setup multi-sampling or any other extension. I tried looking at several examples, and used 3 different set of headers.

Also although the game is being written for windows, i do want the option to be able to have it crossplatform compatible, and from my understanding SDL and OpenGL are probably my best option.

Its not so much of a problem with restarting the game, it was more of an issue with in my editors, because it takes between 1-5 minutes to load all the data on start-up, and this will be a huge pain if you have to wait for soo long everytime the window is resized. Its kinda odd, i heard that this was a problem only with windows, but i dont recall having this problem before i started this project :s

[Chebmaster]

Well, you can use SDL for the fullscreen mode and the custom-created window for the windowed mode.

[JSoftware]

Yeah i looked into creating the windows manually, however i had several problems with initializing the OpenGL extensions, for some reason it never let me setup multi-sampling or any other extension. I tried looking at several examples, and used 3 different set of headers.

You did remember that you need to allocate a temporary opengl context to get the addresses of the extension procedures used with multisample initialization BEFORE you set your main pixelformatdescriptor?

[M109uk]

Well, you can use SDL for the fullscreen mode and the custom-created window for the windowed mode.


I had though of having 2 Engine classes, 1 SDL and the other manual, how ever the problem is that i need the extensions in most of the editors, although the scene/world editor is now being built in to the game itself as a live (local/server) editor, and this is'nt as much of a problem. but in the end i though i'd just turn off resizing of the SDL window, until i can figure the problems out.



You did remember that you need to allocate a temporary opengl context to get the addresses of the extension procedures used with multisample initialization BEFORE you set your main pixelformatdescriptor?

Yeah, thats the strange thing, i initialized opengl then created the window and context, then initialized the extensions and then recreated the window, sometimes the extensions would load and sometimes they would but when i came to use them i kept getting access violations, i double checked my entire code for anywhere else that was initializing the opengl and/or extensions but there was none.

I then tried to download the delphi translation of NeHes example, i compiled it using the original headers which failed, i then changed the headers to dglOpenGL which also did the above, and i then tried the the Gl, Glu headers and this just failed before the i even got to initialize the extensions.

I have the latest OpenGL drivers, although they are still beta vista drivers.. but that cant be the problem otherwise it would'nt work in SDL :s

[Chebmaster]

and then recreated the window,
Maybe this is the cause. When you create OpenGL rendering context, you create it for a given device context. The device context, in turn, belongs to your window.

So, you see, if you re-create the window your rendering context becomes invalid!

Again, AFAIK, any extensions you load are valid *only* for the given rendering context. If you re-create the context, you must reload all extensions.



...Well, that was for Windows. In Linux it's even more painful, as user clicking on your window's "close" button leads to your program's instant death, and you should use some elaborate hack-arounds to allow to at least auto-save graciously..

[M109uk]

It is possible, i tryed build it in to a class.. i more than likly missed something out in my class, specially when i tried to get multisampling working, since then my oringally engine didnt work :s

Although it does'nt explain why the NeHe examples did'nt work when i compiled them with 3 different headers, in fact i recall the executable that was included did'nt work either :?

Yeah, i had a quick look at the code required to create a context in linux and safe to say it was giving me a headache.. i know SDL supports many platforms, but does it require much alteration in my engine to support other platforms?

[Chebmaster]

I didn't look into SDL myself yet (I am happy with my moster of a window manages as it is... Although it took me years to build and fine-tune it) but I hope that SDL's main purpose is to protect you from these bloody details.

Still, recheck your OpenGL calls flow. If it's wrapped in a class, attach to all your code block a short WriteLn() to console or log file and see if the blocks order is correct. May seem a bit tedious, but this approach could save you many hours worth of debugging.

Btw., my program is peppered with constructs like
if VerboseLog then AddLog('Some witty comment' + SomeParameter);
, where VerboseLog is a global boolean var. You may think that debugging allows you the same and more, but my method paints a complete picture of what happened after what. Very useful.
I'd recommend using something like this in your OpenGl initizlizing code. So in case one of your users with different driver/videocard runs into problems, all he need is to turn verboselog on and send you the log file.

[M109uk]

It seems to do all the window creating itself, well at least on windows :D i havnt tested it on linux yet, since my linux box is in several pieces on the floor :s

I am hoping it deals with these things, it will save myself from a lot extra debuging, and headaches :roll:

What i have done is all the systems within the engine are in seperate units with their own initialization code, so for example the particle systems is in XParticleSystems.pas, with a TList class that holds all the particle systems, this is created and destroyed in the initialization and finalization of the unit, i have placed a log event in every unit that does this, how ever for seem reason the units get initialized at different times not matter where i put the uses clause in the dpr, for example in both the projects i have:



uses
XParticleSystems,
XLightmaps,
....
XModels;


how ever in the log, in 1 of the projects it would say that the models class was created first, but in another project it would say the lightmaps was created first, for this reason i spent 2 weeks debuging, due to several access violations, because for example the models unit uses a class that gets created in the lightmaps unit.

I found this same problem was why one of my attempts of creating the window manually failed, because it was creating the window before initializing OpenGL.

My engine seems to be riddled with little bugs that jump out of no where :s
last week i rewrote my engine, so that its no longer built within a library, for some reason i had a for loop within the render procedure that always violated the array range, how ever i never got an error about this until a month later when i started on the particle system..

I will take your advice about the verbose log, thanks :) it would help in the long run.

[Chebmaster]

Also, there are two more tricks:

A.) Take the C way, break your units to header and body parts, and {$include xxx} them into a single unit. IMHO, this approach is better for rthe stuff with many inter-dependencies than the standard Pascal units way. Minuses are you have a heap of files and you need to watch the order they included and take care of forward declarations.

B.) Get rid of initialization..finalization blocks in favor of manually called InitXXX() procedures. This way you will always be in control.

The more I see tha more I believe that a Pascal unit must be a bigger thing that people usually make it. It should be a self-sufficient "black box" with no "extra" dependencies, unaffected by initialization order. This may be as small as 50 lines of code or as big as 9000 (my personal record for single unit - it consists of dozens include files).

If your units do not match these criteria, merge them.

[M109uk]

I have a few of the bigger units such as the lightmapping, split in to several files and included into 1 unit, i really only do this if the unit is getting to big to keep track of.. i normally prefer not to do this with the smaller units because i just find it a pain to keep track of everything, and delphis code completion does'nt work with the seperate file either (me just being lazy :p)

I intend to get rid of the initialization blocks at some point soon, specialy now that there is soo many :s (really is a bad habit of mine)

Sadly my entire game & engine does'nt have a single self-sufficient unit (well maybe 2), i have built my engine in a way that i dont need to worry about creating or render *any* part of the engine, all i need to do in the game is create then engine class, initialize it and then load the required data, the engine takes care of everything else, the down side is that when i create an editor i have to include every single file in the game, im sure it is probably bad practice on my part :oops: