• Recent Tutorials

  • Tripping The Class Fantastic: Multi-threading Part 1

    Supending and Resuming Threads

    You will by now have noticed the boolean parameter being passed to the threads 'create' method. This is the 'createSuspended' flag. It tells the operating system the initial state of the thread once it has been created. It can either be running (createSuspended=false) or suspended (createSuspended=true).

    Why is this relevant?

    Well, consider this simple thread class.

    Code:
    interface
    
    uses
      classes;
    
    type
      TAVThread = class(TThread)
      protected
        fPInt : ^integer;
      public
        constructor create;
        procedure execute;
      end;
    
    implementation
    
    constructor TAVThread.create;
    begin
         inherited create(false);
    
         sleep(100);
    
         new(fPInt);
    end;
    
    destructor TAVThread.destroy;
    begin
         dispose(fPInt);
    
         inherited;
    end;
    
    procedure TAVThread.execute;
    begin
         while not self.terminated do
               fPInt^:=fPInt^+1;
    end;
    If you create an instance of this thread, you will notice that you get an access violation in the 'execute' method. This is because the thread wasn't created and placed in its suspended state. When the thread that called the constructor encounters the sleep(100), it yields to the OS which allocates time to the newly created thread. As a consequence, the 'execute' method is called. Obviously, when that happens, it will try to operate on 'fPInt' which is still waiting to be initialised by the constructor. Net result... *boom*

    This is where 'createSuspended' comes in.

    Code:
    interface
    
    uses
      classes;
    
    type
      TNoAVThread = class(TThread)
      protected
        fPInt : ^integer;
      public
        constructor create;
        procedure execute;
      end;
    
    implementation
    
    constructor TNoAVThread.create;
    begin
         inherited create(true);
    
         sleep(100);
    
         new(fPInt);
    
         self.resume;
    end;
    
    destructor TNoAVThread.destroy;
    begin
         dispose(fPInt);
    
         inherited;
    end;
    
    procedure TNoAVThread.execute;
    begin
         while not self.terminated do
               fPInt^:=fPInt^+1;
    end;
    In this case, we have created our thread in its 'suspended' state. Now when the thread that called the constructor encounters the sleep request, it will yield to the OS (as before), but this time around, it will ignore the new thread because as far as it is concerned, its suspended. We can now fully initialise our thread object and then start executing by calling the 'resume' method.

    Of course, you don't have to call 'resume' within the constructor. You may want to create a thread and then have it sit there idle until its needed. In that case, simply ommit the 'self.resume' from the constructor and call the 'resume' method when you want the thread to begin executing.

    Code:
      ...
      myThread.create(true);
      ...
    
      ...
      // We need our thread to get busy
      myThread.resume;
      ...
    
      ...
      // And now we want it to sit idle
      myThread.suspend;
      ...
    Just a note about why in some cases I've overriden the constructors parameters... its my personal preference to override the constructor and remove the 'createSuspended' parameter because I generally try and design my threads to be as simple to use as possible, so any configuration I can put in the constructor, I do. I've also been bitten by the issue I described above, with the 'execute' method firing up before the constructor has completely finished, so I got myself into the habbit of creating all my threads suspended and then resuming them manually as and when I need to.

    Although this example is somewhat artificial because its unlikely you'd put 'sleep' in the constructor, if the constructor takes too long, the effect is the same. The 'execute' method could find itself trying to use variables etc. that are not yet fully initialised. On a single core machine you can get quite a while before the newly created thread gets some CPU time (I have just tried it on my Athlon 800 and I got around 1 second before I experienced an AV - For the curious, I had the constructor perform 2mil repititions of the calculation i=((i*2)/3) ), but on a dual (or more) core machine, this time will be radically reduced as the OS can shuffle threads around the multiple cores of the machine.

    The other option you have to get around this problem of threads running before they are initialised is to handle the initialisation and cleanup in the 'execute' method itself and create the thread in its running state (createSuspended=false).

    Code:
    procedure TMyThread.execute;
    begin
      // Initialise here
      try
        ...
        // Main execute code here
        ...
      finally
        // Cleanup here
      end;
    end;
    How you decide to handle this aspect of threads will ultimately depend on how you are using them and to a certain extent your own preferences which will develop as you get more experienced with multithreading.

    You may be asking yourself why you would want to suspend a thread when the whole idea is to have the thread running in the background. The answer is simple, performance. This is of particular concern when the application is running on a single core machine.

    Whilst a thread is active, the OS will allocate CPU time to it... even if the thread simply checks whether it has a job to do and then yields to the OS, that takes time... time which could probably be better spent doing other things (rendering for example). But, when you suspend the thread, the OS simply skips it. This saves the time it takes to perform two context switches and the time the thread uses in determining it has nothing to do before yielding to the OS as it waits for a job.

    This polling method of checking whether we need to do anything and yielding to the OS if there isn't can (if you have a lot of threads) result in slowdown that could affect the performance of your game as the system is constantly switching threads to check for work. It can also lead to delays in the commencement of job processing. This is because of how sleep works. If you say sleep(10), your thread will suspend its execution for 10ms. If you say sleep(1000), your thread suspends its execution for 1sec. Sleep(10) will result in a lot of context changes that could slow down your game. Sleep(1000) on the other hand won't have so many context switches, but could mean that in the worst case, the job you want your thread to do is effectively put on hold for 1sec.

    This is why you would suspend threads that don't have anything to do. If you have multiple cores, you can get away with leaving more threads in their running state, but this could result in poor performance on single cored machines as the OS allocates time to the threads you haven't suspended. So bear this in mind when designing a multithreaded game.

    So lets look at how you might use 'suspend' and 'resume' to ensure your thread is only running when it has work to do. In this example, I've created a simple job processing thread.

    Code:
    procedure TMyThread.execute;
    var
       aJob : TMyThreadsJobDescription;
    begin
      repeat
        if (self.workToDo) then
          begin
            aJob:=self.getNextJob;
           
            // Process the job here
    
       try
         aJob.onJobCompleted(self,aJob);
            except
              try
              aJob.free;
              except
              end;
            end;
    
          end
        else
          begin
            if not self.terminated then
              self.suspend;
          end;
      until (self.terminated);
    end;
    
    procedure TMyThread.stop;
    begin
      self.terminate;
    
      if (self.suspended) then
        self.resume;   
    end;
    
    procedure TMyThread.addJob(aJob:TMyThreadsJobDescription);
    begin
      // Add 'aJob' to the job queue
    
      if (self.suspended) then
        self.resume;
    end;
    Firstly, the exact content of aJob is not really important. It is an object that contains the data required by your thread to perform the required actions. Part of this is an event handler that your thread will call when it has finished processing the job. This isn't the only way to get data out of a thread, but in the context of a job processing thread, its a nice clean mechanism for letting the job creator know that its job is finished. You will of course notice the multiple try..except blocks around the event raising code. This is done just in case the object that created the job has been destroyed (an enemy requested a path but was blown up in the interim for example). It stops any exceptions raised in that block terminating the thread prematurely and ensures that we do our best to free up the job object that would ordinarily be cleaned up by the job creator when its finished with it.

    The other two methods handle killing the thread and adding jobs.

    The 'terminate' method we would normally use for terminating the thread only sets the 'terminated' flag to true. It does not resume a suspended thread, so if you don't resume the thread yourself, you could be waiting an awfully long time for your thread to terminate. For this reason, I've created a new method 'stop'. This calls 'terminate' and then checks whether or not the thread is suspended... if it is, it calls 'resume'. 'addJob' performs a similar check when a new job is added to the job queue.

    So, with this simple approach, we have a thread that will fire up, and then suspend itself when it has nothing to do (saving clock cycles), but when we add a job or kill it, we have very little delay since it gets straight to work when we resume it.

    Comments 6 Comments
    1. chronozphere's Avatar
      chronozphere -
      Fantastic articles athena. I will definitely read 'em all.

      Multithreading still scares me a bit and the only way to overcome that is by rolling up the sleeves haha... I would like to use them in my engine, to separate rendering and audio from the rest of the code.

      I really need to write an article too btw. There aren't that many and new site does a good job at presenting them.
    1. WILL's Avatar
      WILL -
      Quote Originally Posted by chronozphere View Post
      I really need to write an article too btw. There aren't that many and new site does a good job at presenting them.
      Feel free! We need a lot more beginner articles. Either an introduction to a specific API or a completely non-API specific article about the basics of game programming.
    1. AthenaOfDelphi's Avatar
      AthenaOfDelphi -
      Thanks

      It's not quite as scary as it may seem at first... although I speak from the point of view of someone who nearly got fired because the first time they used multi-threading was in a business critical application which kept crashing because I messed up

      After that, no programming problem has ever seemed quite so scary
    1. code_glitch's Avatar
      code_glitch -
      nice stuff. i have written one slightly less popular article as it would seem but am working on another as will would tell you with the newbie-frindly project or something
    1. Dan's Avatar
      Dan -
      AthenaOfDelphi,
      Do you have any comments on suspend and resume procedures being deprecated starting from D2009. Since you are using these methods in your examples it may confuse some people because stuff normally gets deprecated when better ways of doing things become available.
    1. AthenaOfDelphi's Avatar
      AthenaOfDelphi -
      Dan, apologies for the delay... well, what can I say... I didn't realise they were deprecated, certainly my compiler doesn't complain about anything (I get the feeling from reading one or two articles it should complain).

      Whilst I understand the theories I've read, I can honestly say I've (a) never used the suspend/resume mechanism for synchronising threads and (b) never had any problems using suspend/resume. I generally use it for job processing type scenarios where the thread put's itself to sleep until it is needed. Replacing these calls with an eventing mechanisms (wait_for_single_object IIRC) shouldn't be too much of a problem. Something like this:-

      Code:
      unit classSuspendableThread;
      
      interface
      
      uses
        sysUtils, classes, windows;
      
      type
        TSuspendableThread = class(TThread)
        protected
          fWakeUpEvent        : THandle;
          fSleeping           : boolean;
      
          procedure goToSleep;
        public
          constructor create(createSuspended:boolean);
          destructor Destroy; override;
      
          procedure terminate;
      
          procedure wakeUp;
      
          property sleeping:boolean read fSleeping;
        end;
      
      implementation
      
      constructor TSuspendableThread.create(createSuspended:boolean);
      var
        temp    : string;
        nameBuf : PWideChar;
      begin
        inherited create(createSuspended);
      
        temp:='WUE-'+intToHex(self.ThreadId,8);
        nameBuf:=strAlloc(length(temp));
        strPCopy(nameBuf,temp);
        fWakeupEvent:=createEvent(nil,false,false,nameBuf);
        strDispose(nameBuf);
      end;
      
      destructor TSuspendableThread.destroy;
      begin
        closeHandle(fWakeupEvent);
      
        inherited;
      end;
      
      procedure TSuspendableThread.terminate;
      begin
        TThread(self).terminate;
      
        if (fSleeping) then
        begin
          wakeup;
        end;
      end;
      
      procedure TSuspendableThread.wakeUp;
      begin
        if (fSleeping) then
        begin
          try
            setEvent(fWakeupEvent);
          except
          end;
        end;
      end;
      
      procedure TSuspendableThread.goToSleep;
      begin
        fSleeping:=true;
        waitForSingleObject(fWakeupEvent,INFINITE);
        fSleeping:=false;
      end;
      
      end.
      This class works, call 'wakeUp' instead of resume (although resume, or it's replacement start would still be needed if you created it suspended). You will of course note that the 'gotoSleep' method which replaces suspend is now protected. Calling it from anywhere other than the execute method will hold the calling thread.

      This solution is far from perfect. The Windows API calls keep a count of suspends and resumes which this class does not. I suspect there are numerous ways of solving the puzzle and your solution will depend very much on usage.

      Do I think deprecating them is a good idea... it would be a cold day in hell if I said yes. They are legitimate mechanisms for controlling threads (and as I say, ones I've used myself on numerous occasions with no problems). So, I've just gone looking for an answer. I think this excerpt sums it up nicely (taken from this article by Peter Richie):-

      For time eternal there has been a way to suspend (and by association resume) another thread. It wasn't until .NET 2.0 that someone took leadership and conceded that suspending another thread is not a safe thing to do. .NET 2.0 deprecates System.Threading.Thread.Suspend() and Resume() (although, Resume() isn't dangerous by association...).

      Basically, Suspend() has no care that the thread may actually be managing one or more complex invariants that can't be changed atomically. Normally with complex invariants synchronization is used to ensure while one thread is modifying the individual components of an invariant another thread doesn't try to access it. This is important because the validity of an invariant is rarely atomic; meaning changing an invariant may take several steps and may be in an unknown or invalid state between the first and the last step (a date is a good example, setting the day and month is two steps; until both steps are complete the date might be invalid). Suspending a thread circumvents the synchronization primitives and cuts the thread off at the knees.
      Basically, calling suspend from outside a thread is bad... well all I can say is "there's a suprise". I've never done it and I wouldn't encourage you to do it for the reasons Peter gives.... the caller of suspend has no idea about the status of the thread. So, maybe a better solution would to have been to change visibility of the suspend method so that it was only accessible to the thread itself.

      Who knows, but I certainly don't agree with removing them as it's going to mean we have some changes to do at work when we finally port to an up to date Delphi version.