• Recent Tutorials

  • Tripping The Class Fantastic: Multi-threading Part 1

    Resource Protection

    Resource protection is all about ensuring that two threads don't access the same resource at the same time. The implications of two threads accessing a TList for example could be quite bad. One checks for an item count... it finds the list has a single item so it reads it. Another thread checks for the item count from the same list... it also finds the list has a single item... in the meantime, the first thread deletes the object its just grabbed from the list. The second thread goes for it and consequently goes bang.

    To avoid this kind of situation, various tools are provided... Synchronisation and object locking, we've already covered... that leaves 'TCriticalSection' and the nicely named 'TMultiReadExclusiveWriteSynchronizer'.

    These two are essentially the same but with one subtle difference... the long one can help you avoid deadlock by allowing you to specify what kind of operation you are going to perform. But first, lets look at the short one... 'TCriticalSection'.

    'TCriticalSection' can be considered to be a token. Without a token, your thread can't enter the code it protects, and there is only a single token meaning that only a single thread can enter code protected by a critical section at any one time. In our revised job processing thread, we have these two blocks of code.

    Code:
      fJobListCS.acquire;
      try
        fJobList.add(aJob);
      finally
        fJobListCS.release;
      end;
    From the 'addJob' method, and...

    Code:
      fJobListCS.acquire;
      try
        fJob:=TMyJobDescription(fJobList[0]);
        fJobList.delete(0);
      finally
        fJobListCS.release;
      end;
    From the 'execute' method. Its not a good idea to be adding and removing items from the same list at the same time, so we've protected the list with a critical section ('fJobListCS'). This is created in the constructor (fJobListCS:=TCriticalSection.create) and destroyed in the destructor (fJobListCS.free). It is then used to ensure that only one thread can access the job list at once.

    At this point, just in case its not clear... when you call a method of a thread object, even though the method belongs to a thread running its its own context, the method will be executed within the context of the calling thread. So whilst we will only ever remove jobs from the queue within the context of the job processing thread, any other thread (the main VCL thread included) that has access to the job processor can add a job.. thats why protecting the list when we add a job is so important.

    Thankfully, using a critical section is pretty straight forward as you can see from this example. The methods we use are 'acquire' to obtain control of the critical section and 'release' to relinquish control.

    Critical sections are not without their problems... they take time and can present a processing bottleneck, especially if you have alot of threads all trying to access common data through the same critical section. But these issues pale into insignificance when compared with deadlock. Deadlock occurs when two (or more) threads try to access the same critical section and for whatever reason, one thread is waiting for something else to happen before it releases the critical section for the others to use.

    There are two key ways this can happen. The first is when you unexpectedly leave a block of code and fail to release the critical section. This will most likely be courtesy of an exception. For that reason you should ALWAYS use try...finally when using critical sections as illustrated in the example. Failure to do so could result in your thread retaining control of a critical section when it should have relinquished it.

    The other way you can end up with deadlock is when you have multiple critical sections protecting different data sets.

    Code:
    procedure TMyThread1.execute;
    begin
     
      ...
    
      fGlobalDataQueueCS.acquire;
      try
        fGlobalResultBufferCS.acquire;
        try
    
        ...
    
        finally
          fGlobalResultBufferCS.release;
        end;
      finally
        fGlobalDataQueueCS.release;
      end;
    
      ...
    
    end;
    
    procedure TMyThread2.execute;
    begin
    
      ...
    
      fGlobalResultBufferCS.acquire;
      try
        fGlobalDataQueueCS.acquire;
        try
    
        ...
    
        finally
          fGlobalDataQueueCS.release;
        end;
      finally
        fGlobalResultBufferCS.release;
      end;
    
      ...
    
    end;
    In this example, fGlobalDataQueueCS and fGlobalResultBufferCS are property variables that hold references to two global critical sections. Thread 1 hits the first critical section which protects the global data queue... thread 2 on the other hand gets the critical section that protects the global result buffer. Thread 1 trys for the global result buffer critical section but of course thread 2 has it. Thread 2 trys for the global data queue critical section but thread 1 has it. Neither can progress because they are both trying for the critical section the other has.

    To avoid this scenario, if you use multiple critical sections to protect different resources, always make sure that the critical sections get nested in the same order.

    There are other ways to end up in a deadlock situation, but by thinking ahead and planning your threads and how they interact with common objects, you can practically reduce the chances of it happening to 0. To help make your life a little easy in this respect, we come to the aptly named 'TMultiReadExclusiveWriteSynchronizer'.

    So what does such a nicely named object do... well, like a critical section, its purpose is to protect resources from simultaneous access by multiple threads... unlike a critical section however, it allows you to do that according to the type of operation you are going to perform.

    If you have multiple threads and they all want to read the protected resource... no problem, they can all read it at the same time providing another thread isn't writing to it... if they want to write to it however, they will have to wait until everyone has finished reading and then they will have to take it in turns as only a single thread is allowed to write to the protected resource at once.

    Code:
      ...
    
      // Reading from the protected resource
    
      fOurMultiReadExclusiveWriteSynchronizer.beginRead;
      try
        // Read from the protected resource
      finally
        fOurMultiReadExclusiveWriteSynchronizer.endRead;
      end;
    
      ...
    
      // Writing to the protected resource
      fOurMultiReadExclusiveWriteSynchronizer.beginWrite;
      try
        // Write to the protected resource
      finally
        fOurMultiReadExclusiveWriteSynchronizer.endWrite;
      end;
    
      ...
    This allows you to reduce the bottlenecks that critical sections can introduce as you only ever restrict access to a single thread when you're writing to the resource.

    Comments 6 Comments
    1. chronozphere's Avatar
      chronozphere -
      Fantastic articles athena. I will definitely read 'em all.

      Multithreading still scares me a bit and the only way to overcome that is by rolling up the sleeves haha... I would like to use them in my engine, to separate rendering and audio from the rest of the code.

      I really need to write an article too btw. There aren't that many and new site does a good job at presenting them.
    1. WILL's Avatar
      WILL -
      Quote Originally Posted by chronozphere View Post
      I really need to write an article too btw. There aren't that many and new site does a good job at presenting them.
      Feel free! We need a lot more beginner articles. Either an introduction to a specific API or a completely non-API specific article about the basics of game programming.
    1. AthenaOfDelphi's Avatar
      AthenaOfDelphi -
      Thanks

      It's not quite as scary as it may seem at first... although I speak from the point of view of someone who nearly got fired because the first time they used multi-threading was in a business critical application which kept crashing because I messed up

      After that, no programming problem has ever seemed quite so scary
    1. code_glitch's Avatar
      code_glitch -
      nice stuff. i have written one slightly less popular article as it would seem but am working on another as will would tell you with the newbie-frindly project or something
    1. Dan's Avatar
      Dan -
      AthenaOfDelphi,
      Do you have any comments on suspend and resume procedures being deprecated starting from D2009. Since you are using these methods in your examples it may confuse some people because stuff normally gets deprecated when better ways of doing things become available.
    1. AthenaOfDelphi's Avatar
      AthenaOfDelphi -
      Dan, apologies for the delay... well, what can I say... I didn't realise they were deprecated, certainly my compiler doesn't complain about anything (I get the feeling from reading one or two articles it should complain).

      Whilst I understand the theories I've read, I can honestly say I've (a) never used the suspend/resume mechanism for synchronising threads and (b) never had any problems using suspend/resume. I generally use it for job processing type scenarios where the thread put's itself to sleep until it is needed. Replacing these calls with an eventing mechanisms (wait_for_single_object IIRC) shouldn't be too much of a problem. Something like this:-

      Code:
      unit classSuspendableThread;
      
      interface
      
      uses
        sysUtils, classes, windows;
      
      type
        TSuspendableThread = class(TThread)
        protected
          fWakeUpEvent        : THandle;
          fSleeping           : boolean;
      
          procedure goToSleep;
        public
          constructor create(createSuspended:boolean);
          destructor Destroy; override;
      
          procedure terminate;
      
          procedure wakeUp;
      
          property sleeping:boolean read fSleeping;
        end;
      
      implementation
      
      constructor TSuspendableThread.create(createSuspended:boolean);
      var
        temp    : string;
        nameBuf : PWideChar;
      begin
        inherited create(createSuspended);
      
        temp:='WUE-'+intToHex(self.ThreadId,8);
        nameBuf:=strAlloc(length(temp));
        strPCopy(nameBuf,temp);
        fWakeupEvent:=createEvent(nil,false,false,nameBuf);
        strDispose(nameBuf);
      end;
      
      destructor TSuspendableThread.destroy;
      begin
        closeHandle(fWakeupEvent);
      
        inherited;
      end;
      
      procedure TSuspendableThread.terminate;
      begin
        TThread(self).terminate;
      
        if (fSleeping) then
        begin
          wakeup;
        end;
      end;
      
      procedure TSuspendableThread.wakeUp;
      begin
        if (fSleeping) then
        begin
          try
            setEvent(fWakeupEvent);
          except
          end;
        end;
      end;
      
      procedure TSuspendableThread.goToSleep;
      begin
        fSleeping:=true;
        waitForSingleObject(fWakeupEvent,INFINITE);
        fSleeping:=false;
      end;
      
      end.
      This class works, call 'wakeUp' instead of resume (although resume, or it's replacement start would still be needed if you created it suspended). You will of course note that the 'gotoSleep' method which replaces suspend is now protected. Calling it from anywhere other than the execute method will hold the calling thread.

      This solution is far from perfect. The Windows API calls keep a count of suspends and resumes which this class does not. I suspect there are numerous ways of solving the puzzle and your solution will depend very much on usage.

      Do I think deprecating them is a good idea... it would be a cold day in hell if I said yes. They are legitimate mechanisms for controlling threads (and as I say, ones I've used myself on numerous occasions with no problems). So, I've just gone looking for an answer. I think this excerpt sums it up nicely (taken from this article by Peter Richie):-

      For time eternal there has been a way to suspend (and by association resume) another thread. It wasn't until .NET 2.0 that someone took leadership and conceded that suspending another thread is not a safe thing to do. .NET 2.0 deprecates System.Threading.Thread.Suspend() and Resume() (although, Resume() isn't dangerous by association...).

      Basically, Suspend() has no care that the thread may actually be managing one or more complex invariants that can't be changed atomically. Normally with complex invariants synchronization is used to ensure while one thread is modifying the individual components of an invariant another thread doesn't try to access it. This is important because the validity of an invariant is rarely atomic; meaning changing an invariant may take several steps and may be in an unknown or invalid state between the first and the last step (a date is a good example, setting the day and month is two steps; until both steps are complete the date might be invalid). Suspending a thread circumvents the synchronization primitives and cuts the thread off at the knees.
      Basically, calling suspend from outside a thread is bad... well all I can say is "there's a suprise". I've never done it and I wouldn't encourage you to do it for the reasons Peter gives.... the caller of suspend has no idea about the status of the thread. So, maybe a better solution would to have been to change visibility of the suspend method so that it was only accessible to the thread itself.

      Who knows, but I certainly don't agree with removing them as it's going to mean we have some changes to do at work when we finally port to an up to date Delphi version.