• Recent Tutorials

  • Tripping The Class Fantastic: Versioned Data Storage

    Sample Base Class

    This unit is derived from my own streamed object. It just doesn't include some of the more exotic functions that make it more functional. It doesn't do anything fancy, but it works.

    Code:
    unit streamedObject;
    
    interface
    
    uses
        classes, sysUtils;
    
    type
        (*---Method Prototypes (Required for dynamic procedure calls)---------------------------------------------------------*)
    
        TStreamingReadPrototype = procedure(reader:TReader) of object;
        TStreamingWritePrototype = procedure(writer:TWriter) of object;
    
        (*---TORSStreamedObject---------------------------------------------------------*)
    
        TStreamedObject = class(TObject)
        private
    
        protected
    
          fObjectVersion     : integer;
          fVersionInFile     : integer;
    
          procedure raiseEx(msg:string);
    
        public
          constructor create;
    
          procedure load(reader:TReader); virtual;
          procedure save(writer:TWriter); virtual;
    
          procedure loadFromStream(src:TStream); virtual;
          procedure saveToStream(dst:TStream); virtual;
    
          property objectVersion:integer read fObjectVersion;
    
        published
    
          // Declare these in the derived class (they MUST be published)
          // procedure load_verx(reader:TReader); virtual;
          // procedure save_verx(writer:TWriter); virtual;
        
        end;
    
        (*---Exception class---------------------------------------------------------*)
    
        EStreamedObjectException = exception;
    
    implementation
    
    const
         _streamedObjectBufferSizeLoad = 2048;
         _streamedObjectBufferSizeSave = 2048;
    
    (*---Exception raiser---------------------------------------------------------*)
    
    procedure TStreamedObject.raiseEx(msg:string);
    begin
         raise EStreamedObjectException.create(self.className+msg);
    end;
    
    (*---Constructor---------------------------------------------------------*)
    
    constructor TStreamedObject.create;
    begin
         inherited create;
    
         fObjectVersion:=1;
    end;
    
    (*---IO Method Selectors---------------------------------------------------------*)
    
    procedure TStreamedObject.load(reader:TReader);
    var
       methodPtr     : pointer;
       method        : TMethod;
    begin
         try
            fVersionInFile:=reader.ReadInteger;
         except
           on e:exception do
              raiseEx('.load - Exception occured reading version, message was '+e.message);
         end;
    
         if (fVersionInFile<1) or (fVersionInFile>fObjectVersion) then
            raiseEx('.load - Expecting version 1..'+intToStr(fObjectVersion)+', found '+intToStr(fVersionInFile));
    
         methodPtr:=nil;
    
         while (fVersionInFile>0) and (methodPtr=nil) do
               begin
                    methodPtr:=self.methodAddress('Load_Ver'+intToStr(fVersionInFile));
                    if (methodPtr=nil) then
                       dec(fVersionInFile);
               end;
    
         if (methodPtr<>nil) then
            begin
                 method.data:=self;
                 method.code:=methodPtr;
    
                 TStreamingReadPrototype(method)(reader);
            end
         else
             raiseEx('.load - Could not locate load method Load_Ver'+intToStr(fObjectVersion));
    end;
    
    procedure TStreamedObject.save(writer:TWriter);
    var
       methodPtr     : pointer;
       method        : TMethod;
    begin
         try
            writer.writeInteger(fObjectVersion);
         except
           on e:exception do
              raiseEx('.save - Exception occured writing version, message was '+e.message);
         end;
    
         fVersionInFile:=fObjectVersion;
    
         methodPtr:=nil;
    
         while (fVersionInFile>0) and (methodPtr=nil) do
               begin
                    methodPtr:=self.methodAddress('Save_Ver'+intToStr(fVersionInFile));
                    if (methodPtr=nil) then
                       dec(fVersionInFile);
               end;
    
         if (methodPtr<>nil) then
            begin
                 method.data:=self;
                 method.code:=methodPtr;
    
                 TStreamingWritePrototype(method)(writer);
            end
         else
             raiseEx('.save - Could not locate save method Save_Ver'+intToStr(fObjectVersion));
    end;
    
    (*---Main IO Routines---------------------------------------------------------*)
    
    procedure TStreamedObject.loadFromStream(src:TStream);
    var
       reader        : TReader;
    begin
         if (src.size>0) then
            begin
                 reader:=TReader.create(src,_streamedObjectBufferSizeLoad);
    
                 try
                    self.load(reader);
                 finally
                   try
                      reader.free;
                   except
                   end;
                 end;
            end
         else
             raiseEx('.loadFromStream - Stream is empty');
    end;
    
    procedure TStreamedObject.saveToStream(dst:TStream);
    var
       writer        : TWriter;
    begin
         writer:=TWriter.create(dst,_StreamedObjectBufferSizeSave);
         try
            self.save(writer);
    
            writer.flushBuffer;
    
         finally
           try
              writer.free;
           except
           end;
         end;
    end;
    
    end.
    I'm not going to do a full breakdown of the code as for the most part it is pretty self explanatory and not too complex. The only thing I really want to discuss is the method search section of the load and save methods.

    In both cases, the routines start at a high version (the current fObjectVersion in the case of save and the version read from the stream in the case of load) and work down to 1 looking for the methods load_verx and save_verx respectively.

    This makes our data store capable of handling older formats and ensures that when we save the store (during development/editing) it is always stored using the latest data format.

    And, in case you're new to streams, lets just mention TReader and TWriter.

    TStream (and its descendants) doesn't actually have any methods for reading and writing say an Integer. You can use Read and Write, but then you have to concern yourself with addresses and variable sizes. The easier way (as I have done) is to use TReader and TWriter. These are helper objects that provide methods for direct storage and retrieval of data... TReader.readInteger and TWriter.writeInteger for example. This is not a perfect solution as by denying access to the source and destination streams within the load_verxx and save_verxx methods, life gets difficult if you want to store a stream within the stream, but the trade off is worthwhile because for the majority of the time your life is made considerably easier by using TReader and TWriter.

    If you are wondering why you would want to save a stream within a stream... consider TCollection... you have a collection object within your data store object... the easiest way to load and save it is to use its own loadFromStream and saveToStream methods. One approach would be to use an intermediate stream (TStringStream) for example... save the object to the TStringStream and then save the TStringStream's 'DataString' property using TWriter.writeString. Its a bit clunky, but it is one approach to loading and saving streams within the load_verx and save_verx methods.

    Comments 2 Comments
    1. Brainer's Avatar
      Brainer -
      Very nice one.

      Regarding TReader and TWriter - in the latest Delphi version you can write class helpers, so it's possible to expand the functionality of TStream and add your own methods for, let's say, storing and reading back a floating-point, while still having the possibility to store a stream in another one.
    1. AthenaOfDelphi's Avatar
      AthenaOfDelphi -
      Thanks Brainer