My own archive format

[Robert Kosek]

I'll check your format out shortly, I was just writing this for some general advice plus some code examples.

You should know where to put the archive is always at the end, so you needn't tally a thing. This should be programmatic, rather than arithmetic.

Archive creation:

  //Write the header at position 0
  //Write the compressed files to the stream
  Header.FileListOffset = Stream.Position;
  //Write the array of records to the stream, the file list
  //Refresh the header with the actual file list offset
  Stream.Seek(0, soFromBeginning);
  Stream.Write(Header, sizeof(header));
  Stream.Free;
end;

Archive update:

  Stream.Position := Header.FileListOffset;
  //Add new compressed files: this overwrites the header
  Header.FileListOffset = Stream.Position;
  //Write the array of records to the stream, the file list
  //Refresh the header with the actual file list offset
  Stream.Seek(0, soFromBeginning);
  Stream.Write(Header, sizeof(header));
  Stream.Free;
end

My archive file records always have these elements:

  FileRecord = record
    // It could be anywhere, and an int64 is just a wise choice here.
    start: Int64;
    // Compressed file size (because you can read until there's no input,
    // you don't need an uncompressed length field.)
    len: Longword;
    // And this says whether I used BZip or ZLib, which you can omit.
    bzipped: boolean;
  end;

To save an array of these to the stream is quite simple:

Stream.Write(files[0], Count * SizeOf(FileRecord));

To read:

  SetLength(files,fCount);  // store the file count in the header!
  Stream.Read(files[0],SizeOf(FileRecord)*fCount);

People like to say this saves the metadata for a dynamic array, but I disagree. The main reason I disagree is that I factor the size of only the elements of the array, and not its meta data; because I write from element 0 to the length of the elements, it acts like an extraction of the elements from the metadata. At least, so far as I know.

If you want a short string, 255 characters worth, as your ID then just add "id: shortstring" to it. Pascal can easily handle this data's transference to the stream. Now, if you want longer file names ... you must use a stringlist and treat both the list plus your array of records as an indexed list, and only change positions of records and names together. (You can even compress the file list for better compression ratios.)

Directories, in my format, are just a part of the filename within the archive. Thus if I want "my dir/my file.txt" I tell it to extract just that. Why worry about virtual folders? That's more trouble than it is worth. If you want more fancy folder enumeration stuff, you could build that into your file list class.