I need an optimized file read/write schema that will allow me to read and write my structures in blocks to/from the drive itself. I'll just have to get a sample put together and see if anyone has any good ideas .

The basic idea is to get rid of the swapping that occurs now do to the massive index size (quickly out runs the 64GB of ram when dealing with terabyte's of data). Since I'm already swapping to disk it makes more sense to utilize the drive directly.

IE:
OpenIndexFile;
LocateRootRecord;
TraverseTheRabbitHole;
RetrieveRecord;
CloseIndexFile;

For an idea of the record structure you could look at my article on Trie's in the 1st Pascal Gamer mag. It has the basic concept, just disk bound instead of memory bound.

- Jeremy