Buffer Memory Management

[igmac]

Well, I am one of those who believe that the slowest way to write a program is to start coding immediately. ie, I believe in design before coding.

I might be sensitive to the task though, because I have recently done a bit of work on very high performance networking where using zero-copy with scatter/gather made phenomenal differences in an iSCSI setup. In essence I am taking about zero-copy.

On the other hand, simply designing the code with the thought in mind that I may later wish to implement zero-copy, may be sufficient. It's much easier to modify or add something in if you have designed around that possibility in the beginning.

[LP]

Well, I am one of those who believe that the slowest way to write a program is to start coding immediately. ie, I believe in design before coding.

No, you seem to incline on low-level performance optimizations even before you have started the project. You haven't stated a word about the design, just some bits of (bandwidth) requirements.

I might be sensitive to the task though, because I have recently done a bit of work on very high performance networking where using zero-copy with scatter/gather made phenomenal differences in an iSCSI setup. In essence I am taking about zero-copy.

In original post you have mentioned 10 Mb/sec bandwidth. For this kind of bandwidth using simple Pascal arrays and passing parameters by reference will suffice.

[igmac]

Yes indeed, I am definitely talking a very single, very specific, low level performance optimisation here. Hence the post in 'tools and methodologies' rather than more specifically related to any individual project.

Having now gone and looked at the Linux kernel implementation since I first posted, I am more certain that in very high bandwidth - for example, but by no means an exclusive example, 10Gbit network - this is the way to do it.

They have a very elegant mechanism, essentially shared memory with status flags, that allow synchronised access to buffers between the two parties. This is a much better idea than buffer passing, and better and simpler than what I was thinking of.

I should have thought of this because I recall that some many years ago, Falcon 4 used a similar technique to provide state information, rather than providing a full API. It worked very well.

Thanks for all the thoughts.