So how much of the 6ms on my solution is used to call the function when you could instead just do the return value directly

I use a lot of similar lookups in my code. For example on a 2D map to calculate direction I'll store it in an array and just be using the characters facing I'll access the array to work out how much the x and y values change.

I use this sort of data caching a lot in business apps I write - even to the point of caching complete tables within oracle packages to make access faster.

instead of using integers in cairnswm's N lookup table, couldn't you use bytes and get both a performance and size win?
No because byte only goes to 256 and these numbers are larger - but maybe SmallInt could save space.

I decided a long time ago that memory space was there for programmers to use and not for users to try and save - so a cahe is always cool