I have found where the problem is:
{$I imjconfig.inc} is right after uses now in JpegLib, but MODE must be before uses to work. I had delphi mode hard-coded in fpc.cfg so I didn't notice anything (and it somehow worked in Linux and BSD). I'll update the files and commit them to SVN repository.

Did you have any success getting Imaging work in D4?