Doh, error was in my own CreateMatrix().. it was built wrong sided.

It was your code Brainer that worked first though

I replaced
[pascal]_41 := Dot(Scale(xAxis, -1.0), From);
_42 := Dot(Scale(yAxis, -1.0), From);
_43 := Dot(Scale(zAxis, -1.0), From);[/pascal]
with
[pascal]_41 := Dot(Scale(xAxis, -1.0), At);
_42 := Dot(Scale(yAxis, -1.0), At);
_43 := Dot(Scale(zAxis, -1.0), At);[/pascal]
And it worked straight away.

My function optimized is now:
[pascal]function LookAt(const eye, target, up: TVector): TMatrix;
var x,y,z,p: TVector;
begin
z:=norm(VectorSub(eye,target));
x:=norm(crossproduct(up,z));
y:=crossproduct(z,x);
p:=vector(-dot(eye,x),-dot(eye,y),-dot(eye,z));
result:=CreateMatrix(x,y,z,p);
end;[/pascal]