this sometimes has to do with moving the light of geometry instead of the camera itself. I didn't look trough your code, but I think this is the problem. Matrix problem. Check those.