in which order do you check the collision? does the asteroid check first or the bullet? I suggest you put your collision detection on the fastest object. also, if the bullet is inside the polygon take it as a collision, even if it didn't hit any lines in between movement positions. that should resolve your troubles.
EDIT: another thing to have in mind. bullets and asteroids don't dodge each other, they follow straight paths. so, if from one frame to the next, the bullet position cross the path of an asteroid, it's a hit too even if they didn't hit due to small sized asteroid.
Bookmarks