Ray test mouse picking step size

[Thyandyr]

I don't understand what you mean by steps even, using mouse-ray is not iteration. Usually you would use line-to-triangle intersect, line-to-sphere intersect and so on functions. It's normally not at all about pixels. For that you may be able to do something different, like glReadPixels at cursor's screen coordinates? I don't know, very much depends on what the app needs to do.

A ray from camera that goes through the world coordinate that match the mouse pointer. The vector is normalized for Z.

How to know which world coordinates this ray traverses through? I iterate the ray from camera towards -Z (openGL), and check if anything is supposed to be in the world coordinate (cube based world), and pick the first match, or end up too far without any match. In practice it works great with steps like 0.5 and 0.3. (Z)

The question is more of a theoretical nature. If you have a ray (for example close to diagonal) to the world axes, how do you know that you do not skip the narrow corner of a world coordinate. Say the mouse is middle of screen on the right, at 1,0 normalized. The vector could be something like (0.95, 0, -1). The vector crosses world space (0,0,0), (1,0,-1), (2,0,-2) and so forth. But it could/would cut through the corner of space (0,0,-1) before going to (1,0,-1). Iterating with arbitrary step will occasionally miss and hit these corners in unpredictable fashion. No big deal in practice, but it is unwanted behaviour. I'm sure there is some math to do this properly but I don't know of it.