Why don't you simply trim the selection rectangle to camera rectangle boundaries.

First make yourself a SelectionBox record similar like this:
Code:
RSelectionBox = record
  LeftBorder: Single;
  RightBorder: Single;
  TopBorder: Single;
  BottomBorder: Single;
end;
Then I also recomend making similar record for storing camera viewpoint boundaries:
Code:
RCameraViewpoint = record
  LeftBorder: Single;
  RightBorder: Single;
  TopBorder: Single;
  BottomBorder: Single;
end;
Finaly you simply check to see if selection borders are within camera viewpoint boundaries if not you modify them to be on camera viewpoint boundaries. Code would look something like this:
Code:
function TrimSelectionBox(SelectionBox: RSelectionBox; CameraViewpoint: RCameraViewpoint): RSelectionBox;
begin
  if SelectionBox.LeftBorder < CameraViewpoint.Leftborder then Result.LeftBorder := CameraViewpoint.LeftBorder;
  if SelectionBox.RightBorder < CameraViewpoint.RightBorder then SelectionBox.RightBorder := CameraViewpoint.RightBorder;
  if SelectionBox.TopBorder < CameraViewpoint.TopBorder then SelectionBox.TopBorder := CameraViewpoint.TopBorder;
  if SelectionBox.BottomBorder < CameraViewpoint.BottomBorder then SelectionBox.BottomBorder := CameraViewpoint.BottomBorder;
  result := SelectionBox;
end;