If you're using c++ style template, you can try this on your glCreateWnd procedure:

Code:
  with wndClass do                    // Set up the window class
  begin
    style         := CS_HREDRAW or    // Redraws entire window if length changes
                     CS_VREDRAW or    // Redraws entire window if height changes
                     CS_OWNDC;        // Unique device context for the window
    lpfnWndProc   := @WndProc;        // Set the window procedure to our func WndProc
    hInstance     := h_Instance;
    hCursor       := LoadCursor(0, IDC_ARROW);
    hIcon         := LoadIcon(h_Instance,'MAINICON');
    lpszClassName := 'OpenGL';
  end;
All you have to do is to add an icon by the project menu and goto options, or you can replace MAINICON in the code by the name of your icon in your res file