Hey captain jaster.

Welcome to the forums.

A compiler: Delphi is of course a popular choice. There's the commercial product, but Turbo Delphi works equally well. Chronozphere also mentioned FreePascal as a possible option.

Next, choose a directx/opengl wrapper. There are many available, DelphiX used to be quite popular, but nowadays Asphyre or Hadron Game Engine (aka PyroGine) are two of the better options to choose from. Note that it is not required to use these packages, but they tend to make life a lot easier as much of the hard work is already done for you. i.e You don't have to think anymore setting up the graphical interface, capturing input, setting up sound, working out collision detection systems etc etc. Mind you, these are 2d engines, there are other libraries for 3d. Also, both come with demos. Have a look at them, learn from them.

Ok, with that out of the way, next thing you should be thinking about is what kind of game you want to make. A puzzle game, platform game, shooter etc. Also consider for a moment if you're able to get the assets for it, especially the graphical content.
Judging from your post I'd say you are quite new to all of this so I would recommend choosing something simple like a Space Invaders or an Asteroids clone.

Anyway, I hope this help a bit