The button is a Windows 9x bug for apps closing while in a full-screen window. You can remove it by ensuring that your program doesn't finish in full-screen mode. I forget the exact method involved, but it's something like setting "WindowState := wsNormal" in the main form's OnDestroy event. You may need to give it a border too (or try wsMinimized for the window style). However, that's the basic solution for ya.

Keep in mind that 9x is getting _long_ in the tooth these days - don't care too much about such a small, 9x-specific "bug" .

Also, I found clicking the empty taskbar buttons quite relaxing. Maybe your next game can generate empty taskbar buttons and have the user click them before the taskbar gets filled up (or shoot them if you're into violence). Yes... I command you to make that game.