If all you need is simple playback, it's hard to go wrong with SDL... especially with free pascal now coming with the appropriate SDL units built in and headers available for just about anything. Besides SDL gives you easy to access keyboard, mouse and stick interfaces, relatively simple (and fully cross-platform) 2d graphics, and makes a better wrapper for OpenGL than GLUT ever did.

Though if you need something more complex or somethign that plays nice with directX, I say go with Will's advice and dive into OpenAL.