It depends on what kind of game you are going for; commercial game project, for fun or just experimenting?

Even if it is any one of these, you don't have to use your initial graphics as your final. In many solo-indie game projects a lot of us just use placeholder graphics or what is called programmer's art. It's not meant to be the final graphics, but good enough to get the message across of what's going on and puts something in the hole that will be filled later with free/public domain stuff or volunteer artwork later on. From there you want to get your game to a playable state and work out all the features and gameplay as well as you can before seeking better graphics.

Music, you could wait until the end. This is very common with most games. However if you want to make some of the art or gameplay work with some music music you picked out, you could start adding music and sound effects to your game earlier. If you have not done a lot of game projects before, I'd recommend just adding the music after you've nearly completed the game and are tweaking/balancing and polishing things for release.

My self, I create the programmers art using either Photoshop or MS Paint (which is free!) of course there are others you can use, find which is best! Then after I've made my, not the best looking assests I need I plug them into my project's folder and begin coding my idea.

GIMP is pretty good though there may be some bugs depending on platform and there are some great tricks you can learn to make great graphics using filters and tricks with GIMP or Photoshop. Another option could be using something like Poser where you just buy or find free models that you can plug into Poser and create your sprite animations that way.

There is fan art/sprite websites out there that you can use for either a fan-made project or just to prototype your game. To get your game concept made can be done for free. To get the custom graphics you want for a project you need to find a person to make them for/with you or hire someone to create the graphics you need. Either has their good and bad points. Don't count on the total free option to always work!