That's them. I believe all you need to do is put a file called favicon.ico at the root directory of the website. It is generally a 16x16 256-colour icon with an optional 32x32 icon. I have created a small one based on the green logo at the top of this page.
http://www.users.on.net/~sly/favicon.ico
Someone else could probably do better.