The explorer add ins are called Shell Extensions and you are looking for the Thumbnail Image Handler extension.

http://sourceforge.net/projects/tpshellshock/

Encapsulates a number of the Shell Extensions - I could not find the Thumbnail handler but it could be there somewhere.