Well, not just performance, but complexity and maintainability, too. Reference counting is a single method, usually part of a set of methods in garbage collection, so you'll likely be adding significant more complexity to the engine overall (for example, the caching issue you mentioned.. loading/freeing a constantly-used resource can exact a serious performance hit).

Making it optional is good feature-wise, but that will incur a small bit of overhead, too. Ultimately, it boils down to one question: how much of the garbage collection / caching work do you want your engine to do versus how much you want the programmer using your engine to do? Also, who is your target audience with your game engine? Advanced programmers are going to want more control and performance; beginners like not having to do a lot of "unnecessary stuff". If you can find a way to make both happy with it, then I'd do it; I'm just not sure doing both is feasible. Your mileage may vary, though.