I guess it's a concept that will have to be tested to see how it actually will feel while playing. You've got me interested already!

Be sure to play test lots and be wiling to tweak and fiddle plenty when trying new "out of the box" game ideas. Not every new idea really goes smooth in it's first outing of course. It's also a good idea to be willing to rethink stuff if you just aren't getting your player experience to where you are hoping it would. Also when you are in new territory, sometimes even the most subtle of tweaks or added features can vastly change your whole game without noticing until after play testing it enough times.