It sounds like you need to put the location of the house into the A* routine as unpassable or expensive nodes.