I think it's like this in reality, too. A gray cuboid on a white, reflecting surface is reflected more white than gray. If it's reflected on a gray surface, it seems gray.