The owners said the high cost of rent would make it too expensive on the vendors that offer their wares at the Yellow Door.