When inside a building at night with villagers, at least one villager is prone to stand in the doorway, and refuse attempts to push them away. This makes it impossible to open the door, as any click in that direction will instead open the trading interface.