When an Enderman is angered either by being hit or staring at its face then hit with a sword or weapon, it will automatically deaggro and goes back to walking around.