The issue is that enderman cannot carry blocks with id larger then 127, the block will not render and the game will crash when they try to swap the block again.

They WILL carry any block with id under 127.

All tests where done using the emerald block (id 133).

It first happened when I was creating a bukkit plugin that edited the carried and carriedData NBT tags for the enderman, I was able to spawn them but, as said before, the block did not render and the server crashed when one of these enderman tried to pick another block.

Then I edited some more enderman (with emerald block) and closed the server before they picked any other block, then after loading the map on single player and waiting a few seconds, one of the enderman picked a block and the game crashed.

It seems that as the block id is internally saved as byte it is not properly converted to int when using it as a index of the "material array".....
It keeps the sign bit and messes up the integer sign, probably there is a missing " int idAsInteger = (idAsByte & 0xFF); " somewhere on the code.
This is suported by the fact that the Exception is ArrayIndexOutOfBoundsException and the index is -123.
-123 is the same as 133 (emerald block) when using the last by as sign (two's complement)

The same may occur for carriedData (not tested)

Crash reports below!