When looking at an enderman momentarily, like under a second, the enderman will look as if it is aggressive when it doesn't attack you. I have done this a few times from far away, but doesn't seem to work close up. Another thing I would suggest is that you would only get the enderman sound from looking at it only if it turns aggressive. Looking at an enderman very quickly, and not making it aggressive, will play the sound that you would get from looking at it, and I think that it should only play the sound if it turns aggressive.