When you press space to activate the elytra, they only do so after a round trip to the server. Seeing that the acceleration of gravity is 32 m/s^2 for players, you can easily drop multiple blocks and then have to cancel the vertical speed you've built up. The lost height is quadratic until you approach terminal velocity; a ping that's twice as high is four times as bad. As conditions worsen things quickly change from workable to smashing right into the ground.

All other movement is predicted/client-owned and corrected. Why not this as well? The server could still cancel your flight if it turned out to be illegal, much like the way you're warped back when you attempt to walk through server-side blocks.