Items lose durability even when unused

What I expected to happen was...:
Nothing to lose durability

What actually happened was...:
Those items remain active and lose durability even when I don't use them

Steps to Reproduce:
1. Land in a pond with activated elytra and swim around
2. Try swimming up and down a lava lake
3. Ride a minecart, boat or a mob over soul sand or soil while wearing soul speed boots