When setting the players attack speed to a high enough number using the following command:
"/attribute @s minecraft:generic.attack_speed base set 30". The cooldown indicator appears when switching slots shortly and then quickly disappears despite it shouldn't be doing so. This also affects higher values of haste effect. 
This issue doesn't disappear with any higher attack speed values. 

I made a video showcasing this happening at both 30 attack speed and a higher number in survival. You can see the indicator flashing quickly whenever slots are switched. https://www.youtube.com/watch?v=xO5oqheWJOk
This should be a issue that affects most servers which utilise this.