Some illagers take quite a while to join a raid after a raid before them, in which they participated in, has ended. May also affect witches ravagers and illusioners.

They do eventually join, but some just take their sweet time as others join immediately. I don't know if that's a bug or a WAI, but it seems like a bug to me.