I made a standard carrot/potato farm; one side potatoes one side carrots a farmer on each side, a nonfarmer villager trapped in the center (tried both just a fence post as well as a few minecarts with hoppers). The carrot farmer never harvests a single carrot at all.  The potato farmer farms like crazy picking potatoes constantly (high tick speed so the plants grow back fast) but never stops to hand off the potatoes to the nonfarmer.

I have tried replacing all villagers (new eggs) and composters several times but the exact same thing happens, whether their inventory is full or not does not seem to make a difference either.