When a villagers has been modified with a trade that has 2 separate components needed for the trade that happen to be the same item (e.g. BuyA 10 emeralds BuyB 5 emeralds) the villager will only take the BuyA items leaving the rest in the player's inventory. In my testing this only appears to happen on controller and not on keyboard and mouse. It has to do with the quick trade that can be done with a controller when pressing the A button instead of clicking onto the "Trade" button in the villager's interface in which case it'll work with controller too. Not sure if this is what's causing the issue but in my case the BuyA count was at 64.