I just figured out that it isn't possible to test for a players inventory...
It works if i test for an empty inventory like so:

/testfor @e[type=Player] {Inventory:[]}

But as soon as i add something like this:

/testfor @e[type=Player] {Inventory:[{id:"minecraft:torch"}]}

it will always output false.
I wanted this to test for an item with a specific name in a specific slot.