Minecraft for Windows 10 crashes when playing with Oculus Rift after you remove your headset.

This happens 100% of the time: When I start playing Minecraft VR everything works perfectly. The game runs very smoothly, everything looks good and works as it should, but whenever I remove the headset and put it back on (or even lift it even for a second), the game completely freezes and I need to kill the process in order to reset it and start again.

When I remove the headset, I can see MC running on the Oculus Rift mirror window and it reacts to movement and all, but if the headset is activated again, the 'Minecraft' logo appears to the mirror window and everything freezes (Oculus and the game). It's not random, it happens every single time.

All other VR games work perfectly, no other issues or problems in my system. All drivers are up-to-date, W10 has just been reinstalled including a DDU-assisted graphics card installation.