I named a guardian, I went away for a bit, when I came back he was gone.

It doesn't seem to happen on multiplayer though, only on singleplayer.