I've got a scoreboard displayed on the sidebar and when I do /scoreboard players set * score 0 every logged player appeard on the scoreboard even though the score for some players for that objective has never been set before.

Resetting and setting the score to 0 for * fixes it although after relogging all scores appear again (Singleplayer).

It only appears when changing from another displayed scoreboard to the one you want OR when you set an scoreboard to sidebar when sidebar is cleared. https://www.youtube.com/watch?v=oz-gmsrjKA4

I could use # in front of offline players to hide them althouh players that have been online before but aren't online currently will still be displayed.