"Leaderboard is hidden" not working

When I check the option "leaderboard is hidden" in the editor, the leaderboard is still visible for non-admins users:

It just adds a message below the leaderboard.