Task removal affects old submissions to

I had task 1 connected to competition and the scores were showing in the leaderboard:

Then I removed this task from the competition and added a new one and now I see n/a for old submissions.

It makes sense why this is happening but should this happen?