Making a queue private does not remove it from benchmarks

See #1012 (closed)

• Make a queue public
• Another user add the queue to its benchmark as the queue is public
• Make the queue private
• The queue should be back to default in the benchmark