Submission manager

• Finalize submission Manager

Clean up the division of work between the ingestion program and the scoring program, such that they can be run in parallel in tight compartments, interacting via storage only. Check security of data: only the scoring program should have access to truth values.

Support a variety of use cases with result or code submission and the possibility of using a scoring program interacting with the outside world (e.g. scores provided by humans or by an external web service).

You can see how the ingestion program and such work currently on the old compute worker, here: https://github.com/codalab/competitions-v1-compute-worker/blob/feature/legacy-azure-version-fix/worker.py#L511

Example scenario where a user makes a CODE submission to a phase with an INGESTION PROGRAM..

Compute worker:

1. Receives task args
2. Check whether is_scoring -- otherwise we're doing a prediction
3. If predicting AND we have task args with an ingestion AND our submission is code:
   • we need to run something along side the regular submission
4. If submission is results, not code, just zip it up as a result and submit that (can explain more)
5. Run submission AND ingestion at the same time, ingestion should receive the hidden_directory of data to work with.