pg_dump.py env references and repo name for paths
@ mention of reviewers`
@Didayolo`
A brief description of the purpose of the changes contained in this PR.
Attempting to make postgres db backups and upload to MINIO storage. Generally everything works, but the names of variables in files and the names of containers\services\folders changed as we renamed the repo to "codabench" from "competitions-v2".
Issues this PR resolves
None.
A checklist for hand testing:
First Set of Tests:
-
Create Dump following instructions on edited Creating and Restoring from Backup
DB_NAME=...
DB_USERNAME=...
DB_PASSWORD=...
DUMP_NAME=test_dump.dump
docker exec codabench-db-1 bash -c "PGPASSWORD=$DB_PASSWORD pg_dump -Fc -U $DB_USERNAME $DB_NAME > /app/backups/$DUMP_NAME.dump"
-
Shell into django container docker-compose exec django bash
-
Execute command to upload dump:
root@9270c8888b98:/app# python manage.py upload_backup test_dump.dump
Uploading backup '/app/backups/test_dump.dump' to 'https://XXXXXXXXXXXXXXXXXX/codabench-private/backups/test_dump.dump?AWSAccessKeyId=XXXXXXXXXXXXXXXX&Signature=XXXXXXXXXXXXXXXXXXX&Expires=1664403807'
Success!
Removing local dump file '/app/backups/test_dump.dump'
Second Set of Tests (Automated):
-
cd to codabench/bin and run pg_dump.py file:
$ bin/pg_dump.py
-
Check that an automatically generated dump is made and moved to storage and named as "%Y-%m-%d_%H-%M-%S.dump": -
Following along with Creating and Restoring from Backup we can restore from backup.
container$ dropdb $DB_NAME -U $DB_USERNAME
container$ createdb $DB_NAME -U $DB_USERNAME
container$ pg_restore -U $DB_USERNAME -d $DB_NAME -1 /app/backups/<filename>.dump
-
Restart django and compute_worker containers as they seem to get confused.
docker-compose restart django
docker-compose restart compute_worker
Any relevant files for testing
I edited this file to go along with the checklist for testing and to keep docs updated. https://github.com/codalab/codabench/wiki/Creating-and-Restoring-from-Backup
Misc. comments
My changes to https://github.com/codalab/codabench/wiki/Creating-and-Restoring-from-Backup are not permanent and there are versions we can go back to. I plan to live demo this to you all.
Checklist
-
Code review by me -
Hand tested by me -
I'm proud of my work -
Code review by reviewer -
Hand tested by reviewer -
Ready to merge