Related to issue #1451: Catch non-existant object error and clear object attribute in DB accordingly
requested to merge github/fork/OhMaley/bugfix/storage/monitoring-tool/handle-inexistant-object into bugfix/storage/monitoring-tool/handle-inexistant-object
A brief description of the purpose of the changes contained in this PR.
This piece of code avoids crash when trying to access the size of a file that does not exist anymore in the storage. If an error is caught it then clears the file_size and data_file attribute in the DB.
Issues this PR resolves
Should solve Problem 1 of issue #1451 (closed)
Simple test
A simple test would be to:
- create a private competition
- upload a submission
- change in the database (manually or by using the django admin interface) the
file_size
attribute to -1 of one of the submission's dataset (table datasets_data) - remove in the storage (by using the minio interface) the corresponding file (its path is the value of the column
data_file
of the row in which you changed thefile_size
attribute) - Open up the shell_plus console inside the container of django
- Call the
save()
method of the dataset object you modified. It will do some checks on the existence of the file in the storage and do the necessary updates. It should be something in those lines:
docker-compose exec django ./manage.py shell_plus
>>> d = Data.objects.get(id=<your_dataset_id>)
>>> d.save()
- Check that their is no crash and that some changes occured in the DB
Use case test
- Remove a file from the storage
- Wait for the next create_storage_analytics_snapshot task to be ran (or trigger it manually)
- Check that everything worked correctly
Checklist
-
Code review by me -
Hand tested by me (on a local setup) -
I'm proud of my work -
Code review by reviewer