Feature/storage/monitoring tool
Description
This PR aims to offer a monitoring tool similar to the one available in Codalab. The following features have been developed:
- The monitoring tool is only accessible by admins
- It is accessible under the user menu -> analytics -> storage tab
- 3 metrics are available
- Usage history
- Competitions usage history and details
- Users usage history and details
- Metrics can be download using the download buttons
- The metrics only displays what has been computed by the server beforehand
- 2 background tasks has been coded:
- a reset task that get executed every 3 month (based on the value that is set in the settings/base.py). It reset the value of all computed file sizes so they will be re-computed again without any shifting on the next run of the storage analytics task
- a storage analytic snapshot task that get executed every week (based on the value that is set in the settings/base.py). It measures all files with unset size. Then it measures the storage usage per category (competition, user or admin) and per day. It then check for database <-> storage inconsistency (and created a log file in the logs/ folder). And save everything into the database so it becomes query-able by the front-end.
Issues this PR resolves
Checklist
-
Code review by me -
Hand tested by me -
I'm proud of my work -
Code review by reviewer -
Hand tested by reviewer -
CircleCi tests are passing -
Ready to merge