V1.5 compatibility - Unsupported features
This issue regroups several reported issue of compatibility and missing features with V1.5 (CodaLab Competitions).
My submissions
-
Show the submission score Solved by: #877 -
Submissions left: #1015 Solved by: #1094
Leaderboard
Set number of digits/decimals shown
-
Leaderboard: limit/specify significant digits from leaderboard's level, For exmaple in V1, https://github.com/codalab/codalab-competitions/wiki/Organizer_Leaderboard, numeric_format is used Solved by: #834
Computation columns
Linked to #406 and #851
I have a V1/V2 challenge with multiple tasks. The final rank of a participant is computed regarding its average score on all tasks.
It works on Codalab V1:
But the exact same bundle and submission on Codalab V2 gives:
Here is the YAML configuration of the leaderboard:
leaderboard:
columns:
auc_classification:
label: Classification (AUC ROC)
leaderboard: &id001
label: RESULTS
rank: 1
rank: 2
sort: desc
acc_selection:
label: Feature Selection (accuracy)
leaderboard: *id001
rank: 3
sort: desc
acc_causality:
label: Causal Inference (accuracy)
leaderboard: *id001
rank: 4
sort: desc
ave_score:
label: < Rank >
leaderboard: *id001
rank: 1
sort: asc
computed:
operation: Avg
fields: auc_classification, acc_selection, acc_causality
leaderboards:
RESULTS: *id001
Download buttons, HTML files
Related:
-
#764 Solved by: #944 #970
Public data and starting kit
-
#277 (closed) (taken care by #764) Solved by: #970
In V1, after uploading the bundle, it automatically adds "Starting Kit" and "Public data" to the Files tab:
With the exact same bundle in V2, it doesn't:
HTML documentation not interpreted
This issue is not specifically related to v1 bundles, it also happen with v2 bundle. However we can consider it a v1 compatibility issue because all v1 bundles use HTML, while you can use Markdown in v2 bundles.
-
#836 Partially solved by #878 Fully solved by #1090