Skip to content

Intermittently failing analytics test fixed.

Placeholder Bailey Lind-Trefts requested to merge #468_fix_intermitently_failing_analytics_test into develop

The setup of analytics tests uses factories to create users, submissions, and competitions. The failing test, test_date_range_is_inclusive_of_bounds, includes an assertion of the number of users that joined within a date range. Occasionally, one of the subfactories inside of submissions or competitions would generate a user whose date_joined field existed within the date range in question, intermittently offsetting the number of users by one more than the expected value, thus triggering test failure.

The fix involves the explicit control of the users attached to those subfactories, so that a new User with a random date_joined field is not created.

@ mention of reviewers`

@ckcollab @dde6khkg`

Issues this PR resolves

#468 (closed)

A checklist for hand testing

I checked before that this test would fail just under 1 in 5 times. I've run it hundreds of times in a row now without a failure. Feel free to do the same!

Here's the script I used to repeatedly run the test

SUCCESSFUL=0
TOTAL=10
for i in $(seq 1 $TOTAL)
do 
    py.test -k test_date_range_is_inclusive_of_bounds && SUCCESSFUL=$((SUCCESSFUL+1))
    echo "successful: $SUCCESSFUL"
done
echo "successful: $SUCCESSFUL"
echo "total: $TOTAL"

Checklist

  • Code review by me
  • Hand tested by me
  • I'm proud of my work
  • Code review by reviewer
  • Hand tested by reviewer
  • Ready to merge

Merge request reports