Skip to content

Changing rabbit connection timeout

Placeholder github Source User requested to merge changing-rabbit-connection-timeout into develop

@Didayolo ...

Changing the default worker connection timeout of RabbitMQ

Since the version 3.8, RabbitMQ added consumer_timeout as 1800 s by default. It means that if the compute worker doesn't answer RabbitMQ server after 30 minutes, RabbitMQ closes the connection. Therefore, if we have a submission running for more than 30 minutes, RabbitMQ suggests that the compute worker disappeared and distributes this submission to another compute worker. It ends up with all compute worker having submission running and blocking.

Issues this PR resolves

Render this parameter as environment variable and fix it to 100000000 milliseconds, more 27h.

Checklist

  • Code review by me
  • Hand tested by me
  • I'm proud of my work
  • Code review by reviewer
  • Hand tested by reviewer
  • CircleCi tests are passing
  • Ready to merge

Merge request reports