CFn: configure wait time for polling resources#13455
Conversation
localstack-core/localstack/config.py
Outdated
| CFN_IGNORE_UNSUPPORTED_RESOURCE_TYPES = is_env_not_false("CFN_IGNORE_UNSUPPORTED_RESOURCE_TYPES") | ||
|
|
||
| # Decrease the waiting time for resource deployment | ||
| CFN_NUM_NO_WAIT: str | int | None = os.environ.get("CFN_NUM_NO_WAIT") |
There was a problem hiding this comment.
I am not happy with this name. Any better ideas?
There was a problem hiding this comment.
Yeah that's better, thanks!
S3 Image Test Results (AMD64 / ARM64) 2 files 2 suites 8m 13s ⏱️ Results for commit ecee01c. ♻️ This comment has been updated with latest results. |
Test Results - Alternative Providers587 tests 331 ✅ 18m 11s ⏱️ Results for commit ecee01c. ♻️ This comment has been updated with latest results. |
Test Results (amd64) - Integration, Bootstrap 5 files 5 suites 2h 33m 43s ⏱️ Results for commit ecee01c. ♻️ This comment has been updated with latest results. |
263dad2 to
c299bf7
Compare
localstack-core/localstack/config.py
Outdated
| CFN_IGNORE_UNSUPPORTED_RESOURCE_TYPES = is_env_not_false("CFN_IGNORE_UNSUPPORTED_RESOURCE_TYPES") | ||
|
|
||
| # Decrease the waiting time for resource deployment | ||
| CFN_NUM_NO_WAIT: str | int | None = os.environ.get("CFN_NUM_NO_WAIT") |
| if current_iteration == 0: | ||
| time.sleep(0) | ||
| # This should have been set up in the provider | ||
| assert isinstance(config.CFN_NUM_NO_WAIT, int) |
There was a problem hiding this comment.
Question: is it necessary to assert this in every iteration?
There was a problem hiding this comment.
That's a fair point, thanks!
c299bf7 to
35bd89f
Compare
Since we perform 5 updates in quick succession, we probably want to use a higher number of minimum iterations
35bd89f to
ecee01c
Compare
Motivation
While deploying our re:Invent demo stack I found it takes ~85 seconds for a warm deploy on my system. This time is dominated by deploying lambda functions which each take 5 seconds.
When we deploy resources, we create a loop where
AWS CloudFormation waits between polling loops (to reduce burden on their networking and services). On LocalStack we don't need to do this - a lot of our resources are in-memory so are instantly available. Therefore we set the sleep time for the first iteration of this loop only to 0 seconds, and wait 5 seconds for every other iteration.
Since the lambda functions take a minimum of 5 seconds each, I suspect the deployment loop is going round more than twice.
Increasing the number of loop iterations that we don't sleep for would reduce this deployment time greatly.
Changes
CFN_NO_WAIT_ITERATIONSto control the number of loop iterations we don't wait 5 seconds between, defaulting to 5 iterationsCFN_PER_RESOURCE_TIMEOUTof e.g. 0!)After these changes, our re:Invent demo stack takes ~30 seconds, leading to a 2.8x total speedup 🎉
Tests
1, and with the new default of5and observe the total time to deploy has decreasedRelated
Resolves UNC-122