X Tutup
Skip to content

chore(ci): bring test environment back to life#1269

Merged
max-wittig merged 9 commits intopython-gitlab:masterfrom
nejch:fix/test-env
Jan 28, 2021
Merged

chore(ci): bring test environment back to life#1269
max-wittig merged 9 commits intopython-gitlab:masterfrom
nejch:fix/test-env

Conversation

@nejch
Copy link
Member

@nejch nejch commented Jan 27, 2021 •
edited
Loading

This does a few things to hopefully bring back a workable environment so people's PRs can go green :)

  • at some point docker-compose started using python-dotenv which processes .env files differently (at least for paths to env files) and this broke everything in functional tests. This PR explicitly tracks a version in the tox environment so it works everywhere regardless of the system.
  • replaces Travis with GHA. On .org the Travis job queues were getting into 3+ hours for jobs, the migration to .com seemed like a whole process, and it was getting less and less reliable.
  • I like that Actions have test failure annotations (akin to inline coverage in GitLab) and it was basically zero-config to add it with https://pypi.org/project/pytest-github-actions-annotate-failures. Jobs get queued fast and builds are under 10 min including functional tests and they seem very stable 👍 (I was spamming it while writing the YAML and none of the functional tests failed). test(users): force 10 seconds sleep to improve reliability #1215 or similar will probably still be necessary but this looks promising with GitHub Actions IMO.

Edit: well I take that back about tests being stable haha

@codecov-io
Copy link

codecov-io commented Jan 27, 2021 •
edited
Loading

Codecov Report

Merging #1269 (2de64cf) into master (fac0874) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1269   +/-   ##
=======================================
  Coverage   78.05%   78.05%           
=======================================
  Files          11       11           
  Lines        2866     2866           
=======================================
  Hits         2237     2237           
  Misses        629      629
Flag Coverage Δ
unit 78.05% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fac0874...2de64cf. Read the comment docs.

@nejch nejch requested a review from max-wittig January 27, 2021 01:03
@nejch nejch marked this pull request as draft January 27, 2021 01:45
@max-wittig
Copy link
Member

max-wittig commented Jan 27, 2021

Very very nice. I wanted to switch to Github actions for so long, but didn't do it. Thanks!

max-wittig
max-wittig requested changes Jan 27, 2021
View reviewed changes
Copy link
Member

@max-wittig max-wittig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good. Just one question regarding black.

@max-wittig
Copy link
Member

max-wittig commented Jan 28, 2021

@nejch would this be ready to merge? I would love to rebase #1252 on it.

@nejch
Copy link
Member Author

nejch commented Jan 28, 2021 •
edited
Loading

@nejch would this be ready to merge? I would love to rebase #1252 on it.

In the end I noticed some of the flakiness in functional tests is still there, but if we don't mind hitting retry on those jobs sometimes we can solve it in a follow-up! :) (e.g. #1215) In that case it's ready.

Edit: maybe we need to remove Travis as a required check for this to get merged

@nejch nejch marked this pull request as ready for review January 28, 2021 07:43
@max-wittig max-wittig merged commit fd179d4 into python-gitlab:master Jan 28, 2021
@max-wittig
Copy link
Member

max-wittig commented Jan 28, 2021

I've uninstalled the travis app. I think it should work now.

@nejch
Copy link
Member Author

nejch commented Jan 28, 2021

I've uninstalled the travis app. I think it should work now.

The checks run really fast now with GHA. Nice! 🥳

@max-wittig
Copy link
Member

max-wittig commented Jan 28, 2021

Soooo fast. Woah. And I had to disable branch protection rules for travis as well.

@max-wittig
Copy link
Member

max-wittig commented Jan 29, 2021

The tests are flacky because they are not isolated and rely on order of execution, which is not always the same.

See: test_merge_request.py. We would need many more fixtures. E.g. to create random branches etc.

@max-wittig
Copy link
Member

max-wittig commented Jan 29, 2021

and create new onces everytime @nejch

This was referenced Jan 31, 2021
Merged
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@max-wittig max-wittig max-wittig requested changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nejch @codecov-io @max-wittig
X Tutup