I like writing functional/integration tests because the GitLab API keeps changing so it's the only way to really check the responses are what is assumed in unit test fixtures. But the custom bash scripting in the environment was kind of unreliable with the REUSE_CONTAINER option, the custom venv creation, hardcoded temp dirs (I'm sometimes on a Windows machine 😢 ). It was also not a consistent experience compared to unit tests, so might be why people don't usually add them.

I saw that gitbeaker was doing something similar so I'm hoping this is a stable approach.

The idea was to:

• have all virtualenvs fully managed by tox for consistency
• fully cross-platform I hope (even tested on Windows cmd 😁 )
• use native docker-compose, making it easy to add real integration tests with other services like gitlab-runner, LDAP etc.
• create a static token directly with the rails runner, so no need for requests-html dependency and generate_token.py
• remove all unneeded/wrapper bash scripts when it's all put together, use pytest libraries that can be more robust instead
• easier to rewrite python_test_v4.py into real test cases that run independently (though it will make the runs longer I assume)
• can use pytest's tmpdir fixtures later which might be more reliable than /tmp

Cons/TODOs before merge:

• performance - not sure if it's just me but seems like the container takes a bit longer to spin up.
• more pytest dependencies to install - I've added a separate docker-requirements.txt to avoid polluting other environments, does that make sense?
• still need to implement a "reuse existing container" option upstream on pytest-docker. I have something working locally with pytest_addoption and there is some work already upstream in Provide fixture for docker cleanup command avast/pytest-docker#51

It's also possible this is a bit overkill 😁 but would love any feedback on this if anyone else was also having issues with the bash scripts before.