This is an simple example to use UnitTesting to test a Sublime Text package on a local machine and via continuous integration services such as travis-ci and appveyor. It also provides DeferrableTestCase to allow interacting with the graphic interface and processing GUI events.

For testing syntax_test files, go directly to testing syntax_test files.

1. Before testing anything, you have to install UnitTesting via Package Control or clone it from source.
2. Your package! In our case, it is helloworld.py
3. You also have to know how to write unittest testcases. TestCases should be placed in test*.py under the directory tests (configurable, see below). They are loaded by a modified TestLoader.
   • ST2 developers should read this for unittest documentation.
   • ST3 developers should read this.
   • You may also want to look that the file test.py under the tests directory for the minimal example.

UnitTesting can be triggered via the command palette command UnitTesting. Enter the package name in the input panel and hit enter, a console should pop up and the tests should be running. To run only tests in particular files, enter <Package name>:<filename>. <filename> should be a unix shell wildcard to match the file names, <Package name>:test*.py is used in default.

There are also quick commands to run tests of the current project or file. If PackageReloader is installed, you could run the command UnitTesting: Test Current Project (Reload) to reload and run the current project.

Besides the test results, UnitTesting also provides a command UnitTesting: Test Current Project (Coverage) to check test coverage via coverage (avaiable on Sublime Text 3, Linux/MacOS, if PackageReloader is installed).

If the tests can be run locally, let's put them to travis-ci and let travis-ci takes care of them. First, you have to copy a important file: .travis.yml (caution: with a beginning dot) to your repo. Then change the env variable PACKAGE in .travis.yml to the name of your package. Don't forget to login travis-ci and enable travis-ci for your repo. Finally, push to github and wait..

To enable Appveyor for windows platform tests, copy the file appveyor.yml to your repo, change the PACKAGE variable in appveyor.yml. The last but not least, login appveyor to add your repo as a project.

Installation of Sublime Text on Travis and Appveyor are handled by the scripts in sublime-text- installer.

To generate coverage report for coveralls.io, you just have to specific three things in .travis.yml

1. run the test with the --coverage flag

   sh travis.sh run_tests --coverage
   

2. install python-coveralls

3. run coveralls after success

Check .travis.yml for details. The file .coveragerc is used to control the coverage configuations. If it is missing, UnitTesting would still ignore the tests directory. This feature should be enabled for Sublime Text 3 only.

If your package uses Package Control dependencies, you may want to install Package Control by umcommenting the line of install_package_control in .travis.yml or appveyor.yml.

To enable testing of the syntax_test files, please copy the .travis.yml or appveyor.yml, and use the run_syntax_tests in those files. Check [syntax](https://github.com/randy3k /UnitTesting-example/tree/syntax) branch for an example.

The default test directory is "tests". To change the test directory, add a file unittesting.json to your repo with the corresponding directory name, eg unittest:

{
    "tests_dir" : "unittest"
}

The test result could be redirected to a file by specifying the output variable in unittesting.json.

{
    "output" : "foo.txt"
}

Tests can also be written using the deferrable testcase.

It provides deferred testcases, such you are able to run sublime commands from your test cases and give control to sublime text and get it back later. Would be useful to test sublime_plugin.EventListener. To make deferred tests, you will need DeferrableTestCase.

A example would be found in deferred branch.

To activate deferred testing on travis and appveyor. Add the file unittesting.json to your repo with the following:

{
    "deferred": true,
}

Tests are running in the main thread and blocking the UI. Asychronized testing could be used if you need the UI to respond. Async tests are usually slower than the sync tests because the UI takes time to repond. It is useful when there are non-blocking codes in the tests. A example would be found in async branch. Only consider this option if deferred testing does not work for you.

To activate async testing on travis and appveyor. Add the file unittesting.json to your repo with the following:

{
    "async": true,
}

Note:

1. async is forced to be false on ST2
2. if async is true, deferred is forced to be false.

If you keep encountering unexpected errors, you may want to check the close_windows_when_empty setting. I have spent at least a few hours to realize this was the cause of a error. The true value would close the last window if there is no view. It is fine if you are running blocking code. However, if you are using deferred or async testcases, this is something you definitely want to check.

Debugging in travis-ci could be difficult. To mock the travis-ci environment in your computer, you can use vagrant. For most users, this section could be ignored.

# clone the example to somewhere
git clone https://github.com/randy3k/UnitTesting-example
cd UnitTesting-example
# you can also launch `st2` config
vagrant up st3
# enter ssh
vagrant ssh st3
# run tests
sh vagrant.sh run_tests