This is a unittest framework for Sublime Text. It runs unittest testcases on local machines and CI services such as Travis CI, Circle CI and AppVeyor. It also supports testing syntax_test files for the new sublime-syntax format.

It is too much work to maintain supports for circleci, travis, appveyor and github actions at the same time. As most users host their projects on github, we are going to only support github actions and deprecate supports for circleci, travis and appveyor.

There is no plan to remove the corresponding scripts from the repo in a near future, but they are not maintained any more and may be removed if they are broken.

Sublime Text 4 is now supported. However test coverage on Python 3.8 packages is still not working now.

1. Download & Install Sublime Text 3 (https://www.sublimetext.com/3)
2. Go to the menu Tools -> Install Package Control, then, wait few seconds until the installation finishes up
3. Now, Go to the menu Preferences -> Package Control
4. Type Add Channel on the opened quick panel and press Enter
5. Then, input the following address and press Enter

   https://raw.githubusercontent.com/evandrocoan/StudioChannel/master/channel.json
   

6. Go to the menu Tools -> Command Palette... (Ctrl+Shift+P)
7. Type Preferences: Package Control Settings – User on the opened quick panel and press Enter
8. Then, find the following setting on your Package Control.sublime-settings file:

   "channels":
   [
       "https://packagecontrol.io/channel_v3.json",
       "https://raw.githubusercontent.com/evandrocoan/StudioChannel/master/channel.json",
   ],

9. And, change it to the following, i.e., put the https://raw.githubusercontent... line as first:

   "channels":
   [
       "https://raw.githubusercontent.com/evandrocoan/StudioChannel/master/channel.json",
       "https://packagecontrol.io/channel_v3.json",
   ],

   • The https://raw.githubusercontent... line must to be added before the https://packagecontrol.io... one, otherwise, you will not install this forked version of the package, but the original available on the Package Control default channel https://packagecontrol.io...
10. Now, go to the menu Preferences -> Package Control
11. Type Install Package on the opened quick panel and press Enter
12. Then, search for UnitTesting and press Enter

See also:

1. ITE - Integrated Toolset Environment
2. Package control docs for details.

1. Before testing anything, you have to install UnitTesting via Package Control.
2. Your package!
3. TestCases should be placed in test*.py under the directory tests (configurable, see below). The testcases are then loaded by TestLoader.discover.

Here are some small examples

UnitTesting can be triggered via the command palette command UnitTesting. Enter the package name in the input panel and hit enter, a console should pop up and the tests should be running. To run only tests in particular files, enter <Package name>:<filename>. <filename> should be a unix shell wildcard to match the file names, <Package name>:test*.py is used in default.

You could run the command UnitTesting: Test Current Package to run the current package. The current package will be first reloaded by UnitTesting and then the tests will be executed.

It is also possible to generate test coverage report via coverage by using the command UnitTesting: Test Current Package with Coverage. The file .coveragerc is used to control the coverage configurations. If it is missing, UnitTesting will ignore the tests directory.

These environmental variables are used

• PACKAGE: the package name, it is needed if the repo name is different from the package name.
• SUBLIME_TEXT_VERSION: 3 or 4
• SUBLIME_TEXT_ARCH: x32 (Sublime Text 3 only) or x64
• UNITTESTING_TAG: a specific version of UnitTesting to use

To enable GitHub Actions, copy the file build.yml to your repository.

We support Codecov, Coveralls and Codacy. Codecov is recommended as it supports merging reports from different CIs.

To submit coverage report to codecov.io:

1. install codecov
2. run codecov after success

For GitHub Actions, copy the CODECOV_TOKEN from codecov.io to GitHub's Secret tab.

To submit coverage report to coveralls.io:

1. install python-coveralls
2. run coveralls after success

To submit coverage report to codacy.com:

1. install both coverage and codacy-coverage

   pip install coverage codacy-coverage
   

2. generate the xml report: coverage xml -o coverage.xml

3. run python-codacy-coverage

If your package uses Package Control dependencies, you may want to install Package Control by uncommenting the line of install_package_control in Travis CI and AppVeyor configuration files.

Check this for an example.

The default test directory is "tests". To change the test directory, add a file unittesting.json to your repo with the corresponding directory name, eg unittest:

    "tests_dir" : "unittest"

The test result could be redirected to a file by specifying the output variable in unittesting.json.

    "output" : "foo.txt"

Tests can be written using the Deferrable testcase, such that you are able to run sublime commands from your test cases and yield control to sublime text runtime and continue the execution later. Would be useful to test asynchronous codes.

An example would be found in here.

To activate deferred testing, put the following line in unittesting.json.

    "deferred": true,

PS: this idea was inspired by Plugin UnitTest Harness.

DeferrableTestCase is used to write the test cases. They are executed by the DeferringTextTestRunner and the runner expects not only regular test functions, but also generators. If the test function is a generator, it does the following

• if the yielded object is a callable, the runner will evaluate the callable and check its returned value. If the result is True, the runner continues the generator, if not, the runner will wait until the condition is met.

• If the yielded object is an integer, say x, then it will continue the generator after x ms.

• Otherwise, the yield statement would yeild to any queued jobs.

By default, the tests are running in the main thread and can block the graphic inference. Asychronized testing could be used if you need the interface to respond.

Async tests are usually slower than the sync tests because the interface takes time to respond but it is useful when there are blocking codes in the tests. An example would be found in here.

However, it is known that async test does not work very well with coverage. In general, it is recommended to use deferred testing over async testing since there is no need to worry about race condition.

To activate async testing on Travis CI and AppVeyor, put the following line in unittesting.json.

    "async": true,

Note: if async is true, deferred is forced to be false (relaxation of this is in progress)

It is recommended to add the following in your .sublime-project file so that c+b would invoke the testing action.

"build_systems":
[
  {
    "name": "Test Current Package",
    "target": "unit_testing_current_package",
  }
]

Check https://github.com/SublimeText/UnitTesting/tree/master/docker

Thanks guillermooo and philippotto for their efforts in AppVeyor and Travis CI macOS support.