Tests

Directory structure

• test/unit: Unit tests
  • test/unit/test.vimspec: Unit test cases
  • test/unit/runtime: Runtime directory loaded on running unit tests. They mocks several external APIs like ale#* or lsp#*
• test/integ: Integration tests
  • test/integ/test.vimspec: Integration test cases
  • test/integ/deps: Dependant plugins

Unit tests

Unit tests confirm vim-lsp-ale works as intended.

Prerequisites

Unit tests can be run with no dependency because they mock every external API.

vim-themis is used as test runner.

By default, it runs tests with vim command. When running tests with Neovim, set THEMIS_VIM=nvim environment variable.

Run unit tests

cd path/to/vim-lsp-ale
git clone https://github.com/thinca/vim-themis.git

# Run tests with Vim
./vim-themis/bin/themis ./test/unit/

# Run tests with NeoVim
THEMIS_VIM=nvim ./vim-themis/bin/themis ./test/unit/

Measure unit test coverage

covimerage is used to measure test coverage. Install it by pip install covimerage.

Set a file path to THEMIS_PROFILE environment variable and run unit tests. Vim will store the profile data to the file. covimerage command will convert the profile data into coverage data for coverage command provided by Python.

cd path/to/vim-lsp-ale
git clone https://github.com/thinca/vim-themis.git

# Run test case with $THEMIS_PROFILE environment variable
THEMIS_PROFILE=profile.txt ./vim-themis/bin/themis ./test/unit

# Store coverage data at .coverage_covimerage converted from the profile data
covimerage write_coverage profile.txt

# Show coverage report by `coverage` command
coverage report

Integration tests

Integration tests confirm integrity among vim-lsp, ALE, vim-lsp-ale and a language server.

Prerequisites

Integration tests require all dependencies are installed in deps directory.

cd path/to/vim-lsp-ale
git clone https://github.com/prabirshrestha/vim-lsp.git test/integ/deps/vim-lsp
git clone https://github.com/dense-analysis/ale.git test/integ/deps/ale

rust-analyzer is used as language server to run integration tests. Download the binary following the instruction and put the binary in $PATH directory.

And vim-themis is used as test runner.

Note that integration tests were not confirmed on Windows.

Run integration tests

cd path/to/vim-lsp-ale
git clone https://github.com/thinca/vim-themis.git
./vim-themis/bin/themis ./test/integ/

Log files

When some integration tests fail, the following log files would be useful to investigate the failure.

• test/integ/integ_messages.txt: Messages in :message area while running the tests
• test/integ/lsp-log.txt: Log information of vim-lsp. It records communications between vim-lsp and a language server

CI

Tests are run continuously on GitHub Actions.

https://github.com/rhysd/vim-lsp-ale/actions?query=workflow%3ACI

• Unit tests are run on Linux, macOS and Windows with Vim and Neovim
• Integration tests are run on Linux with Vim and Neovim