Weblate testsuite and continuous integration

Testsuites exist for most of the current code, increase coverage by adding testcases for any new functionality, and verify that it works.

Continuous integration

Weblate relies on GitHub Actions to run tests, build documentation, code linting, and other tasks to ensure code quality.

Codecov collects the code coverage information from the tests that were run.

There are several jobs to verify different aspects:

  • Unit and functional tests using pytest.

  • Documentation build and external links using Sphinx.

  • Code linting and quality assurance using ruff and pylint.

  • Code security scanning using CodeQL.

  • Visual changes testing is utilizing Argos CI.

  • Code formatting using prek, a faster third-party reimplementation of the pre-commit framework.

  • Migration testing from all supported releases

  • Setup verification (ensures that generated dist files do not miss anything and can be tested)

The configuration for the CI is in .github/workflows directory. It heavily uses helper scripts stored in ci directory. The scripts can be also executed manually, but they require several environment variables, mostly defining Django settings file to use and test database connection. The example definition of that is in scripts/test-database.sh:

The Selenium screenshot tests in weblate/trans/tests/test_selenium.py serve two purposes. They generate images for visual change testing in CI, and the same images are converted into documentation screenshots by make -C docs update-screenshots. Keep screenshot fixtures close to real rendered pages so CI catches UI regressions. When a screenshot includes volatile runtime data, prefer deterministic server-side test inputs over post-render DOM changes.

GitHub repository URLs in these screenshots use local Git transport rewrites. The shared Git fixture archive contains the ordinary test data and a separate, pinned demo revision, so repository discovery shows only demo content. Regenerate the archive with scripts/pack-test-data.sh; update its DEMO_REVISION when intentionally refreshing the demo data. Runtime tests do not fetch GitHub repositories or resolve their hostname. The TEST_REVISION pin controls the ordinary Git, Mercurial, and Subversion data independently.

# Copyright © Michal Čihař <michal@weblate.org>
#
# SPDX-License-Identifier: GPL-3.0-or-later

# Simple way to configure test database from environment

# shellcheck shell=sh

# Database server configuration
export CI_DB_USER=weblate
export CI_DB_PASSWORD=weblate
export CI_DB_HOST=127.0.0.1

# Django settings module to use
export DJANGO_SETTINGS_MODULE=weblate.settings_test

The simple execution can look like:

source scripts/test-database.sh
./ci/run-migrate
./ci/run-test
./ci/run-docs

Local testing of Weblate

For an isolated environment with dependencies and test services already configured, use the development container. Its CLI can run the same pytest and lint commands documented below in each Git worktree.

Before running tests, please ensure development dependencies are installed:

uv sync --all-extras --dev

Testing using pytest

Prior to running tests you should collect static files as some tests rely on them being present:

DJANGO_SETTINGS_MODULE=weblate.settings_test uv run ./manage.py collectstatic --noinput

You can use pytest to run the test suite locally:

uv run pytest

Property-based tests using hypothesis run as part of the normal pytest suite. They check Markdown output safety, backup archive member paths, and SSH URL parsing with bounded generated inputs. Each property runs up to 100 generated examples deterministically, alongside explicit regression examples. Unexpected exceptions fail the test, and Hypothesis minimizes failing inputs. Add minimized inputs as explicit regression examples when fixing a failure. Timing deadlines are disabled to avoid failures caused by CI load.

Running an individual test file:

uv run pytest weblate/utils/tests/test_search.py

Hint

You will need a database (PostgreSQL) server to be used for tests. By default Django creates separate database to run tests with test_ prefix, so in case your settings is configured to use weblate, the tests will use test_weblate database. See Database setup for Weblate for setup instructions.

The weblate/settings_test.py is used in CI environment as well (see Continuous integration) and can be tuned using environment variables:

export CI_DB_USER=weblate
export CI_DB_PASSWORD=weblate
export CI_DB_HOST=127.0.0.1
export CI_DB_PORT=60000
export DJANGO_SETTINGS_MODULE=weblate.settings_test

Hint

The tests can also be executed inside developer docker container, see Running Weblate locally in Docker.

See also

See Testing in Django for more info on running and writing tests for Django.

Tests that repeatedly import the same component can inherit from ReusableComponentTestCase or ReusableViewTestCase in weblate/trans/tests/test_views.py. These bases build the initial user, project, component, and translations once per class in setUpTestData(). Override create_component() to select a repository fixture or file format, as with ComponentTestCase. For additional shared setup, override the build_fixture(cls, builder) class method, call super().build_fixture(builder), and use the builder’s repository helpers and model instances.

Each test reloads its model instances and receives a restored copy of the working repositories, upstream repositories used during construction, and test Git configuration. Database changes are rolled back by Django. Clients, authentication, and application caches are initialized per test. Snapshots are temporary and local to each class and worker; extra components created within a test still use the normal component factories.

Use the existing bases for TransactionTestCase and concurrency tests, or when component construction itself must run under a method-specific settings override. The reusable bases do not replay component construction signals or commit callbacks before each test.

Local testing of Weblate modules

The tests are executed using pytest. First you need to install development dependencies:

uv sync --all-extras --dev

You can then execute the testsuite in the repository checkout:

uv run pytest

Testing repository

Many of the tests in the Weblate test suite use the test repository. The test suite repository is maintained at https://github.com/WeblateOrg/test. The script scripts/pack-test-data.sh is then used to generate a tarball with a repository for each of the supported version control systems. These are stored as weblate/trans/tests/data/test-base-repo.git.tar, weblate/trans/tests/data/test-base-repo.hg.tar, and weblate/trans/tests/data/test-base-repo.svn.tar in the Weblate repository.

The https://github.com/WeblateOrg/test repository is tagged at the release time, which ensures that the release tags can be used to access test data used at the release time. The script tries to create reproducible tarballs as much as possible.