Skip to content

Testing

uncoded's test suite combines small tests of each module with tests of each command and tests at the boundary with ty. Each technique answers a different question, and no single number is treated as proof that uncoded is correct.

What the tests establish

Branch coverage

The suite measures branch coverage over src/ and fails below 100%. In CI, the coverage of every test job is combined first, so a branch that only one operating system or Python version takes still counts. This establishes that the suite executes every branch in uncoded's source. It helps expose untested alternatives and dead code. It does not establish that the assertions are sufficient, or that every possible input was tried.

Encoding failures

The interpreter is armed to emit EncodingWarning, and pytest treats that warning as an error. A sentinel test in tests/test_encoding_gate.py fails when the gate is not armed. This catches a file or subprocess operation that the suite executes and that relies on the platform's default text encoding. It says nothing about a line that the suite does not execute.

Commands on a real file system

The tests in tests/test_cli.py run each command through main in a temporary project directory, with real files, and check what the command prints, the files that it writes and its exit status. They hold the behaviour and the exit statuses that the stories describe. They do not run the installed uncoded command.

Modules on their own

Most other test modules each test one module of src/uncoded/, such as the stubs, the docs map or the configuration search. These tests pin down the cases that are slow or awkward to reach through a command. They do not show that the module is wired into a command.

A stand-in for ty

uncoded refs runs the ty language server through uvx. The stand_in_for_ty fixture in tests/conftest.py makes it start a Python script instead, with the same pipes. The script answers deterministically, so the tests can check how uncoded refs frames its messages, reads ty's answers, and reports ty when it fails or exits early. These tests do not show that ty itself still answers as the script does.

Integration tests

The tests marked integration run the real ty language server, at the version that uncoded refs pins, through uvx. They need uvx on PATH, and the first run may download ty. They establish that uncoded refs finds the references that it promises with that version of ty. Run them after any change to the pinned version. The default run leaves them out, because they spawn a process and are slower than the rest.

Hung tests

Every test has a time limit to finish, counting its fixture setup and teardown. timeout in pyproject.toml sets the limit, and CI raises it on the systems that run slower. The limit is there to end a run that has hung, not to measure speed.

When the limit passes, pytest-timeout writes the stack of every thread to the terminal. On Linux and macOS it then fails the test, and the run carries on. On Windows it ends the whole run.

What CI runs

For every push and pull request to main, the CI workflow runs the whole suite, the integration tests included, with the encoding gate on every supported Python version. Linux and Windows run every version on every event. Pushes to main also run every version on macOS, while pull requests run macOS on the oldest and newest versions only. It then combines the coverage of those jobs and checks it against 100%. It runs every pre-commit hook. The hooks include a strict build of the documentation site and the regeneration of the --help blocks in the commands reference. On every supported operating system, it builds the sdist, builds the wheel from the sdist, and runs uncoded --version from the wheel.

For every pull request, the dependency review workflow checks the dependencies that the pull request adds or changes for known vulnerabilities. For every push to main, the docs workflow builds the documentation site and publishes it to GitHub Pages.

Run the tests

The Commands section of the agent guide gives the commands that run the tests.