Skip to content

Measurement

This page holds the latest measurement of uncoded against the standard. Each measurement replaces the one before it, and earlier measurements are in Git history.

This is the second measurement. It was taken on 2026-10-28 at 1c0b9ba, on the branch of #355, which ran the release checklist for the first time, after the fixes that the checklist calls quick. The first measurement, the baseline, was taken on 2026-10-27 at 0071673, before the roadmap for the engineering standard (#257) closed any gap.

It was updated for the 4.0.0 release on 2026-10-28, for the pull requests merged after 1c0b9ba: #362, #364, #366, #367 and #368. Only "True documents", "Machine-checked rules" and "Fast readable tests" change. The other criteria stand as measured at 1c0b9ba.

It was updated again for the 5.0.0 release on 2026-10-29, at 38a78f9, for the 28 pull requests merged after v4.0.0. The consistency review was run again in full. Each criterion that those pull requests touched was measured again; the others stand as measured at 1c0b9ba.

It was updated again for the 6.0.0 release on 2026-10-29, at 9c6db7e, for the 4 pull requests merged after v5.0.0. The consistency review was run again in full, and each criterion that those pull requests touched was measured again. The notes below record the measurement at 1c0b9ba, except where "Update for 5.0.0" or "Update for 6.0.0" says otherwise.

An open issue owns each shortfall below that a change to the repository can fix.

Results

Criterion Measured Status Owner
Cohesive modules 0 modules without one nameable responsibility; the largest are cli.py (440 lines) and refs.py (292) met
Focused functions 0 functions without one nameable job; the longest is cli.main (103 lines) met
No unused exports 1 public name that no other module imports, besides main: docs_map.DocFile not met #411
One way 0 facts derived twice met
No suppressions 0 noqa, 0 type: ignore, 0 pragma: no cover; ruff's PGH checks on met
One vocabulary 5 disagreements, all names in the code's model of symbols and files; 0 replaced words not met #411
True documents 0 open bugs contradict a story or the concepts page; 4 claims in the docs corrected in the release met
Docs by purpose Tutorial, how-to pages, references and development pages are kept apart; no user page links to a development page met
Versioned contract A compatibility page and a changelog; each Breaking entry says what changes for the user met
Accurate references --help blocks generated by cog; 0 behaviours that a reference or skill describes differently from the code met
Machine-checked rules 7 of the 7 rules checked; no rule found that review missed more than once met
Fast readable tests 460 tests, the default 446 passed and 1 skipped in 2.17 seconds at 100% branch coverage; 1 test name falls short of the bar not met #411
Accurate tracker 0 of 3 open issues use a replaced word or ask for work done or ruled out met
Reviewed changes 4 pull requests merged since v5.0.0; 3 link an issue or the upstream change, and all 4 answer the definition of done met
Loud failures strict = true, pytest>=9.0, a 30-second test timeout, raised to 120 seconds on macOS and Windows in CI met
Rules not cases 12 entries on the ledger, each with an external cause; 0 conditionals that name a situation off the ledger met
Invariants by construction 0 branches that defend against a state the types could rule out met
Few concepts 35 terms on the concepts page; "project" and "configuration file" narrowed, none added met
Symmetry 0 asymmetries between the code and documentation index paths, or between uncoded body and uncoded refs met

Notes

Update for 6.0.0

  • Cohesive modules, focused functions. #415 grew cli.py to 440 lines and cli._init to 52, which still reads as one job: append the configuration table if there is none, then sync. cli.main, at 103 lines, is still the longest function.
  • No unused exports. docs_map.DocFile is used only by docs_map.py and its tests. It predates this release, and the earlier measurements missed it. #411 owns it, with the rename it already asks for.
  • One vocabulary. The consistency review found 9 disagreements. The release fixed 3, counted under "True documents". #411 owns the other 6: the 4 that the 5.0.0 review found, which all still stand, and 2 new ones. SkippedFile.path and rel_path name the same path, and TypeAliasSymbol is a type statement. One of the 6 is a test name, counted under "Fast readable tests".
  • True documents. The release corrected the release checklist, which called the CI's Python list the one record of the versions uncoded supports, though requires-python and the docs state the floor. It added the kind of project that uncoded supports, and the file that holds its settings, to the contract, which did not name them though 6.0.0 changes both. It said "setting" for "configuration key", and corrected the setup story, which told the developer to write a configuration file by hand.
  • Fast readable tests. #415 removed the tests of .uncoded.toml. The 13 integration tests passed in 0.80 seconds.
  • Reviewed changes. v5.0.0 is the previous major or minor release. #417 links the upstream rename in dreamcatcher, and #413, which upgraded this repository to 5.0.0, has no issue to link.
  • Rules not cases. #416 removed the ledger's entry for a SyntaxError that names its file in its message, since uncoded no longer gives tokenize.detect_encoding a file name.

Update for 5.0.0

  • One way. #404 made failure_reason the one home of why a file cannot be used, so uncoded sync and uncoded body describe a parse failure alike. #409 owns a case in which it still names the file twice.
  • One vocabulary. The consistency review found 6 disagreements. The release fixed 3 claims in the docs and a skill, counted under "True documents". The other 3 are names in the code, which #411 owns: the Symbol alias leaves out properties, stubs._render_assignment also renders type statements, and DocFile names a Markdown file.
  • True documents. #317, #330, #332, #351 and #356 are fixed. The release corrected the symbol kinds that the code navigation skill and AGENTS.md list, where the namespace map lists a redefined name, and an example in the ledger's introduction.
  • Fast readable tests. #359 renamed the 17 names, and #340 removed the dispatch tests. Of the names added since, test_definition_name_follows_token names a mechanism, in two files; #411 owns it. The 13 integration tests passed in 5.78 seconds.
  • Reviewed changes. v4.0.0 is the previous major or minor release. The two pull requests that link no issue, #392 and #394, change the triage skill, for which there is no issue to link.
  • Rules not cases. #366 and #404 each added an entry to the ledger: the block that a type statement shares, and a SyntaxError that names its file in its message. #409 would remove the second.
  • Symmetry. #358 gave the two index paths one walk over their roots and one helper that writes or removes a generated file, named the two maps' functions alike, and made uncoded body and uncoded refs check their input in one order.

Measurement at 1c0b9ba

  • Cohesive modules, focused functions. Review read every module under src, largest first, and named one responsibility for each: cli.py defines and runs the commands, config.py finds, reads and checks the configuration file, refs.py finds references through ty, extract.py lists the symbols of Python files, docs_map.py builds the docs map, stubs.py renders and syncs the stubs, skill.py writes the skills, resolver.py finds a symbol by its name path, sync.py writes and removes generated files, ast_helpers.py decides which statements are symbols, namespace_map.py builds the namespace map, read_helpers.py reads source and says why a file could not be used, yaml_tree.py nests paths and renders the maps, body.py slices out a definition, and markers.py holds the provenance marker. The longest functions each do one job:
Function Lines Its job
cli.main 101 Parse the command line and run the chosen command.
refs._run_exchange 81 Run one exchange with ty's language server.
cli._sync 68 Run uncoded sync or uncoded check.
docs_map._extract_headings 40 List a Markdown file's headings.
stubs.sync_stubs 39 Write the stubs for one source root.
cli._init 39 Set uncoded up in a project, then sync.
cli._sync_code_index 37 Write or remove the code index.
extract.read_python_files 36 Read and list the symbols of one root's files.
sync.sync_file 33 Write a generated file when its content differs.
sync.remove_file 33 Remove a generated file and the parents left empty.

cli.main grew from 78 lines, because each command now binds how it finds the configuration and the action it runs. It still reads as one sequence: build the grammar, parse, run the command, report errors.

  • One way. Both facts that the baseline found derived more than once are now derived once. Config.project_directory is the one project directory, and ast_helpers.module_symbols and class_symbols decide which statements are symbols for extract.py and resolver.py, with extract.PythonFile the one model of a Python file's symbols. One new case: uncoded sync says SyntaxError at line 1: invalid syntax for a file that uncoded body reports as invalid syntax (b.py, line 1), because extract.read_python_files builds its own message instead of using read_helpers.failure_reason (#357). The references that refs.py reads from ty and the ones it prints are two types, but one is ty's format and the other uncoded's, converted at the edge, so they are not counted.
  • No suppressions. The subprocess waivers are per-file ignores in pyproject.toml, each with its reason, and ruff's PGH checks reject a blanket noqa or type: ignore.
  • One vocabulary. The consistency review reported 6 disagreements. #355 fixed 4: AGENTS.md promised "full signatures" in a stub, the docstring of cli.main said uncoded init syncs only the index, cli.main bound the function that finds the configuration as find_project, and the code navigation skill, docs/commands.md and a docstring in resolver.py said "type alias" for a type statement. The other 2 are in src/uncoded/extract.py, and #347 owns both: a property is modelled as an AssignmentSymbol in a class's attributes, a word the concepts page does not use, and PythonFile.assignments also holds the type statements. No word that the concepts page replaces is in use outside this page, which quotes the baseline's words. The --help text and the package's description say "AI coding agents", which the concepts page allows, since each introduces the term.
  • True documents. Every behaviour change that the design's Stories section lists is built. Four open bugs contradict a story or the concepts page: #351, where uncoded refs finds nothing for a def with two spaces before its name; #332, where a file under two overlapping source roots is warned about and counted twice; #317, where uncoded refs drops the host of a UNC path on Windows; and #356, where the docs map lists a heading inside a code block that a shorter fence seems to close. The message for a configuration file that names no roots does not name the file, as its story says (#330). #367 fixed #343, where a stub did not show which of two definitions of a name comes later, and #368 fixed #345, a traceback when standard output could not encode a definition. #355 corrected the standard, which said that only refs.py imports subprocess, when tests/conftest.py and tests/test_refs.py do too. The architecture contradicts nothing.
  • Docs by purpose. "Get started" is the tutorial, "Use uncoded" holds the how-to pages, "Reference" holds the configuration, command, compatibility, upgrading and changelog pages, and "Development" holds the rest. The changelog linked the concepts page three times, and #355 removed those links.
  • Versioned contract. docs/compatibility.md gives the rules, and docs/changelog.md uses the headings Breaking, Added, Changed and Fixed. The two Breaking entries under "Unreleased" each say what now happens, and the release that takes them will add an upgrading section, as the release checklist says.
  • Accurate references. cog --check passes on docs/commands.md, and every exit status it gives matched the commands' behaviour. #355 fixed two descriptions that differed from the code: the documentation navigation skill and the docs map's header said that a " (N)" suffix marks a heading with the same text and level, when the level does not count, and docs/configuration.md said that a file is generated when a setting is set, when the setting must name a root. The code navigation skill gives uvx uncoded body and uvx uncoded refs as the commands. In a project with a required version, uncoded refuses them at any other version and gives the command to use, as the stories say, so they are not counted.
  • Machine-checked rules. Each rule that the standard lists has its check: ruff's subprocess rules with their per-file waivers, the writes-only-in-sync hook, the import-linter layers contract, tests/test_markers.py, the stub-style hook that #366 added, PLW1514 with the runtime gate, and PLR0917 with max-positional-args = 0. One rule to watch: a test that spawns ty must carry the integration marker, which review has missed once (#336).
  • Fast readable tests. The time is the default run's, as pytest reports it. The 8 integration tests passed in 0.43 seconds. Review read the 361 distinct test function names against the bar that #310 settled, and 17 fall short: #359 lists 16, and #340 removes the other. #346 kept three names of steps of ty's protocol, which are not counted. The four test functions that #366, #367 and #368 added meet the bar.
  • Accurate tracker. The triage corrected the words that #348 listed in #340, #332, #330, #278, #257, #235 and #162, along with code names that have changed since, and brought #347 up to date with the code. It closed #162, #226, #229, #278 and #348, each with its reason. A search of every open issue's title and body finds no word that the concepts page replaces.
  • Reviewed changes. v3.0.0 is the previous major or minor release. Of the 41 pull requests merged since, #252 links no issue or spec, and 8 answer the definition of done. Four commits reached main outside a pull request, the last on 2026-10-24, before branch protection required one. AGENTS.md asks every pull request to answer the definition of done from #355 on.
  • Rules not cases. Every entry on the ledger is where it says, with a cause outside uncoded, and review found no other conditional that names a situation. #355 removed an override in yaml_tree.py that indented lists, since neither map holds a list.
  • Invariants by construction. No branch, raise or assert defends against a state the types could rule out. sync.remove_file takes a path relative to the project directory as a Path, and a path outside it raises the ValueError of Path.relative_to, which a test pins, but no branch of uncoded's own defends against it.
  • Symmetry. uncoded body and uncoded refs share one list of the errors they report, which settled #205. Four asymmetries remain, which #358 lists: extract.read_python_files and docs_map.read_doc_files walk a root the same way without sharing code, cli._sync_code_index and _sync_documentation_index repeat one sequence, the names of the two maps' functions are not parallel, and uncoded body and uncoded refs check their input in a different order. The two navigation skills have the same steps in the same order.

Method

Each number above was taken this way, from the project directory of a checkout with the development dependencies installed.

  • Line counts come from wc -l src/uncoded/*.py. Function lengths and public names that no other module imports come from the script below, run with uv run python. Its counts are prompts for review, which judges each against the bar.
import ast
from pathlib import Path

trees = {
    p.stem: ast.parse(p.read_text(encoding="utf-8"))
    for p in Path("src/uncoded").glob("*.py")
}
imported = {
    (node.module.removeprefix("uncoded."), alias.name)
    for tree in trees.values()
    for node in ast.walk(tree)
    if isinstance(node, ast.ImportFrom) and node.module
    for alias in node.names
}
functions, unimported = [], []
for module, tree in trees.items():
    for node in ast.walk(tree):
        if isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef):
            start = min([node.lineno] + [d.lineno for d in node.decorator_list])
            functions.append((node.end_lineno - start + 1, f"{module}.{node.name}"))
    for node in tree.body:
        names = [node.name] if isinstance(node, ast.FunctionDef | ast.ClassDef) else []
        if isinstance(node, ast.Assign | ast.AnnAssign):
            targets = node.targets if isinstance(node, ast.Assign) else [node.target]
            names = [t.id for t in targets if isinstance(t, ast.Name)]
        unimported += [
            f"{module}.{name}"
            for name in names
            if not name.startswith("_") and (module, name) not in imported
        ]
print("Longest functions:", *sorted(functions, reverse=True)[:10], sep="\n  ")
print("Public names no other module imports:", *sorted(unimported), sep="\n  ")
  • Suppressions come from grep -rn "noqa\|type: ignore\|pragma: no cover" src tests.
  • Positional parameters come from uv run ruff check --preview --select PLR0917 --config 'lint.pylint.max-positional-args=0' --statistics src, and the same command over tests.
  • Writes and removals outside sync.py come from grep -nE "\.(write_text|write_bytes|unlink|rmdir|mkdir|rename|replace|touch)\(|shutil" src/uncoded/*.py.
  • Imports between modules come from grep -n "^from uncoded" src/uncoded/*.py, read against the layers in the design.
  • Test time and coverage come from PYTHONWARNDEFAULTENCODING=1 uv run pytest -q, and the integration tests' from PYTHONWARNDEFAULTENCODING=1 uv run pytest -q -m integration --no-cov. The distinct test names come from uv run pytest --collect-only -q -m "", with the test class and parameters stripped.
  • Vocabulary comes from the consistency review, run by an agent following the uncoded-consistency-review skill over src, tests, README.md, AGENTS.md, docs and the skill sources, with the concepts page as the authority, and the specifications and generated files left out. The review counts one disagreement for each pair of conflicting claims, however many places repeat a claim. Replaced words come from a case-insensitive search of the files that Git tracks in the same places, apart from the concepts page, for each word after "Replaces:" on the concepts page, in prose and in identifiers such as project_root, with a phrase allowed to break across lines. A use of "body" or "source" is counted only when it means a whole definition or a skill source, which review decides, and the command name uncoded body is not counted.
  • The tracker comes from gh issue list --state open, each title and body searched for the replaced words, plurals included, and read against the code and the stories.
  • Reviewed changes come from git log --first-parent v3.0.0..origin/main and from the bodies of the pull requests merged since v3.0.0, read through gh pr list --state merged. A pull request answers the definition of done when its body has a "Definition of done" heading and every question's title in bold. A commit outside a pull request is one that git log --first-parent --no-merges v3.0.0..origin/main lists.
  • Terms on the concepts page are its ### headings under "Terms".
  • Docs by purpose comes from the nav in mkdocs.yml, and from grep -nE "\]\((concepts|stories|architecture|standard|measurement|testing|special-cases|releasing)\.md" README.md docs/*.md, which must find nothing in the pages outside "Development".
  • Versioned contract comes from looking for a compatibility statement and a changelog under docs.
  • Loud failures come from [tool.pytest.ini_options] and the dev group in pyproject.toml.
  • Conditionals that name a situation start from grep -nE "^\s*(if|elif|while)\b.*(==|!=| in | not in )\s*[\(\{\[]?['\"]|startswith|endswith|legacy|\.name ==" src/uncoded/*.py, read with every other conditional in src.
  • Branch protection comes from gh api repos/alimanfoo/uncoded/branches/main/protection.
  • Cohesive modules, focused functions, true documents, accurate references, rules not cases, invariants by construction and symmetry come from review: each module and the longest functions read for one responsibility or job, the stories and the design's Stories section read against the code, docs/commands.md, docs/configuration.md and the skill sources read against the code and each command's --help, every conditional under src read for the situation it names, every branch, raise and assert under src read for a state the types could rule out, and the namespace map read for parallel operations.