diff --git a/README.md b/README.md index 126fdf1..76718d2 100644 --- a/README.md +++ b/README.md @@ -51,8 +51,9 @@ would depend on configuration file ordering. **Ties keep their original file order.** The sort keys only on the count, and Python's sort is stable. -**Unchanged lines are still printed.** Dropping non-matching lines would -silently lose content, so they print last with a count of zero. +**Unchanged lines are still printed.** Dropping lines that matched nothing +would silently lose content, so every line is printed; those with no +replacements sort last. **Only the line terminator is stripped.** `\n` and `\r\n` both; every other character stays replaceable. The format has no quoting, so the trailing space in @@ -61,22 +62,31 @@ character stays replaceable. The format has no quoting, so the trailing space in ## Performance characteristics `replace_line` tries every pair at each position, so cost grows with both the -line length and the number of pairs. On a fixed 20,000-line input of -40-character lines, against two alternatives producing identical output: +line length and the number of pairs. Two alternatives, measured on the same +20,000-line input of 40-character lines and producing identical output: -| Pairs | this implementation | `re` alternation | prefix tree | -| ----: | ------------------: | ---------------: | ----------: | -| 3 | 0.27s | 0.011s | 0.067s | -| 100 | 6.13s | 0.011s | 0.073s | -| 1000 | 58.80s | 0.011s | 0.067s | +| Pairs | this implementation | regex alternation | prefix tree | +| ----: | ------------------: | ----------------: | ----------: | +| 3 | 1x | 1x | 1x | +| 100 | 20x | 1.2x | 1.1x | +| 1000 | 207x | 1.1x | 1.1x | -The regex is a constant-factor win, running the scan in C; the prefix tree is -the algorithmic one, taking the pair count out of the complexity entirely. The -explicit loop is kept because at the sizes described here the difference is not -observable, and the loop keeps the longest-match rule visible in the code. +Median of repeated runs, relative to each column's own three-pair baseline, so +the figures do not depend on the machine. The three-pair run takes about a +quarter of a second. Neither alternative is quite flat, because the larger +configurations in this benchmark also use longer keys, which cost slightly more +to match. -The regex is only equivalent because `prepare_pairs` already orders patterns -longest first: Python's `re` is leftmost-first, not leftmost-longest. +The regex alternation joins every `value1` into one pattern (`a|bb|c`) that +Python's `re` module scans in C: a constant-factor win, the same work done +faster. The prefix tree is the algorithmic one, taking the pair count out of the +complexity rather than making each step cheaper. The explicit loop is kept +because at the sizes described here the difference is not observable, and it +keeps the longest-match rule visible in the code. + +The alternation is only equivalent because `prepare_pairs` orders the patterns +longest first: `re` is leftmost-first rather than leftmost-longest, so `b|bb` +would otherwise match `b` against the input `bb`. Memory is proportional to the input size, since every line is held before sorting. @@ -84,12 +94,13 @@ sorting. ## Error handling The configuration file is validated while loading: a line without `=`, and a -line whose value1 is empty, are both rejected reporting the offending line -number. An empty value1 would match at every position without consuming -anything, which is why it is refused rather than ignored. Missing files, -unreadable files, and input that is not valid UTF-8 are reported as messages on -stderr rather than tracebacks, and exit with status 1. Status 2 is reserved by -`argparse` for command line misuse. +line whose `value1` is empty, are both rejected, reporting the offending line +number. An empty `value1` would match at every position without consuming +anything, which is why it is refused rather than ignored. + +Missing files, unreadable files, and input that is not valid UTF-8 are reported +as messages on stderr rather than tracebacks, and exit with status 1. Status 2 +is reserved by `argparse` for command line misuse. ## Tests @@ -99,8 +110,9 @@ pytest ``` - `tests/unit/` covers configuration parsing and the replacement algorithm, - including the overlapping-pattern and empty-input edge cases, plus a line by - line check of the worked example in `test_sample_data.py`. + including the overlapping-pattern case, empty input, and the rejection of an + empty `value1`, plus a line by line check of the worked example in + `test_sample_data.py`. - `tests/integration/test_cli.py` invokes `main.py` as a subprocess to cover what unit tests cannot: argument handling, exit status, stderr messages, and the sort order as actually printed.