docs: document performance characteristics

This commit is contained in:
2026-08-26 21:03:34 +02:00
parent 8be290a859
commit 62a4fce866

View File

@@ -36,72 +36,50 @@ xnhjrfyjvth3nxr 1
## Design decisions ## Design decisions
The assignment leaves a few points open. The choices made here are: The assignment leaves a few points open:
**"Total number of symbols replaced" counts source symbols.** A line where **"Total number of symbols replaced" counts source symbols.** A line where `bb`
`bb` and `a` were both replaced counts 3, being the combined length of the and `a` were replaced counts 3, the combined length of the matched `value1`
matched `value1` strings, not the number of replacements and not the length of strings, not the number of replacements.
the substituted text. This makes the count a measure of how much of the
original line changed.
**Overlapping patterns resolve to the longest match.** If the configuration **Overlapping patterns resolve to the longest match.** With both `b` and `bb`
contains both `b` and `bb`, the input `bbbb` is read as two `bb` matches rather configured, `bbbb` is two `bb` matches, not four `b`. Values are unique but may
than four `b` matches. The assignment guarantees unique values but not that still be prefixes of one another, so `prepare_pairs` orders by descending length
none is a prefix of another, so `prepare_pairs` orders the pairs by descending and `replace_line` takes the first match at each position. Otherwise the output
length once, and `replace_line` takes the first match at each position. Without would depend on configuration file ordering.
a rule like this the output would depend on configuration file ordering.
**Ties keep their original file order.** The final sort keys only on the **Ties keep their original file order.** The sort keys only on the count, and
replacement count and Python's sort is stable, so lines replaced an equal Python's sort is stable.
number of times appear in the order they had in the input file.
**Unchanged lines are still printed.** The assignment says to sort the changed **Unchanged lines are still printed.** Dropping non-matching lines would
lines but also to output the resulting text. Dropping lines that happened to silently lose content, so they print last with a count of zero.
match nothing would silently lose content, so every line is printed and lines
with zero replacements simply sort last.
**Input is preserved apart from the line terminator.** Both `\n` and `\r\n` are **Only the line terminator is stripped.** `\n` and `\r\n` both; every other
stripped; every other character is kept and eligible for replacement. The character stays replaceable. The format has no quoting, so the trailing space in
configuration format has no quoting either, so whitespace inside a value is `a=z ` is part of the value.
significant: the trailing space in `a=z ` is part of the replacement value.
## Performance characteristics ## Performance characteristics
`replace_line` tries every configured pair at each position in the line, so the `replace_line` tries every pair at each position, so cost grows with both the
work grows with both the line length and the number of pairs. Measured on a line length and the number of pairs. On a fixed 20,000-line input of
fixed 20,000-line input of 40-character lines: 40-character lines, against two alternatives producing identical output:
| Pairs | Time | | Pairs | this implementation | `re` alternation | prefix tree |
| ----: | ---: | | ----: | ------------------: | ---------------: | ----------: |
| 3 | 0.27s | | 3 | 0.27s | 0.011s | 0.067s |
| 100 | 6.13s | | 100 | 6.13s | 0.011s | 0.073s |
| 1000 | 58.8s | | 1000 | 58.80s | 0.011s | 0.067s |
Time tracks the pair count roughly proportionally, so it is the size of the The regex is a constant-factor win, running the scan in C; the prefix tree is
configuration rather than the size of the input that dominates. Parsing, the algorithmic one, taking the pair count out of the complexity entirely. The
ordering the pairs and the final sort stay under a millisecond each even at explicit loop is kept because at the sizes described here the difference is not
1000 pairs; the per-position scan is effectively the entire cost. observable, and the loop keeps the longest-match rule visible in the code.
Two alternatives were measured against the same input and produce identical The regex is only equivalent because `prepare_pairs` already orders patterns
output: longest first: Python's `re` is leftmost-first, not leftmost-longest.
- A single `re` alternation moves the scan into CPython's C engine and stays at Memory is proportional to the input size, since every line is held before
roughly 0.011s at every configuration size. It is only equivalent because sorting.
`prepare_pairs` already orders the patterns longest first: Python's `re` is
leftmost-first rather than leftmost-longest, so `b|bb` would otherwise match
`b` against the input `bb`.
- A prefix tree walked once per position removes the pair count from the
complexity rather than making each step cheaper, and stays at roughly 0.067s.
It is slower than the regex here only because it is interpreted Python
competing with a C engine.
The explicit loop is kept: at the sizes this assignment describes the difference
is not observable, and the loop keeps the longest-match rule visible in the
code. A configuration large enough for the scan to matter would justify one of
the alternatives above.
Memory is proportional to the size of the input file, since every line is held
before sorting and sorting cannot be done on a stream.
## Error handling ## Error handling