docs: document performance characteristics
This commit is contained in:
50
README.md
50
README.md
@@ -63,14 +63,54 @@ with zero replacements simply sort last.
|
|||||||
**Input is preserved apart from the line terminator.** Both `\n` and `\r\n` are
|
**Input is preserved apart from the line terminator.** Both `\n` and `\r\n` are
|
||||||
stripped; every other character is kept and eligible for replacement. The
|
stripped; every other character is kept and eligible for replacement. The
|
||||||
configuration format has no quoting either, so whitespace inside a value is
|
configuration format has no quoting either, so whitespace inside a value is
|
||||||
significant — the trailing space in `a=z ` is part of the replacement value.
|
significant: the trailing space in `a=z ` is part of the replacement value.
|
||||||
|
|
||||||
|
## Performance characteristics
|
||||||
|
|
||||||
|
`replace_line` tries every configured pair at each position in the line, so the
|
||||||
|
work grows with both the line length and the number of pairs. Measured on a
|
||||||
|
fixed 20,000-line input of 40-character lines:
|
||||||
|
|
||||||
|
| Pairs | Time |
|
||||||
|
| ----: | ---: |
|
||||||
|
| 3 | 0.27s |
|
||||||
|
| 100 | 6.13s |
|
||||||
|
| 1000 | 58.8s |
|
||||||
|
|
||||||
|
Time tracks the pair count roughly proportionally, so it is the size of the
|
||||||
|
configuration rather than the size of the input that dominates. Parsing,
|
||||||
|
ordering the pairs and the final sort stay under a millisecond each even at
|
||||||
|
1000 pairs; the per-position scan is effectively the entire cost.
|
||||||
|
|
||||||
|
Two alternatives were measured against the same input and produce identical
|
||||||
|
output:
|
||||||
|
|
||||||
|
- A single `re` alternation moves the scan into CPython's C engine and stays at
|
||||||
|
roughly 0.011s at every configuration size. It is only equivalent because
|
||||||
|
`prepare_pairs` already orders the patterns longest first: Python's `re` is
|
||||||
|
leftmost-first rather than leftmost-longest, so `b|bb` would otherwise match
|
||||||
|
`b` against the input `bb`.
|
||||||
|
- A prefix tree walked once per position removes the pair count from the
|
||||||
|
complexity rather than making each step cheaper, and stays at roughly 0.067s.
|
||||||
|
It is slower than the regex here only because it is interpreted Python
|
||||||
|
competing with a C engine.
|
||||||
|
|
||||||
|
The explicit loop is kept: at the sizes this assignment describes the difference
|
||||||
|
is not observable, and the loop keeps the longest-match rule visible in the
|
||||||
|
code. A configuration large enough for the scan to matter would justify one of
|
||||||
|
the alternatives above.
|
||||||
|
|
||||||
|
Memory is proportional to the size of the input file, since every line is held
|
||||||
|
before sorting and sorting cannot be done on a stream.
|
||||||
|
|
||||||
## Error handling
|
## Error handling
|
||||||
|
|
||||||
The configuration file is validated while loading: a line without `=` is
|
The configuration file is validated while loading: a line without `=`, and a
|
||||||
rejected, reporting the offending line number. Missing files, unreadable
|
line whose value1 is empty, are both rejected reporting the offending line
|
||||||
files, and input that is not valid UTF-8 are reported as messages on stderr
|
number. An empty value1 would match at every position without consuming
|
||||||
rather than tracebacks, and exit with status 1. Status 2 is reserved by
|
anything, which is why it is refused rather than ignored. Missing files,
|
||||||
|
unreadable files, and input that is not valid UTF-8 are reported as messages on
|
||||||
|
stderr rather than tracebacks, and exit with status 1. Status 2 is reserved by
|
||||||
`argparse` for command line misuse.
|
`argparse` for command line misuse.
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|||||||
Reference in New Issue
Block a user