128 lines
4.6 KiB
Markdown
128 lines
4.6 KiB
Markdown
# Replace Configured Values in a Text File
|
|
|
|
Reads `value1=value2` pairs from a configuration file, replaces every
|
|
non-overlapping occurrence of `value1` with `value2` in a text file, and prints
|
|
the resulting lines sorted by how many symbols were replaced, most-changed
|
|
first.
|
|
|
|
## Requirements
|
|
|
|
Python 3.9 or newer. No runtime dependencies.
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
python main.py <config_file> <text_file>
|
|
```
|
|
|
|
### Example
|
|
|
|
With the provided `config.txt` and `input.txt`, running:
|
|
|
|
```bash
|
|
python main.py config.txt input.txt
|
|
```
|
|
|
|
produces:
|
|
|
|
```
|
|
b#sjxf_znsyybvo! 6 symbols replaced
|
|
dzjf#zemfzoxfnz% 5
|
|
jgreyk6hnze 3
|
|
xnhjrfyjvth3nxr 1
|
|
```
|
|
|
|
(annotations added here for clarity; the program prints the lines only)
|
|
|
|
## Design decisions
|
|
|
|
The assignment leaves a few points open:
|
|
|
|
**"Total number of symbols replaced" counts source symbols.** A line where `bb`
|
|
and `a` were replaced counts 3, the combined length of the matched `value1`
|
|
strings, not the number of replacements.
|
|
|
|
**Overlapping patterns resolve to the longest match.** With both `b` and `bb`
|
|
configured, `bbbb` is two `bb` matches, not four `b`. Values are unique but may
|
|
still be prefixes of one another, so `prepare_pairs` orders by descending length
|
|
and `replace_line` takes the first match at each position. Otherwise the output
|
|
would depend on configuration file ordering.
|
|
|
|
**Ties keep their original file order.** The sort keys only on the count, and
|
|
Python's sort is stable.
|
|
|
|
**Unchanged lines are still printed.** Dropping lines that matched nothing
|
|
would silently lose content, so every line is printed; those with no
|
|
replacements sort last.
|
|
|
|
**Only the line terminator is stripped.** `\n` and `\r\n` both; every other
|
|
character stays replaceable. The format has no quoting, so the trailing space in
|
|
`a=z ` is part of the value.
|
|
|
|
## Performance characteristics
|
|
|
|
`replace_line` tries every pair at each position, so cost grows with both the
|
|
line length and the number of pairs. Two alternatives, measured on the same
|
|
20,000-line input of 40-character lines and producing identical output:
|
|
|
|
| Pairs | this implementation | regex alternation | prefix tree |
|
|
| ----: | ------------------: | ----------------: | ----------: |
|
|
| 3 | 1x | 1x | 1x |
|
|
| 100 | 20x | 1.2x | 1.1x |
|
|
| 1000 | 207x | 1.1x | 1.1x |
|
|
|
|
Median of repeated runs, relative to each column's own three-pair baseline, so
|
|
the figures do not depend on the machine. The three-pair run takes about a
|
|
quarter of a second. Neither alternative is quite flat, because the larger
|
|
configurations in this benchmark also use longer keys, which cost slightly more
|
|
to match.
|
|
|
|
The regex alternation joins every `value1` into one pattern (`a|bb|c`) that
|
|
Python's `re` module scans in C: a constant-factor win, the same work done
|
|
faster. The prefix tree is the algorithmic one, taking the pair count out of the
|
|
complexity rather than making each step cheaper. The explicit loop is kept
|
|
because at the sizes described here the difference is not observable, and it
|
|
keeps the longest-match rule visible in the code.
|
|
|
|
The alternation is only equivalent because `prepare_pairs` orders the patterns
|
|
longest first: `re` is leftmost-first rather than leftmost-longest, so `b|bb`
|
|
would otherwise match `b` against the input `bb`.
|
|
|
|
Memory is proportional to the input size, since every line is held before
|
|
sorting.
|
|
|
|
## Error handling
|
|
|
|
The configuration file is validated while loading: a line without `=`, and a
|
|
line whose `value1` is empty, are both rejected, reporting the offending line
|
|
number. An empty `value1` would match at every position without consuming
|
|
anything, which is why it is refused rather than ignored.
|
|
|
|
Missing files, unreadable files, and input that is not valid UTF-8 are reported
|
|
as messages on stderr rather than tracebacks, and exit with status 1. Status 2
|
|
is reserved by `argparse` for command line misuse.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
pip install -r requirements-dev.txt
|
|
pytest
|
|
```
|
|
|
|
- `tests/unit/` covers configuration parsing and the replacement algorithm,
|
|
including the overlapping-pattern case, empty input, and the rejection of an
|
|
empty `value1`, plus a line by line check of the worked example in
|
|
`test_sample_data.py`.
|
|
- `tests/integration/test_cli.py` invokes `main.py` as a subprocess to cover
|
|
what unit tests cannot: argument handling, exit status, stderr messages, and
|
|
the sort order as actually printed.
|
|
|
|
## Layout
|
|
|
|
```
|
|
main.py command line entry point, sorting and output
|
|
config_loader.py configuration file parsing and validation
|
|
replacer.py replacement algorithm
|
|
tests/ unit and integration tests
|
|
```
|