138 lines
5.2 KiB
Markdown
138 lines
5.2 KiB
Markdown
# Replace Configured Values in a Text File
|
|
|
|
Reads `value1=value2` pairs from a configuration file, replaces every
|
|
non-overlapping occurrence of `value1` with `value2` in a text file, and prints
|
|
the resulting lines sorted by how many symbols were replaced, most-changed
|
|
first.
|
|
|
|
## Requirements
|
|
|
|
Python 3.9 or newer. No runtime dependencies.
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
python main.py <config_file> <text_file>
|
|
```
|
|
|
|
### Example
|
|
|
|
With the provided `config.txt` and `input.txt`, running:
|
|
|
|
```bash
|
|
python main.py config.txt input.txt
|
|
```
|
|
|
|
produces:
|
|
|
|
```
|
|
b#sjxf_znsyybvo! 6 symbols replaced
|
|
dzjf#zemfzoxfnz% 5
|
|
jgreyk6hnze 3
|
|
xnhjrfyjvth3nxr 1
|
|
```
|
|
|
|
(annotations added here for clarity; the program prints the lines only)
|
|
|
|
## Design decisions
|
|
|
|
The assignment leaves a few points open. The choices made here are:
|
|
|
|
**"Total number of symbols replaced" counts source symbols.** A line where
|
|
`bb` and `a` were both replaced counts 3, being the combined length of the
|
|
matched `value1` strings, not the number of replacements and not the length of
|
|
the substituted text. This makes the count a measure of how much of the
|
|
original line changed.
|
|
|
|
**Overlapping patterns resolve to the longest match.** If the configuration
|
|
contains both `b` and `bb`, the input `bbbb` is read as two `bb` matches rather
|
|
than four `b` matches. The assignment guarantees unique values but not that
|
|
none is a prefix of another, so `prepare_pairs` orders the pairs by descending
|
|
length once, and `replace_line` takes the first match at each position. Without
|
|
a rule like this the output would depend on configuration file ordering.
|
|
|
|
**Ties keep their original file order.** The final sort keys only on the
|
|
replacement count and Python's sort is stable, so lines replaced an equal
|
|
number of times appear in the order they had in the input file.
|
|
|
|
**Unchanged lines are still printed.** The assignment says to sort the changed
|
|
lines but also to output the resulting text. Dropping lines that happened to
|
|
match nothing would silently lose content, so every line is printed and lines
|
|
with zero replacements simply sort last.
|
|
|
|
**Input is preserved apart from the line terminator.** Both `\n` and `\r\n` are
|
|
stripped; every other character is kept and eligible for replacement. The
|
|
configuration format has no quoting either, so whitespace inside a value is
|
|
significant: the trailing space in `a=z ` is part of the replacement value.
|
|
|
|
## Performance characteristics
|
|
|
|
`replace_line` tries every configured pair at each position in the line, so the
|
|
work grows with both the line length and the number of pairs. Measured on a
|
|
fixed 20,000-line input of 40-character lines:
|
|
|
|
| Pairs | Time |
|
|
| ----: | ---: |
|
|
| 3 | 0.27s |
|
|
| 100 | 6.13s |
|
|
| 1000 | 58.8s |
|
|
|
|
Time tracks the pair count roughly proportionally, so it is the size of the
|
|
configuration rather than the size of the input that dominates. Parsing,
|
|
ordering the pairs and the final sort stay under a millisecond each even at
|
|
1000 pairs; the per-position scan is effectively the entire cost.
|
|
|
|
Two alternatives were measured against the same input and produce identical
|
|
output:
|
|
|
|
- A single `re` alternation moves the scan into CPython's C engine and stays at
|
|
roughly 0.011s at every configuration size. It is only equivalent because
|
|
`prepare_pairs` already orders the patterns longest first: Python's `re` is
|
|
leftmost-first rather than leftmost-longest, so `b|bb` would otherwise match
|
|
`b` against the input `bb`.
|
|
- A prefix tree walked once per position removes the pair count from the
|
|
complexity rather than making each step cheaper, and stays at roughly 0.067s.
|
|
It is slower than the regex here only because it is interpreted Python
|
|
competing with a C engine.
|
|
|
|
The explicit loop is kept: at the sizes this assignment describes the difference
|
|
is not observable, and the loop keeps the longest-match rule visible in the
|
|
code. A configuration large enough for the scan to matter would justify one of
|
|
the alternatives above.
|
|
|
|
Memory is proportional to the size of the input file, since every line is held
|
|
before sorting and sorting cannot be done on a stream.
|
|
|
|
## Error handling
|
|
|
|
The configuration file is validated while loading: a line without `=`, and a
|
|
line whose value1 is empty, are both rejected reporting the offending line
|
|
number. An empty value1 would match at every position without consuming
|
|
anything, which is why it is refused rather than ignored. Missing files,
|
|
unreadable files, and input that is not valid UTF-8 are reported as messages on
|
|
stderr rather than tracebacks, and exit with status 1. Status 2 is reserved by
|
|
`argparse` for command line misuse.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
pip install -r requirements-dev.txt
|
|
pytest
|
|
```
|
|
|
|
- `tests/unit/` covers configuration parsing and the replacement algorithm,
|
|
including the overlapping-pattern and empty-input edge cases, plus a line by
|
|
line check of the worked example in `test_sample_data.py`.
|
|
- `tests/integration/test_cli.py` invokes `main.py` as a subprocess to cover
|
|
what unit tests cannot: argument handling, exit status, stderr messages, and
|
|
the sort order as actually printed.
|
|
|
|
## Layout
|
|
|
|
```
|
|
main.py command line entry point, sorting and output
|
|
config_loader.py configuration file parsing and validation
|
|
replacer.py replacement algorithm
|
|
tests/ unit and integration tests
|
|
```
|