Files
Technical-Assignment-CQG-Br…/README.md

5.2 KiB

Replace Configured Values in a Text File

Reads value1=value2 pairs from a configuration file, replaces every non-overlapping occurrence of value1 with value2 in a text file, and prints the resulting lines sorted by how many symbols were replaced, most-changed first.

Requirements

Python 3.9 or newer. No runtime dependencies.

Usage

python main.py <config_file> <text_file>

Example

With the provided config.txt and input.txt, running:

python main.py config.txt input.txt

produces:

b#sjxf_znsyybvo!      6 symbols replaced
dzjf#zemfzoxfnz%      5
jgreyk6hnze           3
xnhjrfyjvth3nxr       1

(annotations added here for clarity; the program prints the lines only)

Design decisions

The assignment leaves a few points open. The choices made here are:

"Total number of symbols replaced" counts source symbols. A line where bb and a were both replaced counts 3, being the combined length of the matched value1 strings, not the number of replacements and not the length of the substituted text. This makes the count a measure of how much of the original line changed.

Overlapping patterns resolve to the longest match. If the configuration contains both b and bb, the input bbbb is read as two bb matches rather than four b matches. The assignment guarantees unique values but not that none is a prefix of another, so prepare_pairs orders the pairs by descending length once, and replace_line takes the first match at each position. Without a rule like this the output would depend on configuration file ordering.

Ties keep their original file order. The final sort keys only on the replacement count and Python's sort is stable, so lines replaced an equal number of times appear in the order they had in the input file.

Unchanged lines are still printed. The assignment says to sort the changed lines but also to output the resulting text. Dropping lines that happened to match nothing would silently lose content, so every line is printed and lines with zero replacements simply sort last.

Input is preserved apart from the line terminator. Both \n and \r\n are stripped; every other character is kept and eligible for replacement. The configuration format has no quoting either, so whitespace inside a value is significant: the trailing space in a=z is part of the replacement value.

Performance characteristics

replace_line tries every configured pair at each position in the line, so the work grows with both the line length and the number of pairs. Measured on a fixed 20,000-line input of 40-character lines:

Pairs Time
3 0.27s
100 6.13s
1000 58.8s

Time tracks the pair count roughly proportionally, so it is the size of the configuration rather than the size of the input that dominates. Parsing, ordering the pairs and the final sort stay under a millisecond each even at 1000 pairs; the per-position scan is effectively the entire cost.

Two alternatives were measured against the same input and produce identical output:

  • A single re alternation moves the scan into CPython's C engine and stays at roughly 0.011s at every configuration size. It is only equivalent because prepare_pairs already orders the patterns longest first: Python's re is leftmost-first rather than leftmost-longest, so b|bb would otherwise match b against the input bb.
  • A prefix tree walked once per position removes the pair count from the complexity rather than making each step cheaper, and stays at roughly 0.067s. It is slower than the regex here only because it is interpreted Python competing with a C engine.

The explicit loop is kept: at the sizes this assignment describes the difference is not observable, and the loop keeps the longest-match rule visible in the code. A configuration large enough for the scan to matter would justify one of the alternatives above.

Memory is proportional to the size of the input file, since every line is held before sorting and sorting cannot be done on a stream.

Error handling

The configuration file is validated while loading: a line without =, and a line whose value1 is empty, are both rejected reporting the offending line number. An empty value1 would match at every position without consuming anything, which is why it is refused rather than ignored. Missing files, unreadable files, and input that is not valid UTF-8 are reported as messages on stderr rather than tracebacks, and exit with status 1. Status 2 is reserved by argparse for command line misuse.

Tests

pip install -r requirements-dev.txt
pytest
  • tests/unit/ covers configuration parsing and the replacement algorithm, including the overlapping-pattern and empty-input edge cases, plus a line by line check of the worked example in test_sample_data.py.
  • tests/integration/test_cli.py invokes main.py as a subprocess to cover what unit tests cannot: argument handling, exit status, stderr messages, and the sort order as actually printed.

Layout

main.py           command line entry point, sorting and output
config_loader.py  configuration file parsing and validation
replacer.py       replacement algorithm
tests/            unit and integration tests