Diff processing

The PyReprism.diffs module parses unified/git diffs and analyzes the changed code in each file’s own language. It powers churn metrics (added/removed lines split into code, comment and blank), cosmetic-change detection, and ML-oriented extraction of the changed code:

from PyReprism import diffs

d = diffs.parse(diff_text)
report = diffs.diff_stats(d)
report.totals()
diffs.cosmetic_files(d)

f = d.files[0]
f.added_text()
f.normalize("added")

By default analysis is fragment mode (from the diff text alone). Setting a DiffFile’s new_source / old_source to the full file contents switches it to accurate full-file mode for that file.

Process unified/git diffs with PyReprism’s language-aware operations.

A diff is parsed into files and hunks; each file’s path is used to detect its language, so the added/removed code can be tokenized, normalized, or measured.

Two accuracy modes:

Fragment mode (default) works from the diff text alone. It is self-contained but best-effort where a block comment or string spans a hunk boundary.
Full-file mode is used automatically for a DiffFile when its new_source / old_source are set (e.g. from git show); changed lines are then classified against the whole file for accuracy.

class PyReprism.diffs.Diff(files: List[PyReprism.diffs.DiffFile] = <factory>)

Bases: object

files: List[DiffFile]

class PyReprism.diffs.DiffFile(old_path: Optional[str] = None, new_path: Optional[str] = None, hunks: List[PyReprism.diffs.Hunk] = <factory>, is_binary: bool = False, is_new: bool = False, is_deleted: bool = False, is_rename: bool = False, new_source: Optional[str] = None, old_source: Optional[str] = None)

Bases: object

added_text() → str: Text of the added (+) lines.

changed_text(side: str = 'added') → str

extract_comments(side: str = 'added') → List[str]

hunks: List[Hunk]

is_binary: bool = False

is_deleted: bool = False

is_new: bool = False

is_rename: bool = False

property language: Detected language class for this file, or None.

new_path: str | None = None

new_source: str | None = None

new_text() → str: The new side of the hunks (context + added lines).

normalize(side: str = 'added', **options) → str: Canonicalize the changed code on side for ML/clone detection.

old_path: str | None = None

old_source: str | None = None

old_text() → str: The old side of the hunks (context + removed lines).

property path: str | None: The most representative path (new side, falling back to old side).

removed_text() → str: Text of the removed (-) lines.

tokenize(side: str = 'added'): Tokenize the changed code on side ('added' or 'removed').

class PyReprism.diffs.DiffFileStats(path: str | None, language: str | None, added_code: int = 0, added_comment: int = 0, added_blank: int = 0, removed_code: int = 0, removed_comment: int = 0, removed_blank: int = 0, is_binary: bool = False)