Conversation
GitPython's Python implementations of repository storage couple its behavior to on-disk formats and object-ID widths. Delegate repository discovery, configuration, references, reflogs, object storage, tree construction, index operations, and revision parsing to `git.cmd.Git` so the default backend works with SHA-1/SHA-256 objects and files/reftable references. Require Git 2.52 or newer, retain the deprecated `GitDB` as an explicit choice, and preserve raw `repo.git` access. Use existing unsafe option/protocol primitives together with operand validation, NUL-framed records, and protected option ordering. Suppress implicit hooks, filesystem monitors, maintenance, lazy fetching, external diffs, and default text conversion in managed plumbing. Explicit commit hooks remain supported; signing and custom archive commands require opt-in. Preserve the established clean/smudge-filter behavior of existing worktree operations. Keep common workflows and map recognizable native failures to established exceptions. Remove or restrict low-level binary index/tree, raw reflog, precompressed object, and direct storage mutation APIs that cannot be exposed faithfully through Git. Preserve semantic index edits through private native indexes, discover worktree storage through Git, and reconnect retained submodule metadata without assuming its object or reference backend. Document the API and CLI changes in `doc/source/changes.rst`, add format and injection coverage, update minimum-Git CI and fuzz tooling, and bound the existing throughput benchmarks for subprocess-based operations. This work is planned for GitPython 3.4 some weeks before Git 3's official release, allowing users to test this branch and report compatibility feedback beforehand. Validation: the full pytest run produced 1,328 passes, 79 skips, one expected failure, and three submodule failures that were resolved and passed reruns. A fresh submodule/offline run had 224 passes; its two metadata-alias failures were fixed and retested, followed by eight passing retained-metadata checks. The Git 2.52 matrix passed 127 tests, plus four later quoted-branch checks. Ruff, mypy, basedpyright, Sphinx with warnings as errors, Python 3.8 package smoke tests, and deterministic fuzz-harness/version-guard checks pass. The updated fuzzing Docker image was not built and no long fuzz campaign was run.
The Cygwin performance job failed all six setup phases because the Cygwin package currently supplies Git 2.51.0, below the new Git 2.52 minimum. The same mismatch prevents the regular Cygwin suite from opening repositories. Build upstream Git `v2.52.0` with Cygwin tools and install it in `/usr/local` before preparing the fixtures. Include the HTTPS development dependencies so the existing clone and remote tests retain network transport support. Verify that the selected `git` is the built version before continuing. Validation: the workflow parses as YAML, the added shell block passes `bash -n` and ShellCheck, and `git diff --check` passes. The native Cygwin build and test suites require the Windows CI runner.
The macOS Python 3.8 and 3.14 CI jobs each passed 1,348 tests but failed `test_refresh_with_good_relative_git_path_arg`. Installing the supported Homebrew Git exposes `/opt/homebrew/opt/git/bin` on `PATH`; changing into that directory resolves it to the versioned Cellar directory. Compute the expected relative executable path from the current directory after changing into it. This preserves the documented `Git.refresh()` behavior and executable symlinks while removing the test's assumption that `shutil.which()` and `os.getcwd()` retain the same directory spelling. Validation: reproduced the failure using a symlinked directory on `PATH`. All 36 refresh tests pass with both ordinary and symlinked `PATH` values. Ruff lint, Ruff format, and `git diff --check` pass.
The Windows Python 3.8 CI job reported 42 failures and 179 setup errors. Most submodule failures shared a discovery bug: Git resolves relative gitfile targets using forward slashes even when the caller supplies a native Windows path. Normalize Git-facing discovery operands with the existing platform helper, and account for native separators in Git's worktree registry output. Use matching `surrogateescape` codecs for Git protocol paths so undecodable tree names round-trip without Windows filesystem encoding changing their bytes. Verify path/stage, mode, and object ID after materializing a private index: `git update-index --index-info` can exit successfully while dropping Windows-incompatible names. Raise `ValueError` before publishing such an index and document the platform restriction in `changes.rst`. Keep unusual names in object-only tests when the host cannot represent them in a checkout. Use native-valid paths for worktree tests, assert rejection of unsupported index names and quoted file references, and keep full quoted reference coverage with reftable. Fix separator and LF assumptions in config, URL, and packed-reference fixtures. Close test-owned repositories before submodule removal and make the fake Windows Git executable discoverable without shell execution. Also restore root paths for `Repo.tree()` results resolved directly from tree IDs while preserving explicit subtree paths. Validation: 116 repository tests and 10 focused submodule tests passed; index/helper tests passed 93 with 2 platform skips; the focused format/safety run passed 72; config/reference/remote tests passed 57 with 24 subtests; 36 refresh tests and 2 revision regressions passed. The Git 2.52 targeted index matrix passed 19 tests. Ruff, mypy, basedpyright, Sphinx with warnings as errors, and `git diff --check` pass. Native Windows validation awaits CI.
The partial Cygwin fast-suite log exposed a failure in `test_valid_unusual_index_names_round_trip`: native Git omitted a literal backslash filename. Cygwin Git recognizes Windows separators and applies NTFS path protection, even though Python reports a POSIX platform. Move that case into the existing unsupported-name assertions for Cygwin. Check that GitPython raises `ValueError`, preserves the published index, and removes its lock. Other POSIX systems retain the round-trip case; Windows retains the control-character and colon rejection cases. Document the Cygwin restriction in `changes.rst`. Validation: three focused index tests pass locally. The Cygwin and Windows test branches pass with only Git's ignored-record behavior simulated; native Cygwin verification awaits CI. Ruff lint, formatting, and `git diff --check` pass.
The Windows Python 3.8 partial CI log showed two failures in `test_submodule_allows_existing_metadata_symlinks`: preparing update and move aliases raised `PermissionError` before invoking GitPython. Git marks the submodule's `.git` file hidden; Python's `write_text()` attempts to recreate it, which Windows rejects for an existing hidden file. Open that fixture file with `r+`, write the replacement target, and truncate it. This preserves the hidden attribute while replacing the full contents. The separate fixture that creates a previously absent gitfile is unchanged. Validation: all six native/windows37 update, move, and remove alias modes pass locally. Ruff lint, formatting, and `git diff --check` pass. The Windows-specific file-attribute behavior will be verified by CI.
The Windows Python 3.15 CI job failed to set up twelve missing-submodule cases because `shutil.rmtree()` cannot remove read-only loose Git objects. The fixture deliberately removes retained metadata to model an absent submodule, so use the existing `git.util.rmtree()` helper, which clears read-only attributes when retrying Windows deletions. All twelve affected cases pass locally, along with Ruff lint and format checks. Production behavior and test coverage are unchanged.
The Windows Python 3.15 CI job failed to remove submodule checkouts because persistent `cat-file` processes still used them as working directories. `Submodule.update()` relied on collection of its temporary `Repo`, but captured log records retained a `Head` argument and therefore the repository and its process. Recursive updates also opened an extra unbounded repository. Close the owned repository after updates and on errors, and reuse it for recursion with final cleanup. This preserves `keep_going` behavior while releasing processes even when logs or callbacks retain repository objects. The compatibility test now scopes its own repository and closes it before removal; allocation tracing identified those separate caller-owned handles. Four regressions retain real logging arguments and verify process cleanup for normal, failing, recursive, and recursive `keep_going` updates. Both previously failing tests pass with tracing asserting no live checkout processes at each removal. Ruff, mypy, basedpyright, and `git diff --check` pass locally. Native Windows validation will run in CI.
The Cygwin full suite reached 1,372 passing tests but failed its native-Git detection check. The new source build installed Git into `/usr/local`, while GitPython's existing detector expects `uname` beside the selected Git executable. Cygwin installs `uname` in its normal `/usr/bin` directory. Install Git 2.52 under `/usr`, replacing the older packaged Git and preserving that standard layout. Verify `Git.is_cygwin()` immediately after installing Python dependencies so a setup regression fails before the long test suite. The detector and its missing-`uname` behavior remain unchanged. YAML parsing, extracted Bash syntax, ShellCheck, and `git diff --check` pass locally. The preceding Cygwin performance suite also passed all six tests; native validation of the corrected installation runs in CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tasks
Created by Codex on behalf of Byron. Byron will review before this is ready to merge.
Summary
Important
This PR will become GitPython 3.4 some weeks before Git 3 is officially released.
Users can already test and run their software against this branch. Please report compatibility problems and provide feedback here in this PR.
To try it with Git 2.52 or newer installed:
python -m pip install --upgrade "git+https://github.com/gitpython-developers/GitPython.git@back-to-the-roots"GitPython now delegates repository discovery, configuration, references, reflogs, object storage, tree construction, index operations, and revision parsing to guarded Git commands. The default backend supports SHA-1 and SHA-256 objects with either files or reftable reference storage, without interpreting Git's binary storage formats in Python.
The deprecated
GitDBremains explicitly selectable, and rawrepo.gitcommand access remains available. The migration notes inchanges.rstdescribe the API changes and differences in Git CLI behavior.Context
The goal is to support Git's object and reference formats while reducing the attack surface of Python implementations and limiting argument injection and unintended CLI side effects.
git.cmd.Git, existing unsafe option/protocol checks, validated operands, and framed stdin records. Shell execution and option reordering past protective flags are rejected. Persistentcat-filerequests use NUL framing.git hook run. Signing/editor options and custom archive format commands require an unsafe-options opt-in.GitCommandErrorwith Git's status and diagnostics.oldhexsha, precompressed object streams and custom object-output writers,Submodule.rename(), and directRepo.alternatesmutation. Configuration follows Git syntax and no longer subclassesRawConfigParser. Detailed replacements and limitations are in the changelog.The change also updates documentation, format/backend and injection regressions, the minimum-Git CI job, and fuzz harnesses. Existing performance benchmarks use bounded samples to keep subprocess-based runs practical. The release announcement above describes the planned 3.4 release; this PR does not change
VERSIONyet.Validation
python -m pytest -q --no-cov --tb=shortrun: 1,328 passed, 79 skipped, 1 xfailed, 38 subtests passed, with three submodule failures. All three were subsequently resolved and passed reruns; this was not a single all-green full-suite run.git diff --checkpass. These static checks were repeated while preparing this PR.User Prompts