Skip to content

feat: add submodules update to follow upstream - #202

Merged
thorsten-klein merged 4 commits into
mainfrom
feat/184-sync-gitlinks
Nov 14, 2026
Merged

thorsten-klein merged 4 commits into
mainfrom
feat/184-sync-gitlinks

Conversation

@thorsten-klein

@thorsten-klein thorsten-klein commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

What this changes

A new command keeps the nested repos of submodules init (#228) in step
with upstream:

git fetch upstream
git nested submodules update --dry-run upstream/main  # see what it would do
git nested submodules update upstream/main            # merge, then update the nested repos

update merges the commit and keeps each upstream commit, with its SHA
and author. Then it changes the nested repos the way upstream changed its
submodules. Each changed submodule gets one commit. It never fetches.

Upstream did update does
moved a pin pull the nested repo to it. Local changes stay
added a submodule nest it, as submodules: in .gitnested says
removed a submodule remove the nested repo. Warn if it had local changes
moved a submodule move the folder. Local changes stay
changed a url write the new remote
  • It keeps what init chose. A new submodule that --only or --ignore
    leaves out stays a submodule. So does one with update = none, as in git.
  • Options: --dry-run (also shows deeper nested repos), --single-commit,
    --continue, --abort, -S, --signoff.
  • A nested repo with a pin stays where it is.
  • If the commit is merged already, update only updates the nested repos
    that lag behind. So after --abort, a second run does the rest.
  • A plain git merge of upstream is not the way. update says to abort it
    and use update instead.
  • Nested repos with patches work: the patches go on the new pin.

It also fixes what a test on tdesktop found (see #184):

  • update = none submodules got nested. Now they stay submodules, also in
    init and clone --recurse-submodules.
  • --abort left a half sync and the stopped worktree.
  • git-nested's block in .gitattributes conflicted with new upstream rules.
  • --dry-run missed deeper submodules.
  • LF files that a parent rule would check out with CRLF now stay LF.
  • A conflict in a nested repo showed the steps of a plain pull. Now it
    shows only update --continue and --abort.

Trade-offs

  • The upstream commits come in as they are. So they still have their
    gitlinks. A checkout of one shows empty submodule folders. Use
    git bisect --first-parent. Your own commits never have a gitlink.
  • By default the merge commit comes first. One commit per submodule
    follows. --single-commit makes one merge commit instead.

Speed

20 submodules, all moved by upstream: about 11 s with local repos. No
submodule moved: 0.25 s. Over a network, each moved submodule adds one
fetch.

Why

A fork of a project with submodules wants plain files, the upstream
history, and updates from upstream. A plain git merge of upstream breaks
on the gitlinks.

Fixes #184

Checklist

  • uv run poe all passes (ruff, mypy, pyright, bandit, complexipy, and the
    tests at 100% coverage)
  • New behaviour has a test
  • README / --help / the man page updated, if the change is visible to users
  • The PR title reads as a Conventional Commit
    (feat:, fix:, docs:, …) — it becomes the squash-merge commit message
    and the changelog line

@codecov

codecov Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (70b690e) to head (21ae6d9).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              main      #202    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           51        51            
  Lines         2985      3474   +489     
==========================================
+ Hits          2985      3474   +489     
Files with missing lines Coverage Δ
src/git_nested/cli/parser.py 100.00% <100.00%> (ø)
src/git_nested/cli/spec.py 100.00% <ø> (ø)
src/git_nested/commands/clone.py 100.00% <100.00%> (ø)
src/git_nested/commands/submodules.py 100.00% <100.00%> (ø)
src/git_nested/content.py 100.00% <100.00%> (ø)
src/git_nested/models.py 100.00% <100.00%> (ø)
src/git_nested/submodules.py 100.00% <100.00%> (ø)

thorsten-klein added a commit that referenced this pull request Nov 10, 2026
- The merge commit and a stopped submodule pull now come from git, not
  from the state file. So the file is never written back.
- --continue checks the file. A broken one asks for --abort.
- Cast the results of dataclasses.replace(), as cli/app.py does.

These are the SonarCloud findings on #202.
thorsten-klein added a commit that referenced this pull request Nov 10, 2026
- The merge commit and a stopped submodule pull now come from git, not
  from the state file. So the file is never written back.
- --continue checks the file. A broken one asks for --abort.
- Cast the results of dataclasses.replace(), as cli/app.py does.

These are the SonarCloud findings on #202.
@thorsten-klein
thorsten-klein force-pushed the feat/184-sync-gitlinks branch 2 times, most recently from 4e8fb43 to beb3796 Compare November 13, 2026 22:33
@thorsten-klein
thorsten-klein changed the base branch from main to fix/windows-relative-submodule-url November 13, 2026 22:34
thorsten-klein added a commit that referenced this pull request Nov 14, 2026
## What this changes

`pull` and `commit` keep the local changes of nested repos inside a
nested
repo.

- Before, `pull` cloned inner nested repos again. Their local changes
were
lost. Now they come back from HEAD. Then each one is pulled to its new
pin.
- `commit` does the same after you finish a pull by hand.
- No commit has a gitlink where a nested repo belongs. `clone
  --recurse-submodules` made one for a moment.
- `pull --force` still drops all local changes.
- If an inner repo stops on conflicts, a new pull does the rest.

## Why

Found while working on #184. The `submodules` command (#202) builds on
it.

Stack: #223 (this) → #224 → #225 → #226 → #202.

## Checklist

- [x] `uv run poe all` passes (ruff, mypy, pyright, bandit, complexipy,
and the
      tests at 100% coverage)
- [x] New behaviour has a test
- [x] README / `--help` / the man page updated, if the change is visible
to users
- [x] The PR title reads as a [Conventional
Commit](https://www.conventionalcommits.org)
(`feat:`, `fix:`, `docs:`, …) — it becomes the squash-merge commit
message
      and the changelog line
thorsten-klein added a commit that referenced this pull request Nov 14, 2026
…ed pull <subdir>` (#224)

## What this changes

The subject of a nested commit names the subdir: `git nested pull
<subdir>`
instead of `git nested pull`.

## Why

A sync of many nested repos makes many commits. With the same subject
for
each, `git log --oneline` can't tell them apart.

Stack: #223 → #224 → #225 → #226 → #202.

## Checklist

- [x] `uv run poe all` passes (ruff, mypy, pyright, bandit, complexipy,
and the
      tests at 100% coverage)
- [x] New behaviour has a test
- [x] README / `--help` / the man page updated, if the change is visible
to users
- [x] The PR title reads as a [Conventional
Commit](https://www.conventionalcommits.org)
(`feat:`, `fix:`, `docs:`, …) — it becomes the squash-merge commit
message
      and the changelog line
thorsten-klein added a commit that referenced this pull request Nov 14, 2026
… `.gitattributes` (#225)

## What this changes

After a clone or pull, git-nested adds `-text` to the root
`.gitattributes`
for files that a rule there would change. This gets a commit of its own.
The
files stay as upstream has them.

## Why

For a submodule, the parent's `.gitattributes` does not count. For a
nested
repo it does. A rule like `*.bat text eol=crlf` then shows upstream's
CRLF
files as changed. The tree is never clean, and the next pull refuses.

Stack: #223 → #224 → #225 → #226 → #202.

## Checklist

- [x] `uv run poe all` passes (ruff, mypy, pyright, bandit, complexipy,
and the
      tests at 100% coverage)
- [x] New behaviour has a test
- [x] README / `--help` / the man page updated, if the change is visible
to users
- [x] The PR title reads as a [Conventional
Commit](https://www.conventionalcommits.org)
(`feat:`, `fix:`, `docs:`, …) — it becomes the squash-merge commit
message
      and the changelog line
@thorsten-klein
thorsten-klein force-pushed the fix/windows-relative-submodule-url branch from d2b05f2 to 09925a2 Compare November 14, 2026 08:08
Base automatically changed from fix/windows-relative-submodule-url to main November 14, 2026 08:19
thorsten-klein added a commit that referenced this pull request Nov 14, 2026
…226)

## What this changes

A relative submodule url resolves right against a local Windows path.

## Why

A local remote on Windows can have backslashes. `posixpath` took the
whole
path as one name, so `../foo` became just `foo`.

Stack: #223 → #224 → #225 → #226 → #202.

## Checklist

- [x] `uv run poe all` passes (ruff, mypy, pyright, bandit, complexipy,
and the
      tests at 100% coverage)
- [x] New behaviour has a test
- [x] README / `--help` / the man page updated, if the change is visible
to users
- [x] The PR title reads as a [Conventional
Commit](https://www.conventionalcommits.org)
(`feat:`, `fix:`, `docs:`, …) — it becomes the squash-merge commit
message
      and the changelog line
@thorsten-klein
thorsten-klein changed the base branch from main to feat/184-submodules-init November 14, 2026 08:32
@thorsten-klein thorsten-klein changed the title feat: add submodules to nest all submodules and follow upstream feat: add submodules merge and submodules pull to follow upstream Nov 14, 2026
thorsten-klein added a commit that referenced this pull request Nov 14, 2026
## What this changes

A new command turns all submodules of a repository into nested
repositories:

```bash
git nested submodules init                  # the first level
git nested submodules init --recursive      # also deeper ones
git nested submodules init --only lib/core  # only some
```

- Each submodule gets one commit.
- `--only` and `--ignore` pick some of them. `init` keeps the choice in
  `.gitmodules`.
- `--resolve-relative <url>` resolves relative submodule urls. Without
it,
  `init` stops at one. In a fork, `origin` is the wrong base.
- The choice goes into `.gitnested` as `submodules: nest` or `keep`.
- A branch or push of a nested repo inside a nested repo leaves the
outer
  one alone.

## Why

Part one of #184. #202 builds on it: `submodules merge` and `submodules
pull` keep the nested repos in step with upstream.

Stack: #228 (this) → #202.

## Checklist

- [x] `uv run poe all` passes (ruff, mypy, pyright, bandit, complexipy,
and the
      tests at 100% coverage)
- [x] New behaviour has a test
- [x] README / `--help` / the man page updated, if the change is visible
to users
- [x] The PR title reads as a [Conventional
Commit](https://www.conventionalcommits.org)
(`feat:`, `fix:`, `docs:`, …) — it becomes the squash-merge commit
message
      and the changelog line
Base automatically changed from feat/184-submodules-init to main November 14, 2026 08:45
`merge` merges an upstream commit and keeps its commits, with their SHAs
and authors. Then it changes the nested repos the way upstream changed
its submodules. Each changed submodule gets one commit. `pull` fetches
first.

| Upstream did | They do |
|---|---|
| moved a pin | pull the nested repo to it. Local changes stay |
| added a submodule | nest it, as `submodules:` in .gitnested says |
| removed a submodule | remove the nested repo. Warn if it had changes |
| moved a submodule | move the folder. Local changes stay |
| changed a url | write the new `remote` |

- With no arguments, both finish a `git merge` that stopped on the
  submodules.
- They keep what `init` chose: new submodules that `--only` or
  `--ignore` leave out stay submodules. .gitmodules is merged for them.
  A conflict there stays readable for git.
- Options: --dry-run, --single-commit, --continue, --abort, -S,
  --signoff.
- A nested repo with a `pin` stays where it is.

Fixes #184
…or a rebase

--continue only finished a stopped merge. With patches, the pull stops
in `git am`. With method rebase, it stops in a rebase, with no branch
checked out. Both were lost. Now --continue finishes each of them.

Also add tests for `submodules` with nested repos that use patches.
`update <commit>` merges what you give it, for example upstream/main.
It never fetches. If the commit is merged already, it updates only the
nested repos that lag behind. During a plain `git merge`, it says to
abort that merge and use `update` instead.

It also fixes what a test on a big project found:

- A submodule with `update = none` stays a submodule, as in git.
  `init` and `clone --recurse-submodules` skip it too.
- `update <commit>` updates the nested repos that a stopped update left
  behind. `--abort` removes the stopped worktree.
- git-nested's block in .gitattributes no longer conflicts with new
  upstream rules.
- `--dry-run` shows the nested repos deeper down too.
- Files that git would check out with CRLF stay as upstream has them.
- A conflict in a nested repo shows only the `submodules` way on.
- A new kept submodule no longer makes .gitmodules conflict when the
  fork has none.
@thorsten-klein thorsten-klein changed the title feat: add submodules merge and submodules pull to follow upstream feat: add submodules update to follow upstream Nov 14, 2026
@sonarqubecloud

Copy link
Copy Markdown

@thorsten-klein
thorsten-klein merged commit 225908f into main Nov 14, 2026
15 checks passed
@thorsten-klein
thorsten-klein deleted the feat/184-sync-gitlinks branch November 14, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support synchronizing a flattened superproject with upstream Git commits and submodule changes

1 participant