Repository navigation
Description
What you are trying to do
I maintain a fork of an upstream Git repository that uses Git submodules, including recursive/nested submodules.
I want to flatten all submodules into regular directories, while keeping the original repository structure and continuing to synchronize with the upstream superproject.
Importantly, I want different history handling for the superproject and its submodules:
- Superproject: Preserve individual upstream commits and their history (do not squash them into a single commit).
- Submodules: Import their contents as regular files, squashing upstream changes into one commit per synchronization rather than importing their entire history.
- Local modifications: Allow local commits inside flattened submodule directories and preserve them across upstream updates.
The main problem arises when upstream changes .gitmodules or submodule gitlinks.
For example, upstream might add or remove a submodule, change its URL, or update its pinned commit. These changes should be reflected in my flattened repository without restoring Git submodules.
What you would like git-nested to do
Support synchronizing a flattened superproject at the repository root with its original upstream repository, including changes to .gitmodules and gitlinks.
For example, a hypothetical command:
git fetch upstream
git nested sync-upstream upstream/main --recurse-submodules
Expected behavior:
- Incorporate upstream superproject commits individually, preserving their commit boundaries, messages, and authors where possible (even if rewritten commit SHAs are necessary).
- Detect changes to
.gitmodulesand submodule gitlinks. - Automatically flatten newly added submodules, recursively.
- Update existing flattened submodules to their upstream-pinned commits.
- Handle removed, renamed, or relocated submodules and changed remote URLs.
- Preserve locally committed modifications inside flattened submodules, using Git's normal merge/conflict-resolution mechanisms.
- Keep all content as regular files and directories, without reintroducing gitlinks.
- Squash imported submodule changes into one commit per submodule update, without squashing unrelated superproject commits.
The distinction between preserving the superproject's commit history and squashing submodule updates is particularly important.
What you do instead today
git nested init --from-submodule can convert individual submodules, but it does not appear to solve subsequent synchronization with the original upstream superproject.
After flattening the submodules, a normal:
git fetch upstream
git merge upstream/main
can introduce conflicts between upstream gitlinks and local regular directories, or reintroduce submodules when upstream adds new ones.
The workaround appears to require manually inspecting .gitmodules and gitlink changes, resolving structural conflicts, and updating each flattened submodule separately.
Alternatively, I could import the entire upstream repository using:
git nested clone --recurse-submodules \
https://example.com/upstream.git project
However, this has two important limitations:
- It changes the repository's root layout, placing the whole superproject inside an additional directory.
- It squashes the superproject's history. The initial import produces a single commit, and subsequent pulls also squash upstream superproject commits into one commit per pull. I need to retain the individual superproject commits, while squashing only the submodule updates.
Is there already a supported workflow for maintaining such a flattened fork, preserving the superproject's commit granularity while recursively flattening and synchronizing its submodules?
Activity
thorsten-klein commentedon Nov 10, 2026
Hi @olfway, thanks for the clear write-up. A first version is ready in #202. Can you try it on your fork, and tell me if it fits what you need?
Install the branch
In a virtual environment:
Try it on your fork
Best on a new branch, so nothing changes on your main branch:
If your
.gitmoduleshas relative urls like../lib.git,initstops. Give it the url of the project you forked:git nested submodules init --recursive --resolve-relative <url>.To nest only some submodules, use
--only <path>or--ignore <path>, more than once if needed. The others stay submodules.If you fetch yourself, this works too:
Or merge with git itself. git stops on the changed submodules. Then
submodules pullfinishes the merge:What you get
git nested submodules pull --continue.Two things differ from your idea
git bisect --first-parent. We did this on purpose: with rewritten commits, every later sync would need a map of old to new SHAs. It would also get much slower and break more easily.--single-commit.The docs are in
doc/cli/submodules.md.One question: is your fork public? If yes, please share the link. Then I can try it on your real repo and fine-tune it.
fix: keep local changes in nested repos inside nested repos
feat: add `submodules` to nest all submodules and follow upstream
olfway commentedon Nov 12, 2026
Hi @thorsten-klein , thanks for your work!
I'll try to test it soon, and will write you how it works
My forks are local only for now, but upstreams are public:
they both has a lot of submodules
olfway commentedon Nov 13, 2026
I tested it on tdesktop, and it already works well enough
to replace my own update scripts. Thanks a lot for building this!
I used
feat/184-sync-gitlinks(8639728) on a fresh clone of tdesktop:submodules init --recursiveatc2694a2merge 7381fb1: 11 pins moved, 3 new submodules, deeper ones toopull origin devto346f10fAfter each step, the tree matched a
git submodule update --init --recursivecheckout, except for the items below.
Local changes in a nested repo stayed.
A conflict there worked with
--continue.Problems I found:
update = nonesubmodules get nested.git skips them.
In tdesktop,
quickjs/.gitmoduleshasupdate = nonefortest262,but git-nested nests it: 57k of 77k files (74%), 239 MB.
--abortleaves a half sync.After a conflict in a nested repo,
--abortkeeps the merge commitand the pulls before the conflict.
The conflicted submodule and the ones after it stay at the old pins.
Then
submodules merge <same target>saysalready up to date,so nothing updates them until upstream moves them again.
The conflict worktree
.git/tmp/nested/<path>also stays(
git nested cleanremoves it).The
# git-nested:block conflicts in.gitattributes.It is at the end of the file, where upstream adds rules too.
In tdesktop, upstream added
*.bin binary, and the merge stopped.git-nested owns the block, so it could take upstream's file
and add the block again at the end.
--dry-runmisses deeper submodules.It showed
widget-sandbox: would pull,but the real run also pulled
widget-sandbox/.../quickjs.CRLF in the checkout (minor).
With
*.bat text eol=crlfin the parent, LF.batfiles in nested reposare checked out with CRLF. A submodule keeps them LF.
The blobs are the same.
The
-textlines cover only files stored with CRLF.The conflict message during
submodulesis confusing (minor).It first shows the 7 steps of a normal
pull, which end withgit nested commit <path>. These steps do not say that you mustthen run
git nested submodules merge --continue, so the othersubmodules stay at the old pins. Then "Shorter: ... run
git nested submodules merge --continue" follows.For
submodules, showing only the--continueway would be clearer.feat: add `submodules` to nest all submodules and follow upstream
fix: keep local changes in nested repos inside nested repos (#223)
feat: add `submodules merge` and `submodules pull` to follow upstream
feat: add `submodules init` to nest all submodules at once (#228)
feat: add `submodules merge` and `submodules pull` to follow upstream
thorsten-klein commentedon Nov 14, 2026
Hi @olfway, thanks a lot for testing it on tdesktop, and for the clear feedback and explanations. The first version is now merged in #202. I tried to tackle the findings from your last comment. If you find more, please open a new issue, so we can tackle it in a follow-up PR.
The steps are the same as before, but
pullis nowupdate. It takes a commit and does not fetch: