Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,12 @@ A small `.gitnested` file in the folder remembers the URL, the branch and
the upstream commit. So the commands stay short.

Your repository has submodules already? Turn them all into nested
repositories:
repositories, and keep following upstream:

```bash
git nested submodules init --recursive
git fetch origin
git nested submodules update origin/main
```

## Why git-nested?
Expand Down
45 changes: 44 additions & 1 deletion doc/architecture/diagrams.md
Original file line number Diff line number Diff line change
Expand Up @@ -783,7 +783,50 @@ sequenceDiagram

## submodules

`init` runs `init --from-submodule` for each submodule.
`init` runs `init --from-submodule` for each submodule. `update` looks
like this:

```mermaid
sequenceDiagram
participant User
participant git-nested
participant Git

User->>git-nested: git nested submodules update <commit>
git-nested->>Git: merge-base --is-ancestor <commit> HEAD
Note over git-nested: Merged already: only update the nested repos that lag behind
opt --dry-run
Note over git-nested: Print the plan, also deeper down, then stop
end
git-nested->>Git: merge --no-commit --no-ff <commit>
Note over git-nested: Write .git/nested-submodules-update.json for --continue and --abort
git-nested->>Git: update-index --force-remove <gitlink>
Note over git-nested: Drop each gitlink and its part of .gitmodules
Note over git-nested: Merge .gitattributes without git-nested's block, then add the block again
git-nested->>Git: commit --no-edit
Note over git-nested: Stop on other conflicts. --continue goes on here
opt A submodule pull stopped before
Note over git-nested: --continue finishes it there, then runs commit
end

loop For each submodule upstream changed
alt moved
git-nested->>Git: mv <old> <new>
else removed
git-nested->>Git: rm -r <path>
else added
Note over git-nested: Same as clone. Not for update = none
else new pin
Note over git-nested: Same as pull -b <pin> -u. Skipped if it has a pin
end
git-nested->>Git: commit
end

opt --single-commit
git-nested->>Git: reset --soft <merge commit>
git-nested->>Git: commit --amend
end
```

## diff

Expand Down
4 changes: 3 additions & 1 deletion doc/cli/clone.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,9 @@ If nothing changed, it does nothing and says "already up to date".
- If your `.gitattributes` would change files of the nested repository,
for example `*.bat text eol=crlf`, `clone` and `pull` add `-text` for
these files to `.gitattributes`, in a commit of their own. So the files
stay as upstream has them.
stay as upstream has them, in git and in your checkout.
- With `--recurse-submodules`, a submodule with `update = none` in
`.gitmodules` stays a submodule, as in git.
- If upstream is empty, `clone` stops. Use [`init`](init.md) and then
[`push`](push.md) to fill it.
- With [patches](../guides/patches.md), `--force` starts over: it takes
Expand Down
5 changes: 3 additions & 2 deletions doc/cli/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Each command has its own page. This page lists what they share.
| [`push`](push.md) | push a nested repository's local changes upstream |
| [`fetch`](fetch.md) | fetch upstream content for a nested repository |
| [`status`](status.md) | report on the nested repositories of this repository |
| [`submodules`](submodules.md) | turn all submodules into nested repositories |
| [`submodules`](submodules.md) | turn all submodules into nested repositories and keep them up to date |
| [`diff`](diff.md) | show a nested repository's local changes against upstream |
| [`patches`](patches.md) | write, check, drop or cherry-pick the patches of a nested repository |
| [`verify`](verify.md) | check that a nested repository is its upstream commit plus patches or diffs |
Expand Down Expand Up @@ -117,7 +117,8 @@ The details are on each command's page. In short:
| `-f`, `--force` | force the operation; what it does differs per command |
| `-S`, `--gpg-sign` | GPG-sign the commits git-nested makes |
| `--signoff` | add a `Signed-off-by:` trailer |
| `--single-commit` | with `--all`: one commit for all |
| `--single-commit` | with `--all`: one commit for all. With `submodules`: one merge commit |
| `--dry-run` | show what would happen; change nothing |

### `--all` and `-b`

Expand Down
185 changes: 169 additions & 16 deletions doc/cli/submodules.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,66 @@
# `git nested submodules`

Turn all submodules into nested repositories.
Turn all submodules into nested repositories and keep them up to date.

```text
git nested submodules init [--recursive] [--resolve-relative <url>] [--only <path> | --ignore <path>]...
git nested submodules update [<options>] <commit>
git nested submodules update --continue
git nested submodules update --abort
```

## What it does

Use it for a repository whose submodules should be plain files. You get one
repository with all the code in it.

The repository itself stays a normal git clone of its upstream. Each
submodule becomes a nested repository.
The repository itself stays a normal git clone of its upstream. Upstream
commits keep their SHAs and authors. Each submodule becomes a nested
repository.

`init` nests every submodule at the commit it pins. Each submodule gets one
commit. With `--recursive`, it nests their submodules too. With `--only` or
`--ignore`, it nests only some of them.
- `init` nests every submodule at the commit it pins. With `--recursive`,
it nests their submodules too. Each submodule gets one commit. With
`--only` or `--ignore`, it nests only some of them.
- `update <commit>` merges an upstream commit, for example `upstream/main`.
Then it changes the nested repositories the way upstream changed its
submodules. Each changed submodule gets one commit. It does not fetch.
If the commit is merged already, it only updates the nested
repositories that lag behind.

No gitlink and no `.gitmodules` entry stays for a nested submodule.
`update` handles these upstream changes:

| Upstream did | `update` does |
|---|---|
| moved a pin | pull the nested repository to the new commit. Your changes stay |
| added a submodule | nest it. Its submodules too, unless a `.gitnested` says `submodules: keep` |
| removed a submodule | remove the nested repository. A warning says if you changed it |
| moved a submodule | move the folder. Your changes stay |
| changed a url | write the new `remote` to `.gitnested` |

No gitlink and no `.gitmodules` entry comes back for a nested submodule.
Nested repositories that you added yourself are not touched.

A submodule with `update = none` in `.gitmodules` stays a submodule, as in
git. git does not check those out either.

## Arguments

| Argument | Meaning |
|---|---|
| `<commit>` | `update`: what to merge. A branch like `upstream/main`, a tag, or a SHA. Fetch it first |

## Options

| Option | Meaning |
|---|---|
| `--only <path>` | nest only the submodules at or below this path. The others stay submodules. Give it more than once for more paths |
| `--ignore <path>` | nest all submodules but the ones at or below this path. Give it more than once for more paths. Not with `--only` |
| `--recursive` | nest the submodules of the submodules too, at any depth. Without it, they stay submodules. The choice goes into each `.gitnested` as [`submodules`](../configuration/gitnested-file.md) |
| `--resolve-relative <url>` | resolve relative submodule urls against this url. Without it, `init` stops at a relative url |
| `--only <path>` | `init`: nest only the submodules at or below this path. The others stay submodules. Give it more than once for more paths |
| `--ignore <path>` | `init`: nest all submodules but the ones at or below this path. Give it more than once for more paths. Not with `--only` |
| `--recursive` | `init`: nest the submodules of the submodules too, at any depth. Without it, they stay submodules. The choice goes into each `.gitnested` as [`submodules`](../configuration/gitnested-file.md), and `update` follows it |
| `--resolve-relative <url>` | `init`: resolve relative submodule urls against this url. Without it, `init` stops at a relative url |
| `--dry-run` | show what would change, also deeper down. Change nothing. It may fetch the commits it needs to look at |
| `--single-commit` | `update <commit>`: make one merge commit with all submodule changes in it |
| `--continue` | go on after a stop on conflicts |
| `--abort` | throw away an update that stopped on conflicts |
| `-S`, `--gpg-sign` | GPG-sign the commits |
| `--signoff` | add a `Signed-off-by:` trailer to the commits |

Expand All @@ -37,6 +70,38 @@ No gitlink and no `.gitmodules` entry stays for a nested submodule.
git clone https://github.com/user/project
cd project
git nested submodules init --recursive
git fetch origin
git nested submodules update origin/main
```

### A fork

Your fork is `origin`. The project you forked is `upstream`:

```bash
git remote add upstream https://github.com/org/project
git nested submodules init --recursive
git fetch upstream
git nested submodules update upstream/main
git push origin main
```

`update` takes a tag or a SHA too. For example, to follow releases:

```bash
git fetch upstream --tags
git nested submodules update v2.0
```

### Not with `git merge` or `git pull`

Merge upstream with `update`, not with git itself. git can't put a gitlink
where a nested folder is, so a plain `git merge` stops, or brings gitlinks
into your history. If you started one, throw it away and use `update`:

```bash
git merge --abort
git nested submodules update upstream/main
```

### Relative urls
Expand All @@ -50,7 +115,8 @@ git nested submodules init --resolve-relative https://github.com/org/project

Now `../lib.git` is `https://github.com/org/lib.git`. This is only for
the first level. With `--recursive`, a deeper relative url is relative to
the url of its parent submodule, as in git.
the url of its parent submodule, as in git. Later, `update` resolves new
relative urls against the remote of the commit it merges.

### Only some submodules

Expand All @@ -61,25 +127,112 @@ git nested submodules init --only lib/core --only third_party
```

This nests `lib/core` and every submodule below `third_party`. The others
stay submodules.
stay submodules. `update` updates them as submodules.

`init` keeps `--only` and `--ignore` in `.gitmodules`, in a `[nested]`
section. So `update` leaves a new submodule out too, if `init` would have
left it out. Upstream's changes to `.gitmodules` come in for the
submodules that stay. Your own changes there stay too. On a conflict
there, the conflict lines start with `#`. So git can still read the file.

Or name the ones to keep as submodules:

```bash
git nested submodules init --ignore vendor/big
```

`init` keeps `--only` and `--ignore` in `.gitmodules`, in a `[nested]`
section.
### See first

```bash
git nested submodules update --dry-run upstream/main
```

```text
would merge 3 commits of upstream/main
modules/core: would pull to 1a2b3c4d5e6f
modules/core/third_party/lib: would pull to 7d8e9f0a1b2c
docs: would nest at 9f8e7d6c5b4a
tools: would remove (your changes in it go too. They stay in the history)
```

### Conflicts

A conflict outside the submodules stops the merge:

```bash
git nested submodules update upstream/main
# fix the files, then
git add <files>
git nested submodules update --continue
```

A conflict inside a submodule stops it too. The message says where:

```text
modules/core: the update stopped on conflicts in src/main.c.
Fix them there, and 'git add' them:

cd .git/tmp/nested/modules/core

Then run 'git nested submodules update --continue'. It finishes modules/core and goes on with the others.
Or throw the update away: 'git nested submodules update --abort'.
```

To give up instead:

```bash
git nested submodules update --abort
```

Nested repositories that were updated already stay updated. The stopped
one goes back. Run `git nested submodules update upstream/main` again to
update the rest.

### Keep a submodule back

Set a pin. Then `update` leaves it where it is, with a warning:

```bash
git nested config modules/core pin 1a2b3c4
```

To follow upstream again, remove the `pin` line from
`modules/core/.gitnested`.

## History

The merge keeps the upstream commits as they are. So they still have their
submodules:

- `git checkout` of an upstream commit or tag shows empty submodule
folders.
- `git log -p` of an upstream commit shows the submodule changes as gitlink
changes.

Your own commits never have a gitlink. Use them for builds and releases. To search for a bug with `git bisect`, add
`--first-parent`.

By default, the merge commit comes first. One commit per submodule follows.
At the merge commit, the submodules are still at their old commits. If every
commit must build, use `--single-commit`. Its message lists the submodule
changes.

## Notes

- `init` needs a clean working tree.
- `init` and `update` need a clean working tree. `update` needs a branch,
not a detached HEAD.
- If upstream moves a submodule to a path you use already, it stays where
it is. A warning says so.
- If upstream adds a submodule at a path you use already, it is not nested.
A warning says so.
- If your `.gitattributes` would change files of a nested repository, for
example `*.bat text eol=crlf`, git-nested adds `-text` for these files to
`.gitattributes`. So they stay as upstream has them, as in a submodule.
`update` keeps this block at the end of the file, also when upstream adds
rules there.
- Push your changes to each submodule's upstream with
[`push`](push.md), as for any nested repository.
- Changes to the upstream superproject itself are not pushed by this command.

## See also

Expand Down
4 changes: 2 additions & 2 deletions doc/configuration/gitnested-file.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,8 +201,8 @@ What happens to the submodules inside the nested repository:
- `nest`: they become nested repositories too.
- `keep`: they stay submodules.

`submodules init` writes it: `keep`, or `nest` with `--recursive`. Unset
means `nest`.
`submodules init` writes it: `keep`, or `nest` with `--recursive`. Later,
`submodules update` follows it. Unset means `nest`.

### `cmdver`

Expand Down
27 changes: 24 additions & 3 deletions doc/guides/submodules.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,35 @@ repository.
## Turn a whole repository into one

Your repository has many submodules, maybe submodules in submodules. You
want all of them as plain files:
want all of them as plain files, and still follow upstream:

```bash
git nested submodules init --recursive
git fetch origin
git nested submodules update origin/main
```

`init --recursive` nests all of them, at any depth. Without
`--recursive`, it nests only the first level. `--only <path>` nests only
some.
some. `update` merges upstream, keeps its commits, and
changes the nested repositories the way upstream changed its submodules.

See [`submodules`](../cli/submodules.md).
To see what `update` would do first:

```bash
git nested submodules update --dry-run origin/main
```

In a fork, update from the project you forked:

```bash
git nested submodules init --recursive
git fetch upstream
git nested submodules update upstream/main
```

Use `update` instead of `git merge` or `git pull`. A plain git merge
can't put a gitlink where a nested folder is.

See [`submodules`](../cli/submodules.md). It also explains how the merged
history looks, and how to keep a submodule back.
2 changes: 1 addition & 1 deletion man/man1/git-nested.1
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ run a git command in the worktree of a nested repository
report on the nested repositories of this repository
.TP
.B submodules
turn all submodules into nested repositories
turn all submodules into nested repositories and keep them up to date
.TP
.B verify
check that a nested repository is its upstream commit plus patches or diffs
Expand Down
Loading
Loading