nostos

                     __
   ____  ____  _____/ /_____  _____
  / __ \/ __ \/ ___/ __/ __ \/ ___/
 / / / / /_/ (__  ) /_/ /_/ (__  )
/_/ /_/\____/____/\__/\____/____/

  the homecoming - a fleet of git repositories

nostos is a Python CLI for batch-updating and curating fleets of git repositories in parallel. Built for developers and platform teams who maintain dozens - or hundreds - of cloned repositories and need a reliable, auditable, scriptable way to keep them in sync.

At a glance


License	MIT
Runtime	Python 3.10+ (one runtime dependency: `argcomplete`)
Platforms	Linux, macOS, Windows (CI matrix: Python 3.10 / 3.11 / 3.12 / 3.13)
Install	`pipx install nostos`, `pip install nostos`, or `pip install -e .` from a clone
Storage	Local SQLite at `$XDG_DATA_HOME/nostos/index.db` (mode `0600` on Unix). No telemetry, no remote service.
Network posture	Fail-closed. Hosts not listed in `~/.config/nostos/auth.toml` are never contacted, ever. `--offline` is a global kill switch.
Supply chain	PyPI releases via GitHub Actions OIDC trusted publishing (no static API tokens) with PEP 740 provenance attestations; third-party Actions are SHA-pinned and Dependabot-managed. Hardened clone path resists argument/transport injection and mitigates CVE-2024-32002 / 32004 / 32465.
Versioning	Strict SemVer 2.0; full per-release rationale in CHANGELOG.md.
Maintainers	See MAINTAINERS.md. Vulnerability disclosure: SECURITY.md.

Overview

nostos is three tools in one:

A batch-pull engine that walks a directory tree (and/or a curated index), discovers every git repository it can reach, and updates them concurrently. Dirty trees, detached HEADs, and missing upstreams are reported and skipped, never overwritten.
A metadata index (SQLite) recording identity, provenance, tags, notes, and triage status for every repository in your fleet.
An upstream probe layer that queries GitHub, GitLab, and Gitea - hosted and self-hosted - for health signals (archived, stars, last push, latest release, license). Fail-closed by default: only hosts listed in ~/.config/nostos/auth.toml are ever contacted.

Use cases:

Developers with many cloned repos that drift. nostos ~/projects pulls them all in parallel; dirty trees and detached HEADs are reported, never overwritten.
Curated project collections - anyone who ingests new repos from blog posts, papers, or colleague pointers. nostos add <url> clones and captures provenance in one step; nostos triage walks the new intake; nostos list --tag python --untouched-over 90 answers "what have I stopped using?" in milliseconds.
Upstream health monitoring for tracked open-source projects. nostos refresh caches archived status, last push, and latest release; nostos list --upstream-archived surfaces repos whose upstream has been archived or taken down.
Build boxes and mirror hosts maintaining read-only clones. JSON output and deterministic exit codes make nostos safe to embed in cron and CI.
Cross-machine fleet replication. nostos export / nostos import ship your fleet (metadata plus clone-on-import) between machines, with path remapping for cross-OS moves.

Quick start

pipx install nostos                         # or: pip install nostos
nostos completion install                   # enable shell tab-completion

nostos --dry-run                            # preview updates under cwd
nostos ~/projects --workers 16              # pull in parallel

nostos add https://github.com/org/tool.git --tag recon
nostos list --tag recon --untouched-over 90
nostos triage                               # classify new intake

No third-party runtime dependencies beyond argcomplete (required for tab-completion, bundled with every install).

Installation

Requirements

Component	Minimum	Recommended
Python	3.10	3.12+
Git	2.25	2.45.1+ (nostos warns at startup on versions with CVE-2024-32002 / 32004 / 32465)
OS	Linux / macOS / Windows	-

Install

# Isolated (recommended)
pipx install nostos

# System-wide
pip install nostos

# From source
git clone https://github.com/prodrom3/nostos.git
cd nostos && pip install .

Verify with nostos --version and nostos --help.

Shell tab-completion

nostos completion install        # auto-detects your shell
exec $SHELL                      # reload
nostos <TAB><TAB>                # lists all verbs

install is idempotent and writes a managed block into ~/.bashrc, ~/.zshrc, or ~/.config/fish/conf.d/nostos.fish. Remove with nostos completion uninstall. Native Windows shells (PowerShell / cmd) are not supported by argcomplete upstream; use Git Bash or WSL.

Usage

nostos is verb-first: nostos <verb> [args]. An invocation without a verb is an implicit pull, so old scripts and cron jobs keep working.

Verb	Purpose
`pull` (default)	Batch-update discovered repositories.
`add`	Ingest a local path or remote URL into the metadata index.
`list`	Filter and print the repo fleet.
`show`	Print full metadata for one repo.
`tag`	Add or remove tags on a repo.
`tags`	List every tag in the index with attachment counts; optionally prune orphans.
`search`	Free-text search across paths, tags, notes, and upstream descriptions.
`note`	Append a timestamped note.
`triage`	Walk newly-added repos interactively and classify them.
`refresh`	Fetch upstream metadata. Opsec-gated. See docs/upstream-probes.md.
`topics`	Manage topic curation rules (deny / alias) applied when `--auto-tags` imports upstream topics.
`digest`	Weekly changeset report (zero network).
`dashboard`	Render a static HTML fleet health report.
`vault`	Obsidian vault bridge (`export` / `sync`). See docs/vault.md.
`export` / `import`	Portable JSON bundles for cross-machine / backup. See docs/bundle-schema.md.
`update`	Self-update. Auto-detects source / pipx / pip install.
`doctor`	Index integrity check; `--fix` auto-remediates safe issues.
`attack`	MITRE ATT&CK technique lookup + tagging helper.
`completion`	Shell tab-completion setup.
`rm`	Remove a repo from the index (optionally `--purge` the clone).

Every verb has --help. Every verb that lists or returns results supports --json.

Key flags - `nostos pull`

Flag	Default	Description
`path`	cwd	Root directory to scan.
`--from-index`	off	Pull every repo registered in the metadata index.
`--dry-run`	off	List discovered repos without pulling.
`--fetch-only`	off	Fetch from remotes; do not merge or rebase.
`--tags`	off	Also fetch all git tags.
`--rebase`	off	Use `git pull --rebase`.
`--depth N`	5	Directory-scan depth limit.
`--workers N`	8	Concurrent worker threads.
`--timeout N`	120	Seconds before a git operation is killed.
`--exclude PATTERN...`	-	Glob patterns to skip repos by directory name.
`--json`	off	Machine-readable output.
`-q`, `--quiet`	off	Suppress progress; print only the summary.

Every pulled repo is automatically registered in the metadata index with an updated last_touched_at.

Examples

# Daily batch pull
nostos ~/projects

# CI-friendly: quiet, JSON, fetch-only
nostos --fetch-only --quiet --json | jq '.counts'

# Ingest + triage
nostos add https://github.com/org/tool.git --tag recon --source "blog:orange.tw"
nostos triage

# Ingest with auto-fetched repo topics from the upstream host (requires auth.toml)
nostos add https://github.com/r4ulcl/Mythic-OSEP-CheatSheet --tag osep,c2 --auto-tags

# Subdomain-recon tools from the Pentesting/Bug-Bounty Mindmap
nostos add https://github.com/OWASP/Amass            --tag subdomain,vertical-corelation,recon --auto-tags
nostos add https://github.com/aboul3la/Sublist3r     --tag subdomain,recon --auto-tags
nostos add https://github.com/guelfoweb/knock        --tag subdomain,recon --auto-tags
nostos add https://github.com/projectdiscovery/subfinder --tag subdomain,recon --auto-tags

# Bulk-ingest every public repo of a GitHub user / org (requires auth.toml)
nostos add --from-owner projectdiscovery --tag recon --auto-tags --workers 8
nostos add --from-owner tomnomnom --tag hacks --auto-tags --lang go --limit 20
nostos add --from-owner orgname --tag c2 --auto-tags \
    --include-forks --match '^(mythic|caldera)' --clone-dir ~/repos

# DevSecOps queries against the cached fleet (run `nostos refresh --all --cves` first)
nostos list --license MIT,Apache-2.0          # only permissively-licensed repos
nostos list --license-not GPL-3.0,AGPL-3.0    # reject copyleft for our product
nostos list --upstream-cve                    # repos with at least one open advisory
nostos list --upstream-severity high          # high+ severity only

# Refresh and curate concurrently for a big fleet
nostos refresh --all --cves --workers 8       # ~10s for 60 repos vs ~60s serial
nostos topics apply --workers 8

# Free-text search when you don't remember the exact tag
nostos search "csrf bypass"                   # matches notes, descriptions, paths
nostos search projectdiscovery --json | jq '.repositories[].path'

# Backfill topics across the whole fleet on the next refresh
nostos refresh --all --auto-tags

# Curate the imported topics: drop junk, collapse synonyms
nostos topics deny foo hacktoberfest ubuntu
nostos topics alias red-teaming redteam
nostos topics alias penetration-testing pentest
nostos topics list
nostos refresh --all --auto-tags         # re-curate the fleet

# Share rules across machines or with colleagues
nostos topics export > team-rules.toml
nostos topics import team-rules.toml             # default: merge with local rules
nostos topics import team-rules.toml --replace   # overwrite local rules
curl -sS https://example.com/rules.toml | nostos topics import -   # via stdin

# Quick start: import the curated default rule set bundled with nostos
nostos topics import extras/topic_rules/default.toml

# Retroactively curate tags already in the index after editing rules
nostos topics apply --dry-run     # preview what would change
nostos topics apply               # apply for real (idempotent)
nostos topics apply --repo /path/to/one/repo

# Find C2 tools you haven't touched in 90 days
nostos list --tag c2 --untouched-over 90

# Portable backup
nostos export --out fleet.json
nostos import fleet.json --clone-dir ~/repos   # on another machine

Configuration

Optional INI file at ~/.nostosrc. CLI flags always override file values.

[defaults]
depth         = 5
workers       = 8
timeout       = 120
max_log_files = 20
rebase        = false
clone_dir     = /home/user/repos

[exclude]
patterns = archived-*, .backup-*, vendor-*

[add]
auto_tags = false             # default. Set true to make `nostos add` always
                              # fetch repo topics from the upstream host.

Environment variables

Variable	Effect
`NO_COLOR`	Disables ANSI color when set to any non-empty value.
`NOSTOS_SHELL`	Overrides `$SHELL` when detecting the target shell for `nostos completion`. Accepts `bash` / `zsh` / `fish`.
`GITHUB_TOKEN`, etc.	Referenced via `token_env = "..."` in `~/.config/nostos/auth.toml` for upstream probes.
`XDG_CONFIG_HOME`	Relocates `~/.config/nostos/` (default `~/.config`).
`XDG_DATA_HOME`	Relocates `~/.local/share/nostos/` - the index DB and the `logs/` subdirectory (default `~/.local/share`).
`APPDATA` / `LOCALAPPDATA`	Windows fallbacks when the XDG variables are unset: config lives under `%APPDATA%\nostos\`, data under `%LOCALAPPDATA%\nostos\`.

Precedence, highest to lowest: CLI flags -> ~/.nostosrc -> built-in defaults.

Core concepts

Metadata index

SQLite at $XDG_DATA_HOME/nostos/index.db (default ~/.local/share/nostos/index.db, 0600 perms, WAL mode, secure_delete=ON). One row per repository plus tags, timestamped notes, provenance, and triage status. Every verb reads from and writes to this file; the batch pull auto-registers every repo it touches.

Column	Description
`path`	Absolute, `realpath`-normalised. Unique.
`remote_url`	`origin` remote (HTTPS credentials stripped).
`source`	Free-text provenance (`"blog:..."`, `"auto-discovered"`, `"legacy-watchlist"`).
`status`	`new`, `reviewed`, `in-use`, `dropped`, `flagged`.
`quiet`	Opsec flag: never probe upstream for this repo.
`added_at` / `last_touched_at`	ISO-8601 UTC timestamps.
tags / notes	Many-to-many tags; append-only timestamped notes.

For at-rest confidentiality, place $XDG_DATA_HOME/nostos/ on a disk-layer encrypted volume (LUKS / FileVault / BitLocker). nostos ships no built-in DB encryption by design.

Upstream probes

nostos refresh populates cached upstream health (archived, stars, last push, release, license) for each repo, gated by ~/.config/nostos/auth.toml. Unconfigured hosts are never contacted. Fail-closed invariants are documented in docs/upstream-probes.md.

Portable bundles

nostos export / nostos import serialise the metadata index as a schema-versioned JSON bundle. The import path resolves each entry against the local filesystem (direct match, $HOME-relative match, --remap) and clones repos that carry a remote_url but do not exist locally. Full schema and algorithm in docs/bundle-schema.md.

Obsidian vault

nostos vault export turns the index into one markdown file per repo with YAML frontmatter. nostos vault sync reconciles operator edits to status / tags back into the DB. Details and Dataview queries in docs/vault.md.

Output

Human-readable

  [1/9] updated: /home/user/projects/repo-a
  [2/9] up-to-date: /home/user/projects/repo-d
  [3/9] skipped: /home/user/projects/repo-e

--- Summary ---
Updated (3): ...
Skipped (1): /home/user/projects/repo-e - dirty working tree
Total: 9 | Updated: 3 | Up-to-date: 5 | Skipped: 1 | Failed: 0

JSON

nostos --json | jq '.counts'

Progress lines go to stderr; --json output on stdout stays clean for pipes.

CI / automation

Exit code	Meaning
`0`	All discovered repos updated or already up-to-date.
`1`	At least one repo failed to update.

Skipped repositories (dirty, detached, no upstream) do not fail the run - they are surfaced in the summary for review.

# GitHub Actions
- name: Refresh vendored clones
  run: |
    nostos ./vendor --quiet --json > /tmp/nostos.json
    jq '.counts' /tmp/nostos.json

# crontab
*/30 * * * *  /usr/local/bin/nostos ~/projects --quiet --fetch-only

Headless / service-account deployment

For internal mirror boxes, build runners, or cron-driven curation jobs:

No interactive prompts. Every verb is non-interactive when given the inputs it needs. nostos triage is the only exception (interactive by design); skip it on headless boxes and use nostos list --status new + nostos tag from a script instead.
Stable JSON for scripting. pull, list, show, refresh, digest, dashboard, topics list / apply, search, and tags all support --json. Schemas evolve only across major versions.
Predictable exit codes. 0 = success, 1 = at least one repo failed, 2 = usage / invalid args. Skipped repos do not fail the run.
Tokens via environment variables only. ~/.config/nostos/auth.toml references token names (token_env = "GITHUB_TOKEN"); the actual secret lives in the service account's environment, never the repo or the index.
Air-gapped operation. nostos refresh --offline, nostos update --offline, nostos add against an already-cloned local path, and every read-only verb work without any network access.
Logs in a known place. $XDG_DATA_HOME/nostos/logs/ (%LOCALAPPDATA%\nostos\logs\ on Windows). Mode 0600. Rotated to the most recent 20 (configurable). Credentials are sanitized before being written.
Index portability. nostos export --redact produces a JSON bundle with notes/source stripped; nostos import rebuilds it on a fresh host. Use this for DR snapshots, cross-region replication, or onboarding a new team member.

# /etc/systemd/system/nostos-pull.service  (oneshot)
[Service]
Type=oneshot
User=nostos
Environment=GITHUB_TOKEN=...
Environment=XDG_DATA_HOME=/var/lib/nostos
Environment=XDG_CONFIG_HOME=/etc/nostos
ExecStart=/usr/local/bin/nostos pull --from-index --quiet --json

# /etc/systemd/system/nostos-pull.timer
[Timer]
OnCalendar=*:0/30          ; every 30 minutes
Persistent=true

[Install]
WantedBy=timers.target

Logging

Each run writes a timestamped log file to $XDG_DATA_HOME/nostos/logs/ (default ~/.local/share/nostos/logs/ on Linux / macOS, %LOCALAPPDATA%\nostos\logs\ on Windows) alongside the metadata index - so logs survive pipx reinstall and are always findable regardless of install method. Example filename: 2026-04-17_14-30-00.log. Rotated to the most recent 20 (configurable via max_log_files). Files are 0600; the logs/ directory is 0700 on Unix. HTTPS credentials of the form https://user:token@host/ are sanitized to https://***@host/ before being written.

Security

nostos treats git operations on untrusted working directories as an attack surface, and the metadata index as an intelligence artifact. Defense-in-depth applies at both layers.

Control	Description
Git version check	Startup warning on git < 2.45.1 (CVE-2024-32002 / 32004 / 32465).
Safe remote clone	Clone paths (`add <url>`, import clone-on-import) validate the URL, pass `--` to git, pin `protocol.ext.allow=never`, clone with `--no-checkout`, and disable hooks via `GIT_CONFIG_*` - blocking option/transport injection plus CVE-2024-32002 / 32004 / 32465.
Credential redaction	HTTPS credentials stripped from all logs and from `remote_url` values in the index.
File permissions	Logs, index DB, and its `-wal`/`-shm` sidecars `0600`; config and data dirs `0700` (Unix).
Ownership checks	`~/.nostosrc`, `auth.toml`, and legacy watchlist rejected if not owned by the invoking user or if group- or world-writable.
Repository ownership	Repos not owned by the current user are skipped on Unix.
Symlink protection	The `logs/` directory is rejected if it is a symlink.
No shell injection	Every subprocess call uses list arguments; `shell=True` is never used.
Index hardening	SQLite `journal_mode=WAL`, `secure_delete=ON`, `foreign_keys=ON`; deleted rows overwritten on disk.
Probe fail-closed	Upstream probes only contact hosts listed in `auth.toml`; `--offline` hard-disables the network layer.
Per-repo quiet flag	`add --quiet-upstream` makes a repo ineligible for upstream probes; the probe layer never queries or logs these repos.
Token hygiene	Tokens sourced from env vars by default, sent as `Authorization: Bearer`, redacted from every log and error path.
Redirect / probe safety	Upstream probes are https-only; `Authorization` / `Cookie` headers are stripped on any cross-host redirect so a token cannot follow a redirect to another host.
Self-update integrity	`update --verify` runs `git verify-tag` and refuses to upgrade on an unverifiable signature (fail-closed; not overridable by `--yes`).
Build & release	OIDC trusted publishing with PEP 740 provenance attestations; third-party Actions SHA-pinned and Dependabot-managed; `bandit`, `pip-audit`, and CodeQL run in CI.

Report security issues privately via a GitHub security advisory. See SECURITY.md for the full disclosure process.

Compatibility

OS	Python 3.10	3.11	3.12	3.13
Ubuntu (latest)	✓	✓	✓	✓
macOS (latest)	✓	✓	✓	✓
Windows (latest)	✓	✓	✓	✓

CI exercises every cell of this matrix on every push and pull request.

Architecture

See docs/architecture.md for the module layout, dependency graph, and end-to-end flows (pull, add -> triage, import).

Versioning & support

nostos follows Semantic Versioning 2.0. Breaking changes appear only in new major versions and are called out in CHANGELOG.md and the corresponding GitHub release notes.

Stable: CLI flags, exit codes, JSON output schema, bundle schema (reader accepts all versions in READABLE_SCHEMAS).
Internal: the core/ Python API is not a supported public API; import at your own risk.

Current version: see VERSION and nostos --version.

Response expectations (best-effort, non-commercial): see MAINTAINERS.md.

Project documents

File	Purpose
CHANGELOG.md	Per-release change log (mirrors GitHub releases).
CONTRIBUTING.md	Dev setup, test / lint / mypy workflow, PR style.
MAINTAINERS.md	Primary maintainer, escalation path, release authority.
SECURITY.md	Private disclosure process.
LICENSE	MIT.

Deep-dive docs under docs/:

File	Topic
docs/architecture.md	Module layout, dependency graph, run / intake / import flows.
docs/upstream-probes.md	Upstream probe auth, commands, opsec invariants.
docs/bundle-schema.md	Portable bundle format v2 and import resolution algorithm.
docs/vault.md	Obsidian vault bridge and narrow two-way sync.

License

Released under the MIT License. Authored by prodrom3; maintained by the radamic organization.

Name		Name	Last commit message	Last commit date
.github		.github
core		core
docs		docs
extras		extras
tests		tests
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
MAINTAINERS.md		MAINTAINERS.md
README.md		README.md
VERSION		VERSION
nostos.py		nostos.py
pyproject.toml		pyproject.toml

Navigation Menu

Repository navigation

About

Repository files navigation

nostos

At a glance

Overview

Quick start

Installation

Requirements

Install

Shell tab-completion

Usage

Key flags - `nostos pull`

Examples

Configuration

Environment variables

Core concepts

Metadata index

Upstream probes

Portable bundles

Obsidian vault

Output

Human-readable

JSON

CI / automation

Headless / service-account deployment

Logging

Security

Compatibility

Architecture

Versioning & support

Project documents

License

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

nostos

At a glance

Overview

Quick start

Installation

Requirements

Install

Shell tab-completion

Usage

Key flags - nostos pull

Examples

Configuration

Environment variables

Core concepts

Metadata index

Upstream probes

Portable bundles

Obsidian vault

Output

Human-readable

JSON

CI / automation

Headless / service-account deployment

Logging

Security

Compatibility

Architecture

Versioning & support

Project documents

License

About

Topics

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

Key flags - `nostos pull`