diff --git a/DESIGN.md b/DESIGN.md index 9c77c18..6f0e4b7 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -1060,3 +1060,69 @@ download that stops in the middle runs nothing. The steps: A trap removes the temporary directory and a new file that was not renamed yet, at every exit. A failure exits with 1, and its last line says that nothing was installed. What the script trusts: THREAT-MODEL.md, T17. + +## 17. Bootstrap (`pullgate bootstrap`) + +`pullgate bootstrap` sets up a new host, as root, after the binary is +installed and the users, the directories and the units are in place +(README "Install"). Its goal: nobody types or pastes the owner ID or the +chat ID, and the token never shows. + +``` +pullgate bootstrap [--token-stdin] [--api-url URL] + --instance NAME --repo URL [--branch B] [--apply PROGRAM] [--known-hosts FILE] + [--instance NAME2 ...] +``` + +It must run as root, and `/etc/pullgate` and its parents must be safe +(rule 6). `--api-url` takes only `https://api.telegram.org` or a test +server on `http://127.0.0.1:PORT` (no user info): another server would see +the token and could plant the IDs. A test server gets a loud warning. +`listener.toml` lists only the instances of this run: give every instance +of the host in each run. + +1. **Checks first.** Every instance name, URL and branch is checked, and + an ssh URL to a host other than `github.com` needs `--known-hosts`. A + new instance (no `NAME.toml`) needs `--apply`. Nothing is asked or + written before these pass. +2. **The token.** It runs `systemd-ask-password` (no echo, not in the + shell history, not an argument), or with `--token-stdin` reads the + first line of stdin. It must look like a bot token. `getMe` checks it. + The token is never printed, never in an error, never an argument, and + never in the environment of a program that bootstrap starts: each one + gets only `PATH` and `LC_ALL`. +3. **Pairing.** It makes a one-time code of 8 random digits, valid for 10 + minutes, and prints it on the console. It skips the updates that came + before the code. It reads messages with `getUpdates`. A message that is + not a number (for example `/start`) is skipped. Any other number is a + wrong code; at 5 wrong codes it stops. The right code (compared in + constant time) counts only from a **private** chat with a person: chat + type `private`, the sender is not a bot, and the sender ID is the chat + ID. The right code from a group or a channel stops it: the code is used + up. Then it writes `bot-token` (root, 0600) and `listener.toml` (root, + group of the listener, 0640; written first) with `owner_id` and `chat_id` from that + message, replacing any file that was there. A wrong code, a group, a + timeout or an error writes nothing; an old `listener.toml` stays, and + bootstrap says that it did not write it. It confirms the updates, so + the listener does not see the code, and sends "paired" to the chat. + There is no flag, config key or variable that sets these IDs. +4. **Each instance.** For an ssh URL: it makes `NAME.key` with `ssh-keygen + -t ed25519` (an existing key is kept only if it is a regular file of + root; a `.pub` without a key is refused), sets it to 0600, and prints the + public key with "add it as a read-only deploy key". It writes + `NAME.known_hosts` from the `github.com` ed25519 key that the binary + carries, or from `--known-hosts FILE`, and prints the SHA256 fingerprint + of each key, to compare with what the git host publishes. A local URL + needs neither. If `NAME.toml` does not exist, it writes a minimal one: + `mode = "ask"`, `on_failure = "pause"`, the repo, the key, the known + hosts, `apply` and `check_user = "pullgate-check-NAME"`. An existing + `NAME.toml` is kept, with its own repo, key and known hosts: then no + key or known hosts file is made, and the probe uses the config. Then it runs `git ls-remote` (section 6: the same git + flags and ssh options) every 10 seconds, for at most 30 minutes, until + the branch can be read. +5. It does not run `pullgate doctor` or enable the units yet (code6x/pullgate#28). + +**pullgate's own files.** `listener.toml`, `bot-token`, `NAME.key`, +`NAME.known_hosts` and the instance `mode` belong to pullgate. A later +config tool, for example Ansible run by an instance, must not write them. +`listener.toml` says so in its first lines. diff --git a/README.md b/README.md index ead15c6..63b493e 100644 --- a/README.md +++ b/README.md @@ -99,19 +99,49 @@ install -d -m 0711 /var/lib/pullgate cp contrib/systemd/* /etc/systemd/system/ ``` -1. Make a Telegram bot with BotFather. Put its token in - `/etc/pullgate/bot-token` (owner root, mode 0600). -2. Find your Telegram user ID and the chat ID of your private chat with the - bot. Write `/etc/pullgate/listener.toml` from - [contrib/examples/listener.toml](contrib/examples/listener.toml). -3. Make a **read-only** deploy key for the repo: - `ssh-keygen -t ed25519 -N '' -f /etc/pullgate/NAME.key`. Add the public key - to the repo as a read-only deploy key. -4. Pin the host key of the git server in `/etc/pullgate/NAME.known_hosts`. - Check the fingerprint against the one your git host publishes. -5. Write `/etc/pullgate/NAME.toml` (owner root, mode 0600). Start from one of - the examples below. -6. Start it: +Then set it up with **`pullgate bootstrap`**, as root, on a console of +the host: + +```sh +pullgate bootstrap --instance config-repo --repo git@github.com:example/config-repo.git \ + --apply /usr/local/lib/config-repo/apply.sh +``` + +1. Make a Telegram bot with BotFather first. Bootstrap asks for its token + with `systemd-ask-password`: it does not show the token, does not keep + it in the shell history, and does not take it as an argument. It checks + the token with Telegram, and stores it as `/etc/pullgate/bot-token` + (owner root, mode 0600). For automation, `--token-stdin` reads it from + the first line of stdin instead. +2. **Pairing.** Bootstrap prints a one-time code of 8 digits, valid for 10 + minutes. Send it to the bot, in your **private** chat with it. + Bootstrap takes your user ID and the chat ID from that message, and + writes `/etc/pullgate/listener.toml`. Nobody types or pastes an ID, so + a script that someone else wrote cannot plant one. A wrong code (5 of + them stop it), a group chat or a timeout writes nothing. Stop + `pullgate-listen.service` while bootstrap runs, if it is running. +3. For each `--instance NAME --repo URL` (and `--branch`, default `main`): + it makes the deploy key on the host (`/etc/pullgate/NAME.key`; the + private key never leaves the host) and prints the public key. Add it + to the repo as a **read-only** deploy key. It pins the host key of the + git server in `/etc/pullgate/NAME.known_hosts`: for `github.com` from + the key that pullgate carries, with its fingerprint, to compare with the + one GitHub publishes; for another git host from `--known-hosts FILE` + (make it with `ssh-keyscan`, and compare the fingerprints). If + `/etc/pullgate/NAME.toml` does not exist, it writes a minimal one with + `--apply PROGRAM` (owner root, mode 0600); add `check`, `health` and + `watch` as the examples below show. Then it waits until `git ls-remote` + can read the branch. +4. Bootstrap does not enable the units yet. Do it as below. + +Each run writes `listener.toml` with the instances of that run only: give +every instance of the host each time. + +`listener.toml`, the token, the deploy key and the known hosts file are +pullgate's own files: no other config tool (for example Ansible, run by +an instance) may write them. + +Start it: ```sh diff --git a/THREAT-MODEL.md b/THREAT-MODEL.md index 727ff24..61bb4bc 100644 --- a/THREAT-MODEL.md +++ b/THREAT-MODEL.md @@ -563,6 +563,43 @@ instance named `pullgate` has its own directories (`TestUpdateQuestionsAreTheirO ship a binary that passes (T16); the owner's tap is the last check. A build from the source (`pullgate dev`) does not update itself. +### T21. Setup: a planted owner ID, the token entry, the pairing code + +*Attack.* The person who sets up a host pastes a script that someone else +wrote. The script writes its own `owner_id` into `listener.toml`, and so +gets the bot and every Apply. Or the token leaks: into the shell history, +a process list, a log, or the environment of a child program. Or someone +else sends the pairing code first, or guesses it. + +*Defense.* `pullgate bootstrap` (DESIGN.md section 17): + +- The IDs come only from the Telegram message that carries the one-time + code. No flag, config key or variable sets them; a planted + `listener.toml` is replaced at the pairing. Bootstrap refuses unknown + options, such as `--owner-id`. `--api-url` takes only the real Bot API + or a test server on `127.0.0.1`: a pasted script cannot send the + pairing through a server that changes the sender ID. The listener's + `api_url` is parsed the same way, so `http://127.0.0.1:80@host` is + refused. +- The code is 8 random digits, valid for 10 minutes, printed only on the + console of the host. 5 wrong codes stop the pairing, so a guess works + with a chance of 5 in 100 million. The right code counts only from a + private chat with a person; from a group it stops the pairing, because + others saw it. A wrong code, a group or a timeout writes nothing. +- The token is read with `systemd-ask-password` (no echo, no history, no + argument) or from stdin. It is checked with `getMe`, stored with mode + 0600, and never printed. Programs that bootstrap starts get only `PATH` + and `LC_ALL`. +- The deploy key is made on the host; only the public key is printed. The + host key of `github.com` comes from the binary, with its fingerprint + shown; another git host needs a file that the owner checked. + +*Left.* Someone who sees the console of the host during the pairing can +send the code first. Check the chat: bootstrap sends "paired" to the chat +that won. A pasted script can still do anything else as root; bootstrap +only takes away the quiet way to plant the owner ID. If the failed run +leaves an old `listener.toml`, bootstrap says that it did not write it. + ## 6. Out of scope - An attacker who is root on the host. diff --git a/cmd/pullgate/main.go b/cmd/pullgate/main.go index 97081ad..8d9dc22 100644 --- a/cmd/pullgate/main.go +++ b/cmd/pullgate/main.go @@ -19,6 +19,7 @@ import ( "time" "github.com/code6x/pullgate" + "github.com/code6x/pullgate/internal/bootstrap" "github.com/code6x/pullgate/internal/config" "github.com/code6x/pullgate/internal/gate" "github.com/code6x/pullgate/internal/listener" @@ -43,6 +44,11 @@ commands: update [--config FILE] look for a new release of pullgate, and install one after a tap (pullgate-update.service runs it, as root) units print the units this binary carries, as JSON + bootstrap [--token-stdin] --instance NAME --repo URL [--branch B] + [--apply PROGRAM] [--known-hosts FILE] [--instance ...] + set up a new host, as root: the bot token (with no echo), + the pairing with the owner's chat by a one-time code, the + deploy key and the host key of each instance version` func main() { @@ -77,7 +83,7 @@ func run(args []string, stdout, stderr io.Writer) int { } rest := fs.Args() if len(rest) == 0 { - return usageErr(stderr, "give a command: run, apply, status, listen, update, units or version") + return usageErr(stderr, "give a command: run, apply, status, listen, update, units, bootstrap or version") } log := slog.New(slog.NewTextHandler(stderr, nil)) ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) @@ -129,6 +135,21 @@ func run(args []string, stdout, stderr io.Writer) int { } return 0 } + case "bootstrap": + o, err := bootstrap.Parse(rest[1:]) + if err != nil { + return usageErr(stderr, err.Error()) + } + o.EtcDir, o.Out, o.Err = *configDir, stdout, stderr + if euid != 0 { + log.Error("bootstrap", "err", "it must run as root. Fix: run it with sudo, or as root") + return 1 + } + if err := bootstrap.Run(ctx, o); err != nil { + log.Error("bootstrap", "err", err) + return 1 + } + return 0 case "listen": lf := flag.NewFlagSet("listen", flag.ContinueOnError) lf.SetOutput(io.Discard) diff --git a/internal/bootstrap/bootstrap.go b/internal/bootstrap/bootstrap.go new file mode 100644 index 0000000..4585db3 --- /dev/null +++ b/internal/bootstrap/bootstrap.go @@ -0,0 +1,625 @@ +// Package bootstrap is "pullgate bootstrap": it reads the bot token +// without echo, pairs the owner's private chat with a one-time code, and +// sets up the deploy key, the host key and a minimal config of each +// instance (DESIGN.md section 17). The owner and chat IDs come only from +// the Telegram message that carries the code: no flag, file or variable +// sets them. +package bootstrap + +import ( + "bufio" + "bytes" + "context" + "crypto/rand" + "crypto/sha256" + "crypto/subtle" + "encoding/base64" + "errors" + "fmt" + "io" + "math/big" + "net/url" + "os" + "os/exec" + "os/user" + "path/filepath" + "regexp" + "strconv" + "strings" + "syscall" + "time" + + "github.com/BurntSushi/toml" + + "github.com/code6x/pullgate/internal/config" + "github.com/code6x/pullgate/internal/gitx" + "github.com/code6x/pullgate/internal/safedir" + "github.com/code6x/pullgate/internal/state" + "github.com/code6x/pullgate/internal/telegram" +) + +// Instance is one --instance with its options. +type Instance struct { + Name, URL, Branch string + KnownHosts string // a file with the host keys, for a git host other than GitHub + Apply string // the apply program, for a new config +} + +// Options are the inputs of a bootstrap. Parse makes them from the +// command line; tests set the program paths and the times. +type Options struct { + EtcDir string + APIURL string + TokenStdin bool + Instances []Instance + AskPassword []string // reads the token from the console, with no echo + SSHKeygen string + ListenerGroup string + PairTTL time.Duration + RemoteWait time.Duration + RemoteEvery time.Duration + PollSeconds int + Stdin io.Reader + Out io.Writer // the console: the code, the keys, the steps + Err io.Writer // the stderr of systemd-ask-password + Now func() time.Time +} + +// DefaultAPI is the Bot API of Telegram. +const DefaultAPI = "https://api.telegram.org" + +// Usage is the help text of the command. +const Usage = `pullgate bootstrap [--token-stdin] [--api-url URL] + --instance NAME --repo URL [--branch B] [--apply PROGRAM] [--known-hosts FILE] + [--instance NAME2 ...]` + +// Parse reads the arguments after "bootstrap". There is no flag for the +// owner or the chat ID, on purpose. +func Parse(args []string) (*Options, error) { + o := &Options{ + EtcDir: "/etc/pullgate", APIURL: DefaultAPI, + AskPassword: []string{"systemd-ask-password", "--timeout=0", "Telegram bot token (from BotFather):"}, + SSHKeygen: "ssh-keygen", + ListenerGroup: "pullgate", + PairTTL: 10 * time.Minute, RemoteWait: 30 * time.Minute, RemoteEvery: 10 * time.Second, + PollSeconds: 25, + Stdin: os.Stdin, Out: io.Discard, Err: io.Discard, Now: time.Now, + } + var cur *Instance + for i := 0; i < len(args); i++ { + a := args[i] + if a == "--token-stdin" { + o.TokenStdin = true + continue + } + if i+1 >= len(args) { + return nil, fmt.Errorf("%s: a value is missing, or the option is unknown. Fix: see pullgate --help", a) + } + v := args[i+1] //nolint:gosec // checked just above + i++ + switch a { + case "--api-url": + o.APIURL = v + case "--instance": + o.Instances = append(o.Instances, Instance{Name: v, Branch: "main"}) + cur = &o.Instances[len(o.Instances)-1] + case "--repo", "--branch", "--apply", "--known-hosts": + if cur == nil { + return nil, fmt.Errorf("%s comes before any --instance. Fix: give --instance NAME first", a) + } + switch a { + case "--repo": + cur.URL = v + case "--branch": + cur.Branch = v + case "--apply": + cur.Apply = v + default: + cur.KnownHosts = v + } + default: + return nil, fmt.Errorf("%s: unknown option. The owner and chat IDs cannot be given: the pairing takes them from Telegram. Fix: see pullgate --help", a) + } + } + return o, nil +} + +func (o *Options) say(format string, a ...any) { _, _ = fmt.Fprintf(o.Out, format+"\n", a...) } + +// childEnv is the whole environment of every program bootstrap starts. +// It never holds the token. +func childEnv() []string { + return []string{"PATH=/usr/sbin:/usr/bin:/sbin:/bin", "LC_ALL=C"} +} + +// check looks at every instance before anything is asked or written. +func (o *Options) check() error { + if len(o.Instances) == 0 { + return errors.New("no instance. Fix: give --instance NAME --repo URL") + } + // Another server would see the token and every message, and could + // plant the IDs. Only the real one, or a local fake for tests. + if err := config.CheckAPIURL(o.APIURL); err != nil { + return fmt.Errorf("--api-url: %w", err) + } + if o.APIURL != DefaultAPI && !strings.HasPrefix(o.APIURL, "http://127.0.0.1:") { + return fmt.Errorf("--api-url %q: bootstrap talks only to %s, or to a test server on http://127.0.0.1:PORT. Another server would see the token and could plant the owner ID", o.APIURL, DefaultAPI) + } + if err := safedir.CheckParents(o.EtcDir, uint32(os.Geteuid())); err != nil { //nolint:gosec // a uid fits in uint32 + return fmt.Errorf("%s is not safe: %w", o.EtcDir, err) + } + seen := map[string]bool{} + for _, in := range o.Instances { + if !config.ValidName(in.Name) || seen[in.Name] { + return fmt.Errorf("--instance %q: not a valid name, or given twice. Fix: use [a-z0-9][a-z0-9-]{0,31}, once", in.Name) + } + seen[in.Name] = true + local, err := gitx.LocalPath(in.URL) + if err != nil { + return fmt.Errorf("--instance %s --repo: %w", in.Name, err) + } + if !gitx.ValidBranch(in.Branch) { + return fmt.Errorf("--instance %s --branch %q: not a plain branch name", in.Name, in.Branch) + } + if in.Apply != "" && !filepath.IsAbs(in.Apply) { + return fmt.Errorf("--instance %s --apply %q: not an absolute path. Fix: give the full path of the apply program", in.Name, in.Apply) + } + if _, err := os.Lstat(o.instanceConfig(in.Name)); errors.Is(err, os.ErrNotExist) && in.Apply == "" { + return fmt.Errorf("--instance %s: %s does not exist, and --apply is missing. Fix: give --apply /path/of/the/apply/program", in.Name, o.instanceConfig(in.Name)) + } + if local != "" { + continue + } + host, err := sshHost(in.URL) + if err != nil { + return fmt.Errorf("--instance %s --repo: %w", in.Name, err) + } + if in.KnownHosts == "" && host != "github.com" { + return fmt.Errorf("--instance %s: pullgate knows the host keys of github.com only. Fix: run ssh-keyscan %s > FILE, compare the fingerprints with the ones your git host publishes, and give --known-hosts FILE", in.Name, host) + } + } + return nil +} + +func (o *Options) instanceConfig(name string) string { return filepath.Join(o.EtcDir, name+".toml") } + +// Run is one "pullgate bootstrap". +func Run(ctx context.Context, o *Options) error { + if err := o.check(); err != nil { + return err + } + gid, err := o.listenerGID() + if err != nil { + return err + } + token, err := o.readToken(ctx) + if err != nil { + return err + } + tg := telegram.New(o.APIURL, token) + me, err := tg.GetMe(ctx) + if err != nil { + return fmt.Errorf("check the token with getMe: %w. Fix: copy the token from BotFather again", err) + } + if o.APIURL != DefaultAPI { + o.say("WARNING: the Bot API is %s, not %s. That server sees the token and every tap. Use it only for tests.", o.APIURL, DefaultAPI) + } + o.say("The token works: the bot is @%s.", me.Username) + owner, chat, err := o.pair(ctx, tg, me.Username) + if err != nil { + if _, serr := os.Lstat(filepath.Join(o.EtcDir, "listener.toml")); serr == nil { + o.say("Note: an old %s exists. This run did not write it. Remove it if you did not make it.", filepath.Join(o.EtcDir, "listener.toml")) + } + return err + } + if err := o.writeListener(token, owner, chat, gid); err != nil { + return err + } + o.say("Paired. Wrote %s and %s.", filepath.Join(o.EtcDir, "bot-token"), filepath.Join(o.EtcDir, "listener.toml")) + _, _ = tg.SendMessage(ctx, chat, "✅ pullgate bootstrap: this chat is paired. Questions of the instances come here.", nil, 0) + for _, in := range o.Instances { + if err := o.instance(ctx, in); err != nil { + return err + } + } + o.say("Done. Next, as root: systemctl daemon-reload, then enable the units as the README says.") + return nil +} + +func (o *Options) listenerGID() (int, error) { + g, err := user.LookupGroup(o.ListenerGroup) + if err != nil { + return 0, fmt.Errorf("the group %q: %w. Fix: make the users first, as the README says", o.ListenerGroup, err) + } + return strconv.Atoi(g.Gid) +} + +var tokenRE = regexp.MustCompile(`^[0-9]{1,20}:[A-Za-z0-9_-]{20,100}$`) + +// readToken reads the token from systemd-ask-password (no echo, not in +// the shell history, not an argument), or with --token-stdin from the +// first line of stdin. The token is never printed. +func (o *Options) readToken(ctx context.Context) (string, error) { + var raw []byte + if o.TokenStdin { + line, err := bufio.NewReader(io.LimitReader(o.Stdin, 256)).ReadString('\n') + if err != nil && !errors.Is(err, io.EOF) { + return "", fmt.Errorf("read the token from stdin: %w", err) + } + raw = []byte(line) + } else { + cmd := exec.CommandContext(ctx, o.AskPassword[0], o.AskPassword[1:]...) //nolint:gosec // fixed program + cmd.Env = childEnv() + cmd.Stderr = o.Err + var out bytes.Buffer + cmd.Stdout = &out + if err := cmd.Run(); err != nil { + return "", fmt.Errorf("read the token with %s: %w. Fix: run pullgate bootstrap on a console, or use --token-stdin", o.AskPassword[0], err) + } + raw = out.Bytes() + } + tok := strings.TrimSpace(string(raw)) + if !tokenRE.MatchString(tok) { + return "", errors.New("this is not a bot token (it should look like 123456:ABC-...). Fix: copy the token from BotFather again") + } + return tok, nil +} + +// MaxWrong is the most wrong codes before the pairing stops. +const MaxWrong = 5 + +// newCode is 8 random digits. +func newCode() (string, error) { + n, err := rand.Int(rand.Reader, big.NewInt(100000000)) + if err != nil { + return "", err + } + return fmt.Sprintf("%08d", n.Int64()), nil +} + +// pair prints a one-time code and waits for it in a private chat with the +// bot. It returns the user and the chat of the message that carries it. +func (o *Options) pair(ctx context.Context, tg *telegram.Client, bot string) (int64, int64, error) { + code, err := newCode() + if err != nil { + return 0, 0, err + } + if err := tg.DeleteWebhook(ctx); err != nil { + return 0, 0, fmt.Errorf("delete the webhook: %w", err) + } + // Messages from before the code cannot carry it: skip them. + offset := int64(0) + for range 1000 { // a batch has at most 100 updates + old, err := tg.GetUpdates(ctx, offset, 0) + if err != nil { + return 0, 0, fmt.Errorf("read the updates: %w. Fix: stop pullgate-listen.service while bootstrap runs", err) + } + if len(old) == 0 { + break + } + for _, u := range old { + offset = u.UpdateID + 1 + } + } + deadline := o.Now().Add(o.PairTTL) + o.say("Pairing: in Telegram, open a private chat with @%s and send this code:\n\n %s %s\n\nIt works once, until %s. Nobody needs to type an ID.", + bot, code[:4], code[4:], deadline.Format("15:04")) + wrong := 0 + for { + left := deadline.Sub(o.Now()) + if left <= 0 { + return 0, 0, errors.New("the code did not come in time. Nothing was written. Fix: run pullgate bootstrap again") + } + wait := min(o.PollSeconds, int(left/time.Second)) + ups, err := tg.GetUpdates(ctx, offset, wait) + if err != nil { + return 0, 0, fmt.Errorf("read the updates: %w. Nothing was written", err) + } + for _, u := range ups { + offset = u.UpdateID + 1 + m := u.Message + if m == nil { + continue + } + got := strings.Map(func(r rune) rune { + if r == ' ' || r == '-' { + return -1 + } + return r + }, strings.TrimSpace(m.Text)) + if got == "" || strings.Trim(got, "0123456789") != "" { + continue // not a code, for example /start + } + if subtle.ConstantTimeCompare([]byte(got), []byte(code)) != 1 { + wrong++ + if wrong >= MaxWrong { + return 0, 0, fmt.Errorf("%d wrong codes came. Someone else may be trying. Nothing was written. Fix: run pullgate bootstrap again", wrong) + } + continue + } + if m.Chat.Type != "private" || m.From == nil || m.From.IsBot || m.From.ID <= 0 || m.From.ID != m.Chat.ID { + return 0, 0, errors.New("the code came from a group or a channel, not from a private chat. The code is used up and nothing was written. Fix: run pullgate bootstrap again, and send the new code in the private chat with the bot") + } + // Confirm the updates, so the listener does not see the code. + _, _ = tg.GetUpdates(ctx, offset, 0) + return m.From.ID, m.Chat.ID, nil + } + } +} + +// listenerFile is listener.toml as bootstrap writes it. +type listenerFile struct { + OwnerID int64 `toml:"owner_id"` + ChatID int64 `toml:"chat_id"` + Instances []string `toml:"instances"` + APIURL string `toml:"api_url,omitempty"` +} + +const header = "# Written by pullgate bootstrap on %s. pullgate owns this file:\n# no other config tool may write it (DESIGN.md section 17).\n" + +func (o *Options) writeListener(token string, owner, chat int64, gid int) error { + lf := listenerFile{OwnerID: owner, ChatID: chat} + for _, in := range o.Instances { + lf.Instances = append(lf.Instances, in.Name) + } + if o.APIURL != DefaultAPI { + lf.APIURL = o.APIURL + } + var b bytes.Buffer + fmt.Fprintf(&b, header, o.Now().Format("2006-01-02")) + if err := toml.NewEncoder(&b).Encode(lf); err != nil { + return err + } + // listener.toml first: if the token write fails, the listener does not + // start with an old token and the new IDs mixed. + if err := writeOwned(filepath.Join(o.EtcDir, "listener.toml"), b.Bytes(), 0o640, gid); err != nil { + return err + } + return state.WriteFileAtomic(filepath.Join(o.EtcDir, "bot-token"), []byte(token+"\n"), 0o600) +} + +// writeOwned writes a file atomically with its group set before the rename. +func writeOwned(path string, data []byte, mode os.FileMode, gid int) error { + f, err := os.CreateTemp(filepath.Dir(path), ".tmp-") + if err != nil { + return fmt.Errorf("write %s: %w", path, err) + } + tmp := f.Name() + defer func() { _ = os.Remove(tmp) }() + _, werr := f.Write(data) + if werr == nil { + werr = f.Chown(-1, gid) + } + if werr == nil { + werr = f.Chmod(mode) + } + if werr == nil { + werr = f.Sync() + } + if cerr := f.Close(); werr == nil { + werr = cerr + } + if werr != nil { + return fmt.Errorf("write %s: %w", path, werr) + } + if err := os.Rename(tmp, path); err != nil { + return err + } + d, err := os.Open(filepath.Dir(path)) + if err != nil { + return err + } + defer func() { _ = d.Close() }() + return d.Sync() +} + +// GitHubKnownHosts is the ed25519 host key of github.com, as GitHub +// publishes it. Its fingerprint is GitHubFingerprint. +const GitHubKnownHosts = "github.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl\n" + +// GitHubFingerprint is the fingerprint that GitHub publishes for it. +const GitHubFingerprint = "SHA256:+DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU" + +// sshHost is the host of an ssh URL (ssh://[user@]host[:port]/path or +// user@host:path). +func sshHost(raw string) (string, error) { + if strings.HasPrefix(raw, "ssh://") { + u, err := url.Parse(raw) + if err != nil || u.Hostname() == "" { + return "", fmt.Errorf("the url %q has no host", raw) + } + if p := u.Port(); p != "" && p != "22" { + return u.Hostname() + ":" + p, nil + } + return u.Hostname(), nil + } + h := strings.SplitN(raw, ":", 2)[0] + if i := strings.LastIndex(h, "@"); i >= 0 { + h = h[i+1:] + } + if h == "" { + return "", fmt.Errorf("the url %q has no host", raw) + } + return h, nil +} + +// Fingerprints are the SHA256 fingerprints of the keys in a known_hosts +// text, one "type fingerprint" per key. A line that is not a key is an error. +func Fingerprints(text string) ([]string, error) { + var out []string + for _, l := range strings.Split(text, "\n") { + l = strings.TrimSpace(l) + if l == "" || strings.HasPrefix(l, "#") { + continue + } + f := strings.Fields(l) + if len(f) < 3 { + return nil, fmt.Errorf("the line %q is not a known_hosts line (host type key)", l) + } + blob, err := base64.StdEncoding.DecodeString(f[2]) + if err != nil { + return nil, fmt.Errorf("the key of the line for %q is not base64", f[0]) + } + sum := sha256.Sum256(blob) + out = append(out, f[1]+" SHA256:"+base64.RawStdEncoding.EncodeToString(sum[:])) + } + if len(out) == 0 { + return nil, errors.New("no host key") + } + return out, nil +} + +// instance makes the deploy key and the known hosts file, writes a +// minimal config if there is none, and waits until git can read the repo. +func (o *Options) instance(ctx context.Context, in Instance) error { + if _, err := os.Lstat(o.instanceConfig(in.Name)); err == nil { + // A kept config: probe with its own repo, key and known hosts. + c, err := config.Load(o.EtcDir, in.Name, uint32(os.Geteuid())) //nolint:gosec // a uid fits in uint32 + if err != nil { + return err + } + o.say("Instance %s: %s exists; it is kept, with its repo, deploy key and known hosts.", in.Name, c.Path) + return o.waitRemote(ctx, Instance{Name: in.Name, URL: c.Repo.URL, Branch: c.Repo.Branch}, c.Repo.DeployKey, c.Repo.KnownHosts) + } + local, _ := gitx.LocalPath(in.URL) + key, kh := filepath.Join(o.EtcDir, in.Name+".key"), filepath.Join(o.EtcDir, in.Name+".known_hosts") + if local == "" { + if err := o.deployKey(ctx, in.Name, key); err != nil { + return err + } + if err := o.knownHosts(in, kh); err != nil { + return err + } + } else { + key, kh = "", "" + } + if err := o.minimalConfig(in, key, kh); err != nil { + return err + } + return o.waitRemote(ctx, in, key, kh) +} + +func (o *Options) deployKey(ctx context.Context, name, key string) error { + fi, err := os.Lstat(key) + if errors.Is(err, os.ErrNotExist) { + if _, perr := os.Lstat(key + ".pub"); perr == nil { + return fmt.Errorf("%s.pub exists without %s. Fix: remove %s.pub, then run bootstrap again", key, key, key) + } + cmd := exec.CommandContext(ctx, o.SSHKeygen, "-q", "-t", "ed25519", "-N", "", "-C", "pullgate-"+name, "-f", key) //nolint:gosec // fixed program, checked name + cmd.Env = childEnv() + if out, err := cmd.CombinedOutput(); err != nil { + return fmt.Errorf("ssh-keygen for %s: %w: %s", name, err, strings.TrimSpace(string(out))) + } + } else if err != nil { + return err + } else if st, ok := fi.Sys().(*syscall.Stat_t); !fi.Mode().IsRegular() || !ok || st.Uid != uint32(os.Geteuid()) { //nolint:gosec // a uid fits in uint32 + return fmt.Errorf("%s is not a regular file of root. Fix: remove it, then run bootstrap again", key) + } else { + o.say("Instance %s: the deploy key %s exists; it is kept. Remove it to make a new one.", name, key) + } + if err := os.Chmod(key, 0o600); err != nil { + return err + } + pub, err := os.ReadFile(key + ".pub") //nolint:gosec // root's own config directory + if err != nil { + return fmt.Errorf("read the public key: %w", err) + } + o.say("\nInstance %s: add this public key to the repo as a deploy key, READ-ONLY (no write access).\nThe private key stays on this host, in %s.\n\n %s\n", name, key, strings.TrimSpace(string(pub))) + return nil +} + +func (o *Options) knownHosts(in Instance, kh string) error { + text := GitHubKnownHosts + from := "the GitHub key that pullgate carries. Compare it with the fingerprint that GitHub publishes in its docs (\"GitHub's SSH key fingerprints\")" + if in.KnownHosts != "" { + b, err := os.ReadFile(in.KnownHosts) + if err != nil { + return fmt.Errorf("--known-hosts: %w", err) + } + text, from = string(b), in.KnownHosts+". Compare them with the fingerprints that your git host publishes" + } + fps, err := Fingerprints(text) + if err != nil { + return fmt.Errorf("--instance %s known hosts: %w", in.Name, err) + } + if err := state.WriteFileAtomic(kh, []byte(text), 0o600); err != nil { + return err + } + o.say("Instance %s: the host keys come from %s:\n %s", in.Name, from, strings.Join(fps, "\n ")) + return nil +} + +type instanceFile struct { + Mode string `toml:"mode"` + OnFailure string `toml:"on_failure"` + Repo struct { + URL string `toml:"url"` + Branch string `toml:"branch"` + DeployKey string `toml:"deploy_key,omitempty"` + KnownHosts string `toml:"known_hosts,omitempty"` + } `toml:"repo"` + Commands struct { + Apply []string `toml:"apply"` + CheckUser string `toml:"check_user"` + } `toml:"commands"` +} + +func (o *Options) minimalConfig(in Instance, key, kh string) error { + p := o.instanceConfig(in.Name) + if _, err := os.Lstat(p); err == nil { + o.say("Instance %s: %s exists; it is kept.", in.Name, p) + return nil + } + var f instanceFile + f.Mode, f.OnFailure = config.ModeAsk, config.OnFailurePause + f.Repo.URL, f.Repo.Branch, f.Repo.DeployKey, f.Repo.KnownHosts = in.URL, in.Branch, key, kh + f.Commands.Apply, f.Commands.CheckUser = []string{in.Apply}, "pullgate-check-"+in.Name + var b bytes.Buffer + fmt.Fprintf(&b, header, o.Now().Format("2006-01-02")) + if err := toml.NewEncoder(&b).Encode(f); err != nil { + return err + } + if err := state.WriteFileAtomic(p, b.Bytes(), 0o600); err != nil { + return err + } + o.say("Instance %s: wrote a minimal %s. Add check, health and watch as the README says.", in.Name, p) + return nil +} + +// waitRemote waits until git ls-remote can read the branch: the owner +// must add the deploy key first. +func (o *Options) waitRemote(ctx context.Context, in Instance, key, kh string) error { + tmp, err := os.MkdirTemp("", "pullgate-bootstrap-") + if err != nil { + return err + } + defer func() { _ = os.RemoveAll(tmp) }() + g := &gitx.Git{Dir: filepath.Join(tmp, "probe.git"), Home: tmp, UID: uint32(os.Geteuid()), Key: key, KnownHosts: kh, Timeout: time.Minute} //nolint:gosec // a uid fits in uint32 + if err := g.Init(ctx); err != nil { + return err + } + until := o.Now().Add(o.RemoteWait) + said := false + for { + out, err := g.Run(ctx, "ls-remote", "--heads", "--", in.URL, "refs/heads/"+in.Branch) + if err == nil && len(bytes.TrimSpace(out)) > 0 { + o.say("Instance %s: git can read the branch %s.", in.Name, in.Branch) + return nil + } + if err == nil { + err = fmt.Errorf("the branch %s does not exist", in.Branch) + } + if !o.Now().Before(until) { + return fmt.Errorf("instance %s: git cannot read the repo: %w. Fix: add the deploy key, check the url and the branch, then run pullgate bootstrap again", in.Name, err) + } + if !said { + o.say("Instance %s: waiting until git can read the repo (add the deploy key now)…", in.Name) + said = true + } + select { + case <-ctx.Done(): + return ctx.Err() + case <-time.After(o.RemoteEvery): + } + } +} diff --git a/internal/bootstrap/bootstrap_test.go b/internal/bootstrap/bootstrap_test.go new file mode 100644 index 0000000..0898271 --- /dev/null +++ b/internal/bootstrap/bootstrap_test.go @@ -0,0 +1,412 @@ +package bootstrap + +import ( + "bytes" + "context" + "os" + "os/user" + "path/filepath" + "regexp" + "strconv" + "strings" + "sync" + "testing" + "time" + + "github.com/BurntSushi/toml" + + "github.com/code6x/pullgate/internal/config" + "github.com/code6x/pullgate/internal/telegram/faketg" + "github.com/code6x/pullgate/internal/testgit" +) + +// The token is made by the test. +const token = "123456:test-token-made-by-the-test-only" + +const owner = 4242 + +type syncBuf struct { + mu sync.Mutex + b bytes.Buffer +} + +func (s *syncBuf) Write(p []byte) (int, error) { + s.mu.Lock() + defer s.mu.Unlock() + return s.b.Write(p) +} + +func (s *syncBuf) String() string { + s.mu.Lock() + defer s.mu.Unlock() + return s.b.String() +} + +type fx struct { + t *testing.T + o *Options + tg *faketg.Server + out *syncBuf + etc string + r *testgit.Remote +} + +func newFx(t *testing.T) *fx { + t.Helper() + testgit.NeedGit(t) + g, err := user.LookupGroupId(strconv.Itoa(os.Getgid())) + if err != nil { + t.Fatal(err) + } + f := &fx{t: t, tg: faketg.New(t, token), out: &syncBuf{}, etc: testgit.PrivateDir(t), r: testgit.New(t)} + f.r.Commit("first", map[string]string{"value": "one\n"}) + o, err := Parse([]string{"--token-stdin", "--api-url", f.tg.URL, "--instance", "config-repo", "--repo", f.r.Bare, "--apply", "/usr/local/lib/example-apply"}) + if err != nil { + t.Fatal(err) + } + o.EtcDir, o.ListenerGroup = f.etc, g.Name + o.Stdin, o.Out = strings.NewReader(token+"\n"), f.out + o.PairTTL, o.PollSeconds, o.RemoteWait, o.RemoteEvery = 20*time.Second, 1, 0, 100*time.Millisecond + f.o = o + return f +} + +var codeRE = regexp.MustCompile(`\n (\d{4}) (\d{4})\n`) + +// start runs bootstrap and returns the code it prints and the result. +func (f *fx) start() (string, <-chan error) { + f.t.Helper() + done := make(chan error, 1) + go func() { done <- Run(context.Background(), f.o) }() + for i := 0; i < 200; i++ { + if m := codeRE.FindStringSubmatch(f.out.String()); m != nil { + return m[1] + m[2], done + } + select { + case err := <-done: + f.t.Fatalf("bootstrap ended before the code: %v\n%s", err, f.out.String()) + case <-time.After(50 * time.Millisecond): + } + } + f.t.Fatalf("no code printed:\n%s", f.out.String()) + return "", nil +} + +func wait(t *testing.T, done <-chan error) error { + t.Helper() + select { + case err := <-done: + return err + case <-time.After(30 * time.Second): + t.Fatal("bootstrap did not end") + return nil + } +} + +func (f *fx) listener() config.Listener { + f.t.Helper() + var l config.Listener + if _, err := toml.DecodeFile(filepath.Join(f.etc, "listener.toml"), &l); err != nil { + f.t.Fatal(err) + } + return l +} + +func (f *fx) nothingWritten() { + f.t.Helper() + for _, n := range []string{"listener.toml", "bot-token", "config-repo.toml"} { + if _, err := os.Lstat(filepath.Join(f.etc, n)); err == nil { + f.t.Fatalf("%s was written", n) + } + } +} + +func TestPairRightCode(t *testing.T) { + f := newFx(t) + code, done := f.start() + f.tg.Say(owner, owner, "/start", false) // not a code: it does not count + f.tg.Say(owner, owner, code[:4]+"-"+code[4:], false) + if err := wait(t, done); err != nil { + t.Fatalf("bootstrap: %v\n%s", err, f.out.String()) + } + l := f.listener() + if l.OwnerID != owner || l.ChatID != owner || len(l.Instances) != 1 || l.Instances[0] != "config-repo" || l.APIURL != f.tg.URL { + t.Fatalf("listener.toml: %+v", l) + } + t.Setenv("CREDENTIALS_DIRECTORY", f.etc) // as the unit gives it + if _, err := config.LoadListener(filepath.Join(f.etc, "listener.toml"), uint32(os.Geteuid())); err != nil { + t.Fatalf("the listener cannot read it: %v", err) + } + fi, err := os.Stat(filepath.Join(f.etc, "listener.toml")) + if err != nil || fi.Mode().Perm() != 0o640 { + t.Fatalf("listener.toml mode: %v %v", fi.Mode(), err) + } + b, err := os.ReadFile(filepath.Join(f.etc, "bot-token")) + fi2, _ := os.Stat(filepath.Join(f.etc, "bot-token")) + if err != nil || strings.TrimSpace(string(b)) != token || fi2.Mode().Perm() != 0o600 { + t.Fatal("the token file is wrong") + } + if _, err := config.Load(f.etc, "config-repo", uint32(os.Geteuid())); err != nil { + t.Fatalf("the minimal instance config does not load: %v", err) + } + if !strings.Contains(f.out.String(), "git can read the branch main") { + t.Fatalf("no ls-remote:\n%s", f.out.String()) + } + if ms := f.tg.Messages(); len(ms) != 1 || ms[0].Chat != owner || !strings.Contains(ms[0].Text, "paired") { + t.Fatalf("messages: %+v", ms) + } + if strings.Contains(f.out.String(), token) { + t.Fatal("the token was printed") + } +} + +func TestPairWrongCode(t *testing.T) { + f := newFx(t) + code, done := f.start() + wrong := "00000000" + if code == wrong { + wrong = "11111111" + } + for i := 0; i < MaxWrong; i++ { + f.tg.Say(666, 666, wrong, false) + } + f.tg.Say(owner, owner, code, false) // too late: the pairing stopped + err := wait(t, done) + if err == nil || !strings.Contains(err.Error(), "wrong codes") { + t.Fatalf("want the wrong codes error, got %v", err) + } + f.nothingWritten() +} + +func TestPairGroupChat(t *testing.T) { + f := newFx(t) + code, done := f.start() + f.tg.Say(owner, -1001, code, false) + err := wait(t, done) + if err == nil || !strings.Contains(err.Error(), "not from a private chat") { + t.Fatalf("want the group error, got %v", err) + } + f.nothingWritten() +} + +func TestPairTimeout(t *testing.T) { + f := newFx(t) + f.o.PairTTL = 2 * time.Second + _, done := f.start() + err := wait(t, done) + if err == nil || !strings.Contains(err.Error(), "did not come in time") { + t.Fatalf("want the timeout, got %v", err) + } + f.nothingWritten() +} + +func TestPlantedIDsAreNotUsed(t *testing.T) { + for _, flag := range []string{"--owner-id", "--chat-id", "--owner_id", "--config"} { + if _, err := Parse([]string{flag, "666", "--instance", "config-repo", "--repo", "/srv/config-repo.git"}); err == nil { + t.Fatalf("%s must be refused", flag) + } + } + f := newFx(t) + t.Setenv("PULLGATE_OWNER_ID", "666") + planted := "owner_id = 666\nchat_id = 666\ninstances = [\"config-repo\"]\n" + if err := os.WriteFile(filepath.Join(f.etc, "listener.toml"), []byte(planted), 0o600); err != nil { + t.Fatal(err) + } + code, done := f.start() + f.tg.Say(owner, owner, code, false) + if err := wait(t, done); err != nil { + t.Fatal(err) + } + if l := f.listener(); l.OwnerID != owner || l.ChatID != owner { + t.Fatalf("the planted IDs stayed: %+v", l) + } +} + +func TestPlantedFileStaysWhenPairingFails(t *testing.T) { + f := newFx(t) + f.o.PairTTL = time.Second + if err := os.WriteFile(filepath.Join(f.etc, "listener.toml"), []byte("owner_id = 666\n"), 0o600); err != nil { + t.Fatal(err) + } + _, done := f.start() + if err := wait(t, done); err == nil { + t.Fatal("want an error") + } + if !strings.Contains(f.out.String(), "This run did not write it") { + t.Fatalf("no note on the old file:\n%s", f.out.String()) + } +} + +// script writes a program that logs its arguments and environment. +func script(t *testing.T, dir, name, body string) string { + t.Helper() + p := filepath.Join(dir, name) + if err := os.WriteFile(p, []byte("#!/bin/sh\necho \"args: $*\" >> "+dir+"/log\nenv >> "+dir+"/log\n"+body), 0o700); err != nil { + t.Fatal(err) + } + return p +} + +func TestTokenNeverLeaks(t *testing.T) { + f := newFx(t) + bin := testgit.PrivateDir(t) + f.o.TokenStdin = false + f.o.AskPassword = []string{script(t, bin, "ask", "echo '"+token+"'\n"), "Telegram bot token:"} + f.o.SSHKeygen = script(t, bin, "ssh-keygen", `for a; do f=$a; done +echo private > "$f"; echo "ssh-ed25519 AAAAexample pullgate" > "$f.pub" +`) + code, done := f.start() + f.tg.Say(owner, owner, code, false) + if err := wait(t, done); err != nil { + t.Fatalf("bootstrap: %v\n%s", err, f.out.String()) + } + if err := f.o.deployKey(context.Background(), "config-repo", filepath.Join(f.etc, "config-repo.key")); err != nil { + t.Fatal(err) + } + log, err := os.ReadFile(filepath.Join(bin, "log")) + if err != nil { + t.Fatal(err) + } + if !strings.Contains(string(log), "args: -q -t ed25519") || !strings.Contains(string(log), "args: Telegram bot token:") { + t.Fatalf("the programs did not run:\n%s", log) + } + if strings.Contains(string(log), "test-token") { + t.Fatal("the token is in the arguments or the environment of a child") + } + if strings.Contains(f.out.String(), "test-token") { + t.Fatal("the token was printed") + } + for _, kv := range os.Environ() { + if strings.Contains(kv, "test-token") { + t.Fatal("the token is in the environment") + } + } + if !strings.Contains(f.out.String(), "READ-ONLY") || !strings.Contains(f.out.String(), "ssh-ed25519 AAAAexample") { + t.Fatalf("the public key is not shown:\n%s", f.out.String()) + } +} + +func TestBadToken(t *testing.T) { + f := newFx(t) + f.o.Stdin = strings.NewReader("not a token at all\n") + err := Run(context.Background(), f.o) + if err == nil || !strings.Contains(err.Error(), "not a bot token") || strings.Contains(err.Error(), "not a token at all") { + t.Fatalf("bad token: %v", err) + } + f.o.Stdin = strings.NewReader("123456:wrong-token-made-by-the-test-only\n") + if err := Run(context.Background(), f.o); err == nil || !strings.Contains(err.Error(), "getMe") || strings.Contains(err.Error(), "wrong-token") { + t.Fatalf("a token that getMe refuses: %v", err) + } + f.nothingWritten() +} + +func TestGitHubKey(t *testing.T) { + fps, err := Fingerprints(GitHubKnownHosts) + if err != nil || len(fps) != 1 || fps[0] != "ssh-ed25519 "+GitHubFingerprint { + t.Fatalf("the carried GitHub key does not match its published fingerprint: %v %v", fps, err) + } + f := newFx(t) + kh := filepath.Join(f.etc, "x.known_hosts") + if err := f.o.knownHosts(Instance{Name: "config-repo"}, kh); err != nil { + t.Fatal(err) + } + if b, _ := os.ReadFile(kh); string(b) != GitHubKnownHosts { + t.Fatal("known_hosts is not the GitHub key") + } + if !strings.Contains(f.out.String(), GitHubFingerprint) { + t.Fatal("the fingerprint is not shown") + } + bad := filepath.Join(f.etc, "bad") + _ = os.WriteFile(bad, []byte("git.example.com ssh-ed25519 !!!\n"), 0o600) + if err := f.o.knownHosts(Instance{Name: "config-repo", KnownHosts: bad}, kh); err == nil { + t.Fatal("a bad known_hosts file must be refused") + } +} + +func TestOtherHostNeedsKnownHosts(t *testing.T) { + f := newFx(t) + f.o.Instances[0].URL = "ssh://git@git.example.com:2222/config-repo.git" + err := f.o.check() + if err == nil || !strings.Contains(err.Error(), "ssh-keyscan git.example.com:2222") { + t.Fatalf("want a known hosts error: %v", err) + } + f.o.Instances[0].URL = "git@github.com:example/config-repo.git" + if err := f.o.check(); err != nil { + t.Fatalf("github.com: %v", err) + } + f.o.Instances = nil + if err := f.o.check(); err == nil { + t.Fatal("no instance must be refused") + } +} + +func TestWaitRemote(t *testing.T) { + f := newFx(t) + later := filepath.Join(filepath.Dir(f.r.Bare), "later.git") + in := Instance{Name: "config-repo", URL: later, Branch: "main"} + if err := f.o.waitRemote(context.Background(), in, "", ""); err == nil || !strings.Contains(err.Error(), "git cannot read the repo") { + t.Fatalf("a missing repo: %v", err) + } + f.o.RemoteWait = 20 * time.Second + go func() { + time.Sleep(500 * time.Millisecond) + _ = os.Rename(f.r.Bare, later) + }() + if err := f.o.waitRemote(context.Background(), in, "", ""); err != nil { + t.Fatalf("the repo came later: %v", err) + } + if !strings.Contains(f.out.String(), "waiting until git can read the repo") { + t.Fatal("no waiting note") + } +} + +func TestAPIURLOnlyTelegramOrLoopback(t *testing.T) { + f := newFx(t) + for _, u := range []string{"https://tg-mirror.example", "http://127.0.0.1:80@evil.example", "http://192.0.2.1:8080", "https://user@api.telegram.org"} { + f.o.APIURL = u + if err := f.o.check(); err == nil { + t.Fatalf("%s must be refused", u) + } + } + f.o.APIURL = DefaultAPI + if err := f.o.check(); err != nil { + t.Fatalf("the real API: %v", err) + } +} + +func TestKeptConfigIsProbedWithItsOwnRepo(t *testing.T) { + f := newFx(t) + code, done := f.start() + f.tg.Say(owner, owner, code, false) + if err := wait(t, done); err != nil { + t.Fatal(err) + } + // A second run: the config is kept, the --repo of the run is not used. + f.out = &syncBuf{} + f.o.Out, f.o.Stdin = f.out, strings.NewReader(token+"\n") + f.o.Instances[0].URL = "/nonexistent/other.git" + code, done = f.start() + f.tg.Say(owner, owner, code, false) + if err := wait(t, done); err != nil { + t.Fatalf("second run: %v\n%s", err, f.out.String()) + } + if !strings.Contains(f.out.String(), "it is kept, with its repo") { + t.Fatalf("not kept:\n%s", f.out.String()) + } +} + +func TestExistingKeyMustBeRootsFile(t *testing.T) { + f := newFx(t) + key := filepath.Join(f.etc, "config-repo.key") + if err := os.Symlink("/etc/hostname", key); err != nil { + t.Fatal(err) + } + if err := f.o.deployKey(context.Background(), "config-repo", key); err == nil { + t.Fatal("a symlink as the key must be refused") + } + _ = os.Remove(key) + _ = os.WriteFile(key+".pub", []byte("ssh-ed25519 AAAA x\n"), 0o600) + if err := f.o.deployKey(context.Background(), "config-repo", key); err == nil || !strings.Contains(err.Error(), ".pub exists") { + t.Fatalf("a stray .pub: %v", err) + } +} diff --git a/internal/config/config.go b/internal/config/config.go index 2e5176d..3f80176 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -6,6 +6,7 @@ package config import ( "errors" "fmt" + "net/url" "os" "path/filepath" "regexp" @@ -406,8 +407,9 @@ func loadListener(path string, configUID uint32, needToken bool) (*Listener, err return nil, fmt.Errorf("%s: chat_id: it is missing. Fix: set the chat ID (the private chat with the owner)", path) case len(c.Instances) == 0: return nil, fmt.Errorf("%s: instances: the list is empty. Fix: list the instance names", path) - case !strings.HasPrefix(c.APIURL, "https://") && !strings.HasPrefix(c.APIURL, "http://127.0.0.1:"): - return nil, fmt.Errorf("%s: api_url: it must be an https:// url", path) + } + if err := CheckAPIURL(c.APIURL); err != nil { + return nil, fmt.Errorf("%s: api_url: %w", path, err) } for _, n := range c.Instances { if !ValidName(n) { @@ -423,6 +425,24 @@ func loadListener(path string, configUID uint32, needToken bool) (*Listener, err return c, nil } +// CheckAPIURL checks the URL of the Bot API: https:// with a host, or +// http://127.0.0.1:PORT for tests. No user info, which could hide another +// host (http://127.0.0.1:80@host.example). +func CheckAPIURL(s string) error { + u, err := url.Parse(s) + switch { + case err != nil: + return fmt.Errorf("%q is not a url. Fix: use https://api.telegram.org", s) + case u.User != nil || u.Opaque != "" || u.RawQuery != "" || u.Fragment != "": + return fmt.Errorf("%q has user info, a query or a fragment. Fix: use https://api.telegram.org", s) + case u.Scheme == "https" && u.Hostname() != "": + return nil + case u.Scheme == "http" && u.Hostname() == "127.0.0.1" && u.Port() != "": + return nil + } + return fmt.Errorf("%q: it must be an https:// url. Fix: use https://api.telegram.org", s) +} + // ReadToken reads the bot token. The file must be owned by uid (the // listener user) and readable by nobody else. The token is never printed. func (c *Listener) ReadToken(uid uint32) (string, error) { diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 399defa..6fbf3ae 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -232,3 +232,16 @@ func TestListenerUpdates(t *testing.T) { } } } + +func TestCheckAPIURL(t *testing.T) { + for _, u := range []string{"https://api.telegram.org", "http://127.0.0.1:8081"} { + if err := CheckAPIURL(u); err != nil { + t.Fatalf("%s: %v", u, err) + } + } + for _, u := range []string{"http://127.0.0.1:80@evil.example", "http://127.0.0.1", "http://api.telegram.org", "https://u:p@api.telegram.org", "https://", "ftp://x"} { + if err := CheckAPIURL(u); err == nil { + t.Fatalf("%s must be refused", u) + } + } +} diff --git a/internal/telegram/faketg/faketg.go b/internal/telegram/faketg/faketg.go index 3af8e41..1b981f0 100644 --- a/internal/telegram/faketg/faketg.go +++ b/internal/telegram/faketg/faketg.go @@ -8,6 +8,7 @@ import ( "strings" "sync" "testing" + "time" "github.com/code6x/pullgate/internal/telegram" ) @@ -59,6 +60,8 @@ func (s *Server) handle(w http.ResponseWriter, r *http.Request) { num := func(k string) int64 { f, _ := req[k].(float64); return int64(f) } str := func(k string) string { v, _ := req[k].(string); return v } switch method { + case "getMe": + writeJSON(w, map[string]any{"ok": true, "result": telegram.User{ID: 1, IsBot: true, Username: "example_bot"}}) case "deleteWebhook": writeJSON(w, map[string]any{"ok": true, "result": true}) case "getUpdates": @@ -99,6 +102,7 @@ func (s *Server) handle(w http.ResponseWriter, r *http.Request) { select { case <-s.wake: case <-r.Context().Done(): + case <-time.After(time.Duration(num("timeout")) * time.Second): } s.mu.Lock() for _, u := range s.updates { @@ -193,7 +197,11 @@ func (s *Server) Tap(from, chat, msg int64, data string) { func (s *Server) Say(from, chat int64, text string, edited bool) { s.mu.Lock() s.nextID++ - m := &telegram.Message{MessageID: s.nextID, From: &telegram.User{ID: from}, Chat: telegram.Chat{ID: chat}, Text: text} + typ := "private" + if chat != from { + typ = "supergroup" + } + m := &telegram.Message{MessageID: s.nextID, From: &telegram.User{ID: from}, Chat: telegram.Chat{ID: chat, Type: typ}, Text: text} u := telegram.Update{UpdateID: s.nextUp, Message: m} if edited { m.EditDate = 3 diff --git a/internal/telegram/telegram.go b/internal/telegram/telegram.go index 64688c1..8161543 100644 --- a/internal/telegram/telegram.go +++ b/internal/telegram/telegram.go @@ -33,12 +33,15 @@ func (c *Client) String() string { return "telegram.Client{token: hidden}" } // User is a Telegram user. type User struct { - ID int64 `json:"id"` + ID int64 `json:"id"` + IsBot bool `json:"is_bot,omitempty"` + Username string `json:"username,omitempty"` } // Chat is a Telegram chat. type Chat struct { - ID int64 `json:"id"` + ID int64 `json:"id"` + Type string `json:"type,omitempty"` // "private", "group", "supergroup" or "channel" } // Message is a Telegram message. @@ -150,6 +153,15 @@ func Clip(s string) string { return s } +// GetMe returns the bot itself. It checks the token. +func (c *Client) GetMe(ctx context.Context) (*User, error) { + var u User + if err := c.call(ctx, "getMe", map[string]any{}, &u); err != nil { + return nil, err + } + return &u, nil +} + // DeleteWebhook removes a webhook, so getUpdates works. func (c *Client) DeleteWebhook(ctx context.Context) error { return c.call(ctx, "deleteWebhook", map[string]any{"drop_pending_updates": false}, nil) diff --git a/test/units/faketg.py b/test/units/faketg.py index 91ef8aa..58e20be 100644 --- a/test/units/faketg.py +++ b/test/units/faketg.py @@ -70,7 +70,7 @@ def do_POST(self): "message": { "message_id": 100000 + state["next_update"], "from": {"id": OWNER}, - "chat": {"id": CHAT}, + "chat": {"id": CHAT, "type": "private"}, "text": str(req.get("text", "")), }, }, @@ -83,7 +83,9 @@ def do_POST(self): with lock: with open(LOG, "a", encoding="utf-8") as f: f.write(method + "\n") - if method == "getUpdates": + if method == "getMe": + result = {"id": 1, "is_bot": True, "username": "example_bot"} + elif method == "getUpdates": result = self.updates(int(req.get("offset") or 0)) elif method == "sendMessage": text = str(req.get("text", "")) diff --git a/test/units/full-loop.sh b/test/units/full-loop.sh index d548d4a..f2ed0cc 100644 --- a/test/units/full-loop.sh +++ b/test/units/full-loop.sh @@ -268,18 +268,6 @@ EOF chown root:root "/etc/pullgate/$name.toml" chmod 0600 "/etc/pullgate/$name.toml" -printf '123:loop-test-token-made-by-the-script\n' >/etc/pullgate/bot-token -chown root:root /etc/pullgate/bot-token -chmod 0600 /etc/pullgate/bot-token -cat >/etc/pullgate/listener.toml <"$boot_out" 2>&1 & +boot_pid=$! +code="" +i=0 +until [ -n "$code" ]; do + code="$(sed -n 's/^ \([0-9]\{4\}\) \([0-9]\{4\}\)$/\1\2/p' "$boot_out")" + [ -n "$code" ] && break + i=$((i + 1)) + [ "$i" -lt 60 ] || { + cat "$boot_out" + die "pullgate bootstrap printed no pairing code" + } + sleep 0.5 +done +python3 -I -c 'import json, sys, urllib.request +urllib.request.urlopen(urllib.request.Request("http://127.0.0.1:%s/test/say" % sys.argv[1], data=json.dumps({"text": sys.argv[2]}).encode(), method="POST"), timeout=5)' "$port" "$code" +wait "$boot_pid" || { + cat "$boot_out" + die "pullgate bootstrap failed" +} +cat "$boot_out" +grep -q '^owner_id = 4242$' /etc/pullgate/listener.toml || die "bootstrap did not take owner_id from the pairing" +grep -q '^chat_id = 4242$' /etc/pullgate/listener.toml || die "bootstrap did not take chat_id from the pairing" +want /etc/pullgate/listener.toml "root pullgate 640" +want /etc/pullgate/bot-token "root root 600" +if grep -q loop-test-token "$boot_out"; then + die "bootstrap printed the token" +fi + if [ -n "$update_to" ]; then step "3b. A fake GitHub for /update" old_v="$(pullgate --version | sed 's/^pullgate //')"