Represent attachments as lazy materialization pointers

README.md

@@ -2,8 +2,10 @@
 
 `git-remote-confluence` is a Git remote helper that treats a Confluence page
 tree or space as a Git remote. It imports Confluence storage-format XML and
-attachments into a Git repository and can push committed body updates for
-existing pages back to Confluence.
+attachment metadata pointers into a Git repository. The optional
+`git-confluence` filter materializes page Markdown and attachment bytes in the
+working tree. The helper can push committed body updates for existing pages
+back to Confluence.
 
 The helper is intentionally narrow: Confluence remains the system that owns page
 identity, hierarchy, version numbers, and storage-format XML. Git becomes the
@@ -23,28 +25,40 @@ content changed, checks the Confluence version and storage hash, and updates the
 existing Confluence page body. It refuses to overwrite a page when Confluence no
 longer matches the imported metadata.
 
-Create, delete, move, title, and attachment changes are not pushed yet.
+Git stores each attachment as a small YAML pointer containing its Confluence
+identity and current version. Clone and fetch therefore do not download
+attachment bytes. With the `git-confluence` filter configured, checkout
+downloads the version named by the pointer and exposes it as a normal working
+file. Create, delete, move, title, and attachment changes are not pushed yet.
 
 ## Working With Markdown
 
 Confluence storage-format XML is the synchronized content format. The companion
-`git-confluence` clean/smudge filter makes that practical for day-to-day
-editing:
+`git-confluence` filter makes pages and attachments practical for day-to-day
+use:
 
 - Git stores each page body as Confluence storage-format XML.
 - The working tree can show the same file as Markdown on checkout.
 - `git add` can convert the edited Markdown back to Confluence storage XML.
+- Git stores attachment pointers and can materialize their bytes on checkout.
 
 That gives users Markdown editing while preserving the storage XML that
 Confluence needs for reliable import and push.
 
 The imported repository includes this `.gitattributes` entry:
 
 ```gitattributes
-*.md filter=confluence-storage diff=markdown
+*.md filter=confluence diff=markdown
+**/attachments/** filter=confluence -text
 ```
 
-Configure the `git-confluence` filter before checking out imported files.
+Without the filter, checkout leaves storage XML and attachment pointers intact.
+This is the lightweight default for agents and offline use. Configure the
+filter once with `git confluence install --global` to expose Markdown and
+attachment bytes for normal human use.
+
+The `install` subcommand registers the unified clean/smudge driver in Git
+configuration. It does not install the executable or download attachments.
 
 ## Repository Layout
 
@@ -64,7 +78,7 @@ Child pages are placed under their parent's page-id directory:
 123456789/123456790.yml
 ```
 
-Attachments are downloaded below the page ID that owns them:
+Attachments use their normal filenames below the page ID that owns them:
 
 ```text
 123456789/attachments/diagram.png
@@ -73,6 +87,26 @@ Attachments are downloaded below the page ID that owns them:
 
 Path separators and control characters in attachment names are replaced with
 underscores so an attachment cannot escape its page's `attachments` directory.
+Git stores a canonical YAML pointer at each attachment path. The pointer records
+the source site, page ID, attachment ID, version, filename, size, media type,
+and stable download path. It contains no token and no attachment bytes. The
+configured filter replaces the pointer with that version's bytes in the working
+tree and restores the same pointer on `git add`. Locally modified attachment
+bytes are rejected because attachment push is not supported yet.
+
+Attachment pointers use the `attachment/v1` YAML schema. For example:
+
+```yaml
+version: https://github.com/hkwi/git-remote-confluence/spec/attachment/v1
+source: https://confluence.example.com
+page_id: "123456789"
+attachment_id: "987654321"
+attachment_version: 3
+filename: diagram.png
+size: 42000
+media_type: image/png
+download_path: /download/attachments/123456789/diagram.png
+```
 
 The `.md` file is stored in Git as Confluence storage-format XML. With the
 `git-confluence` filter configured, it is checked out as Markdown and converted
@@ -124,10 +158,11 @@ finds from:
 
 ## Clone
 
-If the `git-confluence` filter is already configured globally, clone with Git's
-explicit remote-helper syntax:
+For human-oriented working trees, configure the unified page and attachment
+filter once, then clone with Git's explicit remote-helper syntax:
 
 ```sh
+git confluence install --global
 CONFLUENCE_PAT=... git clone \
   'confluence::https://confluence.example.com/pages/viewpage.action?pageId=123456789'
 ```
@@ -140,14 +175,32 @@ CONFLUENCE_PAT=... git clone --no-checkout \
   'confluence::https://confluence.example.com/pages/viewpage.action?pageId=123456789' \
   pages
 cd pages
-git config filter.confluence-storage.clean "/path/to/git-confluence/git-confluence clean"
-git config filter.confluence-storage.smudge "/path/to/git-confluence/git-confluence smudge"
-git config filter.confluence-storage.required true
+git confluence install --local
 git checkout
 ```
 
+To keep attachment pointers during checkout while still converting pages, set
+the skip-smudge variable for that command:
+
+```sh
+GIT_CONFLUENCE_SKIP_SMUDGE=1 git checkout
+```
+
+Afterward, materialize all or selected attachments explicitly:
+
+```sh
+git confluence pull
+git confluence pull 123456789/attachments/diagram.png
+```
+
 The remote URL may identify a page by `pageId`, a display page URL, or a
-Confluence space.
+Confluence space. A URL containing both `pageId` and `attachmentId` can still be
+cloned directly when a standalone repository containing the attachment's full
+binary version history is required.
+
+For Confluence Data Center, attachment history is read by following
+`history.previousVersion` through historical content responses. The helper does
+not require a `GET /content/{id}/version` listing endpoint.
 
 ## REST API Path
 

go.mod

@@ -1,3 +1,5 @@
 module github.com/hkwi/git-remote-confluence
 
 go 1.26
+
+require go.yaml.in/yaml/v3 v3.0.5

go.sum

@@ -0,0 +1,2 @@
+go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
+go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=

internal/confluence/attachments.go

@@ -5,19 +5,49 @@ import (
 	"net/url"
 	"sort"
 	"strconv"
+
+	"github.com/hkwi/git-remote-confluence/internal/confluencetypes"
 )
 
 type Attachment struct {
-	ID    string            `json:"id"`
-	Title string            `json:"title"`
-	Links map[string]string `json:"_links"`
+	ID         string                  `json:"id"`
+	Title      string                  `json:"title"`
+	Version    confluencetypes.Version `json:"version"`
+	History    AttachmentHistory       `json:"history"`
+	Extensions AttachmentExtensions    `json:"extensions"`
+	Links      map[string]string       `json:"_links"`
+}
+
+type AttachmentHistory struct {
+	PreviousVersion confluencetypes.Version `json:"previousVersion"`
+}
+
+type AttachmentExtensions struct {
+	FileSize  int64  `json:"fileSize"`
+	MediaType string `json:"mediaType"`
 }
 
 type attachmentListResponse struct {
 	Results []Attachment      `json:"results"`
 	Links   map[string]string `json:"_links"`
 }
 
+func (c *Client) FetchAttachment(attachmentID string) (Attachment, error) {
+	if attachmentID == "" {
+		return Attachment{}, fmt.Errorf("Confluence attachment id is required")
+	}
+	values := url.Values{}
+	values.Set("expand", attachmentVersionExpand())
+	var attachment Attachment
+	if err := c.getJSON(c.apiPath("content/"+url.PathEscape(attachmentID)), values, &attachment); err != nil {
+		return Attachment{}, err
+	}
+	if attachment.ID == "" {
+		return Attachment{}, fmt.Errorf("Confluence attachment response is missing id")
+	}
+	return attachment, nil
+}
+
 func (c *Client) FetchAttachments(pageID string) ([]Attachment, error) {
 	var attachments []Attachment
 	start := 0
@@ -27,6 +57,7 @@ func (c *Client) FetchAttachments(pageID string) ([]Attachment, error) {
 		values := url.Values{}
 		values.Set("start", strconv.Itoa(start))
 		values.Set("limit", strconv.Itoa(limit))
+		values.Set("expand", attachmentVersionExpand())
 
 		var response attachmentListResponse
 		path := c.apiPath("content/" + url.PathEscape(pageID) + "/child/attachment")
@@ -50,12 +81,81 @@ func (c *Client) FetchAttachments(pageID string) ([]Attachment, error) {
 }
 
 func (c *Client) DownloadAttachment(attachment Attachment) ([]byte, error) {
+	return c.DownloadAttachmentVersion(attachment, 0)
+}
+
+func (c *Client) FetchAttachmentVersions(attachment Attachment) ([]confluencetypes.Version, error) {
+	if attachment.ID == "" {
+		return nil, fmt.Errorf("Confluence attachment response is missing id")
+	}
+	if attachment.Version.Number <= 0 {
+		current, err := c.FetchAttachment(attachment.ID)
+		if err != nil {
+			return nil, err
+		}
+		attachment = current
+	}
+
+	versions := []confluencetypes.Version{attachment.Version}
+	seen := map[int]bool{attachment.Version.Number: true}
+	previous := attachment.History.PreviousVersion
+	for previous.Number > 0 {
+		if seen[previous.Number] {
+			return nil, fmt.Errorf("Confluence attachment %s version history contains a cycle at version %d", attachment.ID, previous.Number)
+		}
+		historical, err := c.fetchHistoricalAttachment(attachment.ID, previous.Number)
+		if err != nil {
+			return nil, fmt.Errorf("fetch historical attachment %s version %d: %w", attachment.ID, previous.Number, err)
+		}
+		if historical.Version.Number != previous.Number {
+			return nil, fmt.Errorf("Confluence attachment %s requested version %d but returned version %d", attachment.ID, previous.Number, historical.Version.Number)
+		}
+		versions = append(versions, historical.Version)
+		seen[historical.Version.Number] = true
+		previous = historical.History.PreviousVersion
+	}
+
+	sort.Slice(versions, func(i, j int) bool { return versions[i].Number < versions[j].Number })
+	return versions, nil
+}
+
+func (c *Client) fetchHistoricalAttachment(attachmentID string, version int) (Attachment, error) {
+	values := url.Values{}
+	values.Set("status", "historical")
+	values.Set("version", strconv.Itoa(version))
+	values.Set("expand", attachmentVersionExpand())
+	var attachment Attachment
+	if err := c.getJSON(c.apiPath("content/"+url.PathEscape(attachmentID)), values, &attachment); err != nil {
+		return Attachment{}, err
+	}
+	if attachment.ID == "" {
+		return Attachment{}, fmt.Errorf("Confluence historical attachment response is missing id")
+	}
+	return attachment, nil
+}
+
+func attachmentVersionExpand() string {
+	return "version,version.by,history.previousVersion,history.previousVersion.by"
+}
+
+func (c *Client) DownloadAttachmentVersion(attachment Attachment, version int) ([]byte, error) {
 	if attachment.ID == "" {
 		return nil, fmt.Errorf("Confluence attachment response is missing id")
 	}
 	download := attachment.Links["download"]
 	if download == "" {
 		return nil, fmt.Errorf("Confluence attachment %s is missing download link", attachment.ID)
 	}
-	return c.getBytes(resolveLink(c.BaseURL, download))
+	downloadURL := resolveLink(c.BaseURL, download)
+	if version > 0 {
+		parsed, err := url.Parse(downloadURL)
+		if err != nil {
+			return nil, fmt.Errorf("invalid Confluence attachment download link: %w", err)
+		}
+		query := parsed.Query()
+		query.Set("version", strconv.Itoa(version))
+		parsed.RawQuery = query.Encode()
+		downloadURL = parsed.String()
+	}
+	return c.getBytes(downloadURL)
 }

internal/confluence/attachments_test.go

@@ -3,9 +3,42 @@ package confluence
 import (
 	"net/http"
 	"net/http/httptest"
+	"strings"
 	"testing"
 )
 
+func TestAttachmentPointerUsesStableMetadata(t *testing.T) {
+	pointer, err := attachmentPointer("https://cf.example.test/wiki", "123", Attachment{
+		ID:         "456",
+		Title:      "diagram one.png",
+		Version:    Version{Number: 7},
+		Extensions: AttachmentExtensions{FileSize: 42, MediaType: "image/png"},
+		Links: map[string]string{
+			"download": "/wiki/download/attachments/123/diagram%20one.png?version=7&modificationDate=99&api=v2",
+		},
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+	text := string(pointer)
+	for _, want := range []string{
+		"version: https://github.com/hkwi/git-remote-confluence/spec/attachment/v1\n",
+		"attachment_id: \"456\"\n",
+		"attachment_version: 7\n",
+		"filename: diagram one.png\n",
+		"size: 42\n",
+		"media_type: image/png\n",
+		"download_path: /wiki/download/attachments/123/diagram%20one.png?api=v2\n",
+	} {
+		if !strings.Contains(text, want) {
+			t.Fatalf("pointer missing %q:\n%s", want, text)
+		}
+	}
+	if strings.Contains(text, "modificationDate") {
+		t.Fatalf("pointer retained volatile download query:\n%s", text)
+	}
+}
+
 func TestSafeAttachmentName(t *testing.T) {
 	tests := []struct {
 		title string

internal/confluence/location.go

@@ -32,6 +32,9 @@ func ParseLocation(rawURL string) (Location, error) {
 	}
 
 	query := parsed.Query()
+	if attachmentID := firstQuery(query, "attachmentId", "attachmentid"); attachmentID != "" && isDigits(attachmentID) {
+		return Location{BaseURL: baseURL(parsed), RootType: "attachment", RootValue: attachmentID, OriginalURL: original}, nil
+	}
 	if pageID := firstQuery(query, "pageId", "pageid", "id"); pageID != "" && isDigits(pageID) {
 		return Location{BaseURL: baseURL(parsed), RootType: "page", RootValue: pageID, OriginalURL: original}, nil
 	}
@@ -59,7 +62,14 @@ func ParseLocation(rawURL string) (Location, error) {
 		return Location{BaseURL: baseURL(parsed), RootType: "space", RootValue: spaceKey, OriginalURL: original}, nil
 	}
 
-	return Location{}, fmt.Errorf("remote URL must identify a pageId or a Confluence space")
+	return Location{}, fmt.Errorf("remote URL must identify a pageId, attachmentId, or a Confluence space")
+}
+
+func AttachmentRemoteURL(baseURL, pageID, attachmentID string) string {
+	values := url.Values{}
+	values.Set("pageId", pageID)
+	values.Set("attachmentId", attachmentID)
+	return "confluence::" + strings.TrimRight(baseURL, "/") + "/pages/viewpage.action?" + values.Encode()
 }
 
 func ResolvePAT(remoteName string) string {

internal/confluence/location_test.go

@@ -31,6 +31,20 @@ func TestParseLocationSpaceURL(t *testing.T) {
 	}
 }
 
+func TestParseLocationAttachmentURL(t *testing.T) {
+	remoteURL := AttachmentRemoteURL("https://cf.example.test/wiki", "123", "456")
+	location, err := ParseLocation(remoteURL)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if location.BaseURL != "https://cf.example.test/wiki" {
+		t.Fatalf("base URL = %q", location.BaseURL)
+	}
+	if location.RootType != "attachment" || location.RootValue != "456" {
+		t.Fatalf("root = %s %s", location.RootType, location.RootValue)
+	}
+}
+
 func TestParseLocationDisplayPageURL(t *testing.T) {
 	location, err := ParseLocation("confluence:https://cf.example.test/display/ABC/%E5%90%BE%E8%BC%A9%E3%81%AF%E7%8C%AB%E3%81%A7%E3%81%82%E3%82%8B")
 	if err != nil {