describe unique inodes

DESIGN.md

@@ -1,6 +1,6 @@
 This file describes the high-level design of "foiled" by Gumnut.
 
-There are three major layers to the system, plus surface UI code (e.g., the text editor, which may eventually move from this repo).
+There are four major layers to the system, plus surface UI code (e.g., the text editor, which may eventually move from this repo).
 
 # Layers
 
@@ -21,12 +21,13 @@ However, there are special inode formats that have particular meanings.
 
 ### Operations
 
-There are four basic operations:
+There are five basic operations:
 
 1. Grow at position, e.g. insert 3 items at offset 2 (would insert after 123 in the example above)
 2. Delete under range, e.g. delete between 2-4 (would delete `false, NaN` above)
-3. Set at position, e.g. set position 3 to `true` (important: position is _after_ the target, 1-indexed).
-4. Mutate at position - this is as set, but modifies data in-place in some way.
+3. Flip data around range, e.g., flip data 2-4-8, moving data 2-4 to after the former 4-8.
+4. Set at position, e.g. set position 3 to `true` (important: position is _after_ the target, 1-indexed).
+5. Mutate at position - this is as set, but modifies data in-place in some way.
 
 All of these operations are OT-like - they can transform over each other and are always safe to apply.
 They may transform to zero - e.g., if a set operation is transformed over a range that deletes its target, the delta will be nil.
@@ -45,7 +46,7 @@ If this condition can't be met, then the server rejects the ops.
 Note however, that all ops are always valid.
 
 The client will be called again to retry or reconfigure the transaction.
-End-users will never see this, but these transactions might help create dictionaries or other atomic operations that we can't represent.
+End-users will never see this, but these transactions might help create dictionaries or other atomic operations that we can't represent with pure OT.
 
 ### Client API
 
@@ -91,19 +92,36 @@ The higher layers will provide this.
 
 This layer will also support a couple of special low-level types.
 
-1. My Client ID: this is data that will be automatically zero-ed out (possibly with a "zero Client ID" type) when that user disconnects.
+1. My Handle ID: this is data that will be automatically zero-ed out (possibly with a "zero Handle ID" type) when that user disconnects.
    This can be used to claim locks, or key session data.
 2. Slice reference: this can reference an offset within another ino.
    It lets us externally reference _some other_ offset which will transform over time.
+3. Other inode reference: this is just a string, but is tagged specially, and may be manipulated on the way in/out of this layer.
 
-It will also support more 'boring' types which do not map to JS:
+### Strings
 
-1. Use another `ino` as an array
-2. Use a pair of `ino` (maybe implied inode and its +1 peer) as a map
+We support "immutable strings" as a type, as well as arrays of numbers.
+Foiled does not distinguish between numbers and characters/runes; the user of an inode decides how to interpret it (i.e., is this a jumble of numbers, or is this actually a string).
 
-### Strings
+This works because JS numbers are just runs of `uint16`, and JSON can represent every possible JS string value in its UTF-8 encoding.
+
+## 2. Special Inodes
+
+The first section describes basic operations on inodes, the data within etc.
+
+However, the server will provide a few ways for clients to orchestrate inodes.
+We have a few broad goals here:
+
+1. Allow the server to allocate unique inodes for us - if the client uses an inode of a special form during its session, the server will allocate a totally unique (and uncreatable by others) inode for it
+2. Allow us to set the "other inode reference" to a special new inode _only once_
+   - This is basically for creating at-most-once string data; strings in JS are not objects, so should not be attached more than once
+3. Allow a client to create data unique to them for a session (e.g. via a known prefix that is deleted when gone).
+
+In general, users should not be creating arbitrary inodes and this may be disallowed- they are typically only creating data via the above approaches.
+
+### Implementation
 
-We support "immutable strings" as a type, as well as _string data_ (i.e., of `uint16`) within the array-like.
+-- TODO --
 
 ### Special Inode Formats
 
@@ -120,7 +138,7 @@ This will be mapped for _other_ clients to an ID like ":blah".
 Any ID prefixed with a ":" has this 1:1 mapping to the other inode.
 The important part here is that clients _cannot_ set ":blah" directly - they must always create it with "-:".
 
-## 2. Object Model
+## 3. Object Model
 
 This will use the above layer to provide a high-level object model _which is not JS or JSON_.
 
@@ -143,7 +161,7 @@ This will surface in JS as something like `GumnutArray` or `GumnutMap`.
 We can relatively trivially map these to JS objects.
 For our first Gemini demo, we will do something minimal here.
 
-## 3. Gumnut Product Layer
+## 4. Gumnut Product Layer
 
 Finally, we use the above layer to create a layer which allows for importing and mirroring data from an end-user's database (the goal of Gumnut as a product).
 

server/pkg/model/doc/doc.go

@@ -38,6 +38,12 @@ func (d *docImpl) RevVersion() (newVersion int) {
 	return d.version
 }
 
+func (d *docImpl) ServerApply(update map[string]node.Patch) (out *Work, err error) {
+	out = &Work{TargetVersion: d.version, Update: update}
+	err = d.Apply(out)
+	return
+}
+
 func (d *docImpl) Apply(work *Work) (err error) {
 	// ensure not too far ahead
 	if work.TargetVersion > d.version {

server/pkg/model/doc/types.go

@@ -15,6 +15,9 @@ type Doc interface {
 	// This does not care about barrier ops, the caller must fail before this based on its own version checks.
 	Apply(work *Work) (err error)
 
+	// ServerApply is as Apply, but is server-initiated.
+	ServerApply(update map[string]node.Patch) (out *Work, err error)
+
 	// RevVersion bumps the version without doing any work.
 	// This is unlike Apply, which will not rev the version if nothing occurs.
 	RevVersion() (newVersion int)
@@ -23,6 +26,8 @@ type Doc interface {
 	Version() (version int)
 
 	// Read reads the current status of this Doc.
+	// This also includes the current Version.
+	//
 	// Internally, this may apply ops forward: we don't always keep a "live" copy around.
 	Read() (dw *Work)
 

server/pkg/model/raw/const.go

@@ -10,3 +10,9 @@ var (
 	ErrForwardSet   = errors.New("got -ve set in normal forward")
 	ErrNormalizeSet = errors.New("couldn't normalize user set")
 )
+
+var (
+	GumnutDataTrue  = GumnutData{Type: DataTypeKnown, Data: 1, String: "true"}
+	GumnutDataFalse = GumnutData{Type: DataTypeKnown, Data: 1, String: "false"}
+	GumnutDataNull  = GumnutData{Type: DataTypeKnown, Data: 1, String: "null"}
+)

server/pkg/model/raw/set.go

@@ -2,6 +2,7 @@ package raw
 
 import (
 	"encoding/json"
+	"iter"
 	"math"
 	"slices"
 
@@ -16,6 +17,28 @@ type SetPart struct {
 // SetOp is a sequence of set operations which will be applied one after another.
 type SetOp []SetPart
 
+func (s SetOp) IterData() (i iter.Seq[*GumnutData]) {
+	return func(yield func(*GumnutData) (more bool)) {
+		for i := range s {
+			for j := range s[i].Data {
+				if !yield(&s[i].Data[j]) {
+					return
+				}
+			}
+		}
+	}
+}
+
+// Length returns the +ve length here.
+// If this targets -ve ops, these are ignored.
+func (arr SetOp) Length() (length int) {
+	for _, each := range arr {
+		length += each.Skip
+		length += len(each.Data)
+	}
+	return
+}
+
 func (arr SetOp) MarshalJSONArray() (out []json.RawMessage, err error) {
 	for i, each := range arr {
 		if len(each.Data) == 0 {

server/pkg/server/const.go → server/pkg/safedoc/const.go

@@ -1,4 +1,4 @@
-package server
+package safedoc
 
 import (
 	"errors"
@@ -11,5 +11,7 @@ const (
 )
 
 var (
-	ErrSeq = errors.New("invalid sequence number")
+	ErrSeq     = errors.New("invalid sequence number")
+	ErrPerform = errors.New("bad inode target")
+	ErrBarrier = errors.New("barrier failed")
 )

server/pkg/safedoc/inode.go

@@ -0,0 +1,65 @@
+package safedoc
+
+import (
+	"fmt"
+	"maps"
+	"slices"
+	"strings"
+
+	"github.com/gumnutdev/foiled/server/pkg/model/node"
+	"github.com/gumnutdev/foiled/server/pkg/model/raw"
+)
+
+func inodeHandleActive(handleID int) (s string) {
+	return fmt.Sprintf("handle-active/%x:m", handleID)
+}
+
+// mapInodeIn updates inodes requested to be changed by a session.
+// It returns false if there are invalid inode targets here.
+func (ds *docSession) mapInodeIn(update map[string]node.Patch) (ok bool) {
+	handleActive := inodeHandleActive(ds.handleID)
+
+	for id, patch := range update {
+		if id == "" {
+			return false
+		} else if strings.HasPrefix(id, "handle-active/") {
+			return false
+		}
+
+		// rewrite "self" target to be the actual ID
+		for gd := range patch.Set.IterData() {
+			if gd.Type == raw.DataTypeInode && gd.String == "handle-active/self:m" {
+				gd.String = handleActive
+			}
+		}
+	}
+
+	return true
+}
+
+// mapInodeOut modifies what gets broadcast to this session.
+// The passed map is already local and can be modified inline.
+func (ds *docSession) mapInodeOut(update map[string]node.Patch) {
+	handleActive := inodeHandleActive(ds.handleID)
+
+	for _, patch := range update {
+		// rewrite "self" target to be the actual ID
+		for gd := range patch.Set.IterData() {
+			// FIXME: we need a better way to say - we're modifying this, copy all the way down :\
+			if gd.Type == raw.DataTypeInode && gd.String == handleActive {
+				gd.String = "handle-active/self:m"
+			}
+		}
+	}
+
+	keys := slices.Collect(maps.Keys(update))
+
+	for _, id := range keys {
+		if id == handleActive {
+			// broadcast self ownership under unique ID
+			update["handle-active/self:m"] = update[id]
+			delete(update, id)
+			continue
+		}
+	}
+}