Simplify policy import model: replace entriesAdditions and importsAliases with entry-level references

[thjaeckle]

Summary

The Ditto 3.9.0 policy import model (not yet released) introduces several new concepts to enable multi-level policy template hierarchies:

• entriesAdditions on imports — merge additional subjects/resources/namespaces into imported entries
• allowedImportAdditions on template entries — control what importing policies may add
• transitiveImports on imports — explicit multi-level resolution (Support transitive resolution of policy imports via transitiveImports #2420)
• importsAliases on the policy — fan out subject operations to multiple entries additions targets

While powerful, this results in a complex model with multiple interacting concepts spread across different parts of the policy. This issue proposes a simplification that achieves the same functionality with fewer, more intuitive building blocks.

Proposal

Replace entriesAdditions, importsAliases, and importReference/alias with a single entry-level references array.

references on policy entries

An entry can declare a references array containing objects that point to other entries. Each reference is either:

• Import reference ({"import": "policyId", "entry": "label"}) — references an entry in an imported policy. The referencing entry inherits resources and namespaces from the referenced template entry, while defining its own local subjects (and optionally resources/namespaces if the template permits via allowedImportAdditions).
• Local reference ({"entry": "label"}) — references another entry within the same policy. The referencing entry inherits subjects, resources, and namespaces from the referenced local entry. This replaces importsAliases without introducing a separate concept: shared subjects live in a single entry, referenced by others.

An entry can have multiple references (both import and local), and all are resolved additively at enforcement time.

What changes

Removed concept	Introduced in	Replaced by
entriesAdditions on imports	#2221, PR #2347	local subjects/resources/namespaces on entry
allowedImportAdditions validation for aliases	PR #2419	pre-enforcer validates import references in references array
importsAliases on policy	#2402, PR #2403	local references pointing to a shared-subjects entry

What stays

Concept	Reason
entries filter on import	backward-compatible entry selection for policies not using references
importable (implicit/never)	template author controls which entries are importable and which are never available
transitiveImports	explicit control over multi-level resolution depth (no surprises)
allowedImportAdditions	template author retains control over what consumers may customize

Example: 3-level policy hierarchy

A fleet management scenario with three levels:

• Template defines roles with resources (what a driver can do)
• Intermediate assigns regional drivers (who is a driver)
• Leaf is the actual thing's policy (combines both)

Current approach (Ditto 3.9.0 as initially implemented)

Template (acme:fleet-roles):

{
  "policyId": "acme:fleet-roles",
  "entries": {
    "driver": {
      "subjects": {},
      "resources": {
        "thing:/features/location": { "grant": ["READ"], "revoke": [] },
        "thing:/features/fuel": { "grant": ["READ"], "revoke": [] },
        "message:/features/fuel/inbox": { "grant": ["WRITE"], "revoke": [] }
      },
      "namespaces": ["acme.vehicle"],
      "allowedImportAdditions": ["subjects"],
      "importable": "implicit"
    }
  }
}

Intermediate (acme:fleet-west):

{
  "policyId": "acme:fleet-west",
  "imports": {
    "acme:fleet-roles": {
      "entriesAdditions": {
        "driver": {
          "subjects": {
            "oauth2:alice@acme.com": { "type": "employee" },
            "oauth2:bob@acme.com": { "type": "employee" }
          }
        }
      }
    }
  },
  "entries": {}
}

The intermediate has no inline entries — it only defines entriesAdditions on its import. The subjects and the entry they target are nested inside the import declaration. The intermediate's structure is opaque: reading it doesn't reveal what entries it provides.

Simplified approach (using references)

Template (acme:fleet-roles) — unchanged:

{
  "policyId": "acme:fleet-roles",
  "entries": {
    "driver": {
      "subjects": {},
      "resources": {
        "thing:/features/location": { "grant": ["READ"], "revoke": [] },
        "thing:/features/fuel": { "grant": ["READ"], "revoke": [] },
        "message:/features/fuel/inbox": { "grant": ["WRITE"], "revoke": [] }
      },
      "namespaces": ["acme.vehicle"],
      "allowedImportAdditions": ["subjects"],
      "importable": "implicit"
    }
  }
}

Intermediate (acme:fleet-west):

{
  "policyId": "acme:fleet-west",
  "imports": {
    "acme:fleet-roles": {}
  },
  "entries": {
    "driver": {
      "references": [
        { "import": "acme:fleet-roles", "entry": "driver" }
      ],
      "subjects": {
        "oauth2:alice@acme.com": { "type": "employee" },
        "oauth2:bob@acme.com": { "type": "employee" }
      }
    }
  }
}

The intermediate declares an explicit entry "driver" with a references entry pointing to the template. Resources and namespaces are inherited; subjects are local. The policy is self-describing — reading it reveals it has a "driver" entry linked to the template.

Leaf (acme.vehicle:truck-42):

{
  "policyId": "acme.vehicle:truck-42",
  "imports": {
    "acme:fleet-west": {
      "transitiveImports": ["acme:fleet-roles"]
    }
  },
  "entries": {
    "driver": {
      "references": [
        { "import": "acme:fleet-west", "entry": "driver" }
      ],
      "subjects": {
        "oauth2:charlie@acme.com": { "type": "temp-driver" }
      }
    },
    "owner": {
      "subjects": { "oauth2:fleet-admin@acme.com": { "type": "admin" } },
      "resources": { "policy:/": { "grant": ["READ", "WRITE"], "revoke": [] } }
    }
  }
}

Same resolved result: resources from template + subjects from intermediate (alice, bob) + subjects from leaf (charlie).

Multi-namespace fan-out example (using local references)

A power plant template with entries scoped to different namespaces:

{
  "policyId": "energy:plant-roles",
  "entries": {
    "reactor-operator": {
      "subjects": {},
      "resources": { "thing:/features/reactor": { "grant": ["READ", "WRITE"], "revoke": [] } },
      "namespaces": ["plant.reactor"],
      "allowedImportAdditions": ["subjects"]
    },
    "turbine-operator": {
      "subjects": {},
      "resources": { "thing:/features/turbine": { "grant": ["READ", "WRITE"], "revoke": [] } },
      "namespaces": ["plant.turbine"],
      "allowedImportAdditions": ["subjects"]
    }
  }
}

Consuming policy with local reference for shared subjects:

{
  "imports": { "energy:plant-roles": {} },
  "entries": {
    "operators": {
      "subjects": { "alice": { "type": "engineer" } }
    },
    "reactor-op": {
      "references": [
        { "import": "energy:plant-roles", "entry": "reactor-operator" },
        { "entry": "operators" }
      ]
    },
    "turbine-op": {
      "references": [
        { "import": "energy:plant-roles", "entry": "turbine-operator" },
        { "entry": "operators" }
      ]
    }
  }
}

PUT /entries/operators/subjects/alice adds alice as a standard single-entry operation. At enforcement time, both reactor-op and turbine-op inherit alice's subject via local reference — each retaining its own namespace scope.

No fan-out logic, no alias concept, no special REST semantics. Standard CRUD on the "operators" entry, with references resolved at enforcement time.

Side-by-side comparison

	Old (entriesAdditions)	New (references)
Subject declaration	nested in imports.*.entriesAdditions.*.subjects	directly in entries.*.subjects
Entry visibility	intermediate has no entries (opaque until resolution)	intermediate has explicit entries (self-describing)
Entry selection	entries filter on the import + entriesAdditions keys	entries filter (backward compat) or references on entries
Fan-out (multi-entry)	separate importsAliases top-level concept	local references to a shared-subjects entry
Subject API path	PUT /entries/{alias}/subjects/... (ambiguous REST)	PUT /entries/{shared}/subjects/... (standard CRUD)
Reference multiplicity	importReference is singular	references array supports N references (import + local)
Concepts to learn	entriesAdditions, allowedImportAdditions, importsAliases, entries filter, importable, importReference, alias	references, allowedImportAdditions, entries filter, importable

Resolution semantics

• Each import reference in references inherits resources, namespaces, allowedImportAdditions, and importable from the referenced entry
• Each local reference inherits subjects, resources, and namespaces from the referenced entry
• Local subjects (and optionally resources/namespaces if allowedImportAdditions permits) are merged with the inherited values
• transitiveImports on the import controls whether the imported policy's own reference chains are resolved (explicit opt-in, no surprises)
• Cycle detection via visited set + depth limit (same as current implementation)
• Entries with no subjects or no resources after resolution are filtered out from the enforcer (optimization: they contribute nothing to access decisions)

REST API

The references field is exposed at:

• GET /api/2/policies/{policyId}/entries/{label}/references — retrieve the references array
• PUT /api/2/policies/{policyId}/entries/{label}/references — set the references array
• DELETE /api/2/policies/{policyId}/entries/{label}/references — remove all references

No special alias endpoints. No fan-out REST semantics. Standard CRUD.

[thjaeckle]

Impact: removed model classes and HTTP API endpoints

This simplification removes the following existing concepts that were added for 3.9.0 (unreleased):

Model classes to remove

entriesAdditions (6 classes):

• EntriesAdditions / ImmutableEntriesAdditions
• EntryAddition / ImmutableEntryAddition
• Removed from: EffectedImports.getEntriesAdditions(), PolicyImport.getEntriesAdditions()

importsAliases (6 classes):

• ImportsAliases / ImmutableImportsAliases
• ImportsAlias / ImmutableImportsAlias
• ImportsAliasTarget / ImmutableImportsAliasTarget
• Removed from: Policy.getImportsAliases(), PolicyBuilder

Signal commands + responses (~22 files):

• ModifyPolicyImportEntriesAdditions + response
• RetrievePolicyImportEntriesAdditions + response
• ModifyPolicyImportEntryAddition + response
• RetrievePolicyImportEntryAddition + response
• DeletePolicyImportEntryAddition + response
• ModifyImportsAliases + response
• RetrieveImportsAliases + response
• DeleteImportsAliases + response
• ModifyImportsAlias + response
• RetrieveImportsAlias + response
• DeleteImportsAlias + response

Signal events (~13 files):

• PolicyImportEntriesAdditionsModified
• PolicyImportEntryAdditionCreated / Modified / Deleted
• ImportsAliasesModified / ImportsAliasesDeleted
• ImportsAliasCreated / ImportsAliasModified / ImportsAliasDeleted
• ImportsAliasSubjectCreated / ImportsAliasSubjectModified / ImportsAliasSubjectDeleted
• ImportsAliasSubjectsModified

Persistence strategies (~22 files): corresponding command + event strategies for all above signals.

HTTP API endpoints to remove

entriesAdditions endpoints:

• GET /api/2/policies/{policyId}/imports/{importedPolicyId}/entriesAdditions
• PUT /api/2/policies/{policyId}/imports/{importedPolicyId}/entriesAdditions
• GET /api/2/policies/{policyId}/imports/{importedPolicyId}/entriesAdditions/{label}
• PUT /api/2/policies/{policyId}/imports/{importedPolicyId}/entriesAdditions/{label}
• DELETE /api/2/policies/{policyId}/imports/{importedPolicyId}/entriesAdditions/{label}

importsAliases endpoints (entire route removed):

• GET /api/2/policies/{policyId}/importsAliases
• PUT /api/2/policies/{policyId}/importsAliases
• DELETE /api/2/policies/{policyId}/importsAliases
• GET /api/2/policies/{policyId}/importsAliases/{label}
• PUT /api/2/policies/{policyId}/importsAliases/{label}
• DELETE /api/2/policies/{policyId}/importsAliases/{label}
• Plus all subject sub-resource endpoints under /importsAliases/{label}/subjects/...

Total: ~69 files removed, ~12 HTTP endpoints removed

What replaces them

Removed	Replacement
entriesAdditions on import	subjects/resources/namespaces directly on entries with importReference
entries filter on import	importReference on individual entries
importsAliases on policy	alias field on entries
importsAliases subject endpoints	Existing /entries/{alias}/subjects/... with fan-out

New model classes needed: ImportReference / ImmutableImportReference (2 files), plus importReference and alias fields added to existing PolicyEntry.

[hu-ahmed]

Big +1 on the direction — collapsing entriesAdditions and importsAliases into entry-level importReference and alias is a clear reduction in concept count, and making the intermediate policy self-describing is a real
usability win.

Before forming a complete picture I'd love to understand a few interactions with existing features. I've grounded each in the current code so you can confirm or correct quickly.

1. Interaction with namespace root policy (#1638)

PolicyEnforcer.mergeImplicitEntries selects entries using a hardcoded ImportableType.IMPLICIT check:

// policies/enforcement/.../PolicyEnforcer.java around line 163
private static Policy mergeImplicitEntries(final Policy rootPolicy, final Policy currentPolicy) {                                                                                                                               
    Policy result = currentPolicy;                                                                                                                                                                                              
    for (final PolicyEntry entry : rootPolicy) {                                                                                                                                                                                
        if (ImportableType.IMPLICIT.equals(entry.getImportableType())                                                                                                                                                           
                && !result.contains(entry.getLabel())) {  
            result = result.setEntry(entry);                                                                                                                                                                                    
        }                                                 
    }                                                                                                                                                                                                                           
    return result;                                        
}                                                                                                                                                                                                                               

With implicit / explicit removed and only importable: "never" remaining, what's the plan for namespace root policy — a new selection marker (e.g. "root-only"), "all non-never entries merged", or is namespace root policy being reworked alongside this proposal?

2. Default value for absent importable

ImmutablePolicyEntry.fromJson currently defaults missing importable to IMPLICIT:

// policies/model/.../ImmutablePolicyEntry.java around line 182
final ImportableType importType = readImportableType(jsonObject).orElse(ImportableType.IMPLICIT);

After #2423, what's the new default — effectively "referenceable by any importReference" (preserves today's common-case behavior), or something stricter?

3. alias endpoint semantics beyond subject fan-out

The proposal covers PUT /entries/{alias}/subjects/{subjectId} fan-out cleanly. Existing importsAliases maps 1:1 (one alias → one ImportsAlias resource with multiple targets), so PolicyImportsAliasesRoute endpoints like GET /importsAliases/{label} return a single object.

Under the new shape, multiple entries share one alias. What's the intended behavior for:

• GET /entries/{alias} — merged view, array response, or 409 when multiple entries match?
• PUT /entries/{alias} — reject (can't replace a multi-entry alias atomically), or fan-out?
• DELETE /entries/{alias} — delete all aliased entries, or just remove the alias linkage?

4. Migration for the entries filter on imports

The table lists:

entries filter on import → importReference on individual entries (always explicit)

The entries include/exclude filter (EffectedImports.getIncluded / getExcluded) predates 3.9.0 and exists in shipped 3.8.x policies. Is the plan to:

• Auto-translate on load (each filtered label → synthetic importReference entry),
• Reject at upgrade with a migration utility, or
• Keep entries filter deprecated for one release?

A short note in the proposal would help adopters plan their 3.8.x → next upgrade.

5. Inheritance semantics for importReference

The resolution section states:

importReference inherits resources, namespaces, allowedImportAdditions, and importable from the referenced entry

Two sub-questions:

• When the entry also declares local resources / namespaces (permitted via allowedImportAdditions), is the merge additive (union with inherited, matching today's PolicyImporter.mergeResources union semantics at ~L152/L155) or replacing?
• For allowedImportAdditions: can a referencing entry tighten (e.g. template allows ["subjects", "resources"], reference narrows to ["subjects"]), and is loosening explicitly rejected?

6. Mutability of importReference

Is importReference a mutable field on an existing entry (e.g. PUT /entries/{label}/importReference to rewire, or PUT /entries/{label} with a new reference body), or is it effectively immutable once the entry is created?

---

For context: #2422 is orthogonal to this proposal — it implements transitiveImports (#2420) without touching ImportableType or namespace root policy resolution, so none of the questions above are addressed there.

Happy to open follow-up issues for any of these if you'd prefer not to expand the scope of this one.

[thjaeckle]

1. Interaction with namespace root policy
   With implicit / explicit removed and only importable: "never" remaining

I don't see that implicit / explicit are removed - they must stay in tact as they are in order to not break backwards compatibility.
Just for enhancing "templates" with e.g. custom subjects they must be explicitly defined - but that is something else than the existing import mechanism which must stay the same.

I also don't see that this changes anything with the "namespace root policy" concept.

2. Default value for absent importable

Stays the same: "implicit"

3. alias endpoint semantics beyond subject fan-out

Yeah, that's an open point I would say - I am not yet totally clear about this one.

GET /entries/{alias}
PUT /entries/{alias}
DELETE /entries/{alias}

Those do not make 100% "sense" for an alias entry - maybe we should simply reject them with a "400 - Bad Request" and a proper error message.

What do you think @hu-ahmed ?

4. Migration for the entries filter on imports

Aah, true .. "entries filter on import" is not removed - that is listed wrong in the table. It cannot be, as this is API before Ditto 3.9.0 (for defining "explicitly" imported policy entries).
So this stays the same - entries which are marked with "importable": "explicit" must be explicitly imported via "entries".

5. Inheritance semantics for importReference

• When the entry also declares local resources / namespaces (permitted via allowedImportAdditions), is the merge additive

Yes, I would say this is additive (same is currently defined via entriesAdditions - which will be replaced by this new approach).

• For allowedImportAdditions: can a referencing entry tighten (e.g. template allows ["subjects", "resources"], reference narrows to ["subjects"]), and is loosening explicitly rejected?

Ah, good one .. It should only be allowed to narrow it down, only that way the "contract" of the policy entry we import from is fulfilled.

6. Mutability of importReference
   Is importReference a mutable field on an existing entry

I would say yes - it shall be able to be modified on existing policy entries, e.g. doing "rewiring" or migrating an existing entry to one which defines an importReference

I think point 3. is still the most unclear point which we still need to define in which direction we want to proceed.

[hu-ahmed]

On point 3 — backing the 400 direction, with two tightening rules and an explicit per-endpoint scope so there's no ambiguity:

1. Reject alias-vs-label collisions at write time. If any alias string equals any concrete entry label in the same policy, reject the PUT with 400. Keeps /entries/{x} unambiguous — x is a label or an alias, never both.

2. Uniform 400 regardless of current match count. Even when an alias currently matches a single entry, still reject; otherwise behavior flips silently when a second entry joins the alias later.

3. Fan-out scope — subjects at {subjectId} granularity only, matching today's ModifySubjectStrategy.handleAliasSubject:

Endpoint	Behavior
PUT /entries/{alias}/subjects/{subjectId}	fan out to all matched entries
DELETE /entries/{alias}/subjects/{subjectId}	fan out to all matched entries
GET /entries/{alias}/subjects/{subjectId}	200 if present in any matched entry, 404 otherwise
GET /entries/{alias}/subjects	merged/union view
PUT /entries/{alias}/subjects	400 (bulk replace would erase each entry's full subject set)
GET / PUT / DELETE /entries/{alias}	400 (whole-entry body carries a single importReference; fan-out would erase per-entry reference differences)
/entries/{alias}/resources, /resources/{resource}	400 (per-entry concern; allowedImportAdditions varies per entry)
/entries/{alias}/namespaces	400 (per-entry concern; templates scope namespaces differently)
/entries/{alias}/allowedImportAdditions	400 (per-entry authorial concern)
/entries/{alias}/importable	400 (per-entry authorial concern)
POST /entries/{alias}/actions/*	400 (would trigger the action N times with compounding side effects, e.g. activateTokenIntegration)

Discovery is covered by GET /entries (lists all entries with their alias field visible), so no separate alias-listing endpoint needed.

[thjaeckle]

@hu-ahmed I updated the issue description with an even more simplified approach. Could you check it out?

[hu-ahmed]

This is a significantly better design than the importReference + alias round. The unification into a single references array with both import and local reference types is a strong simplification:

• One concept replaces two (importReference + alias → references)
• Fan-out moves from write-time (special alias routing) to enforcement-time (standard CRUD on a shared entry, references pulled in at resolution) — which makes the entire alias endpoint discussion from the previous round
  moot
• Entries are self-describing at every level; reading reactor-op tells you where each piece comes from
• Event stream simplifies (SubjectCreated on the shared entry instead of ImportsAliasSubjectCreated with a target list)
• The "filter entries with no subjects or no resources from the enforcer" rule is a clean acknowledgment that shared-subject entries are write-time affordances, not runtime access rules

+1 from me on the direction. A few small follow-ups the updated text doesn't fully spell out:

1. Local-reference cycle detection — A references B, B references A. Same visited-set + depth-limit pattern as transitiveImports? Shared MAX_TRANSITIVE_RESOLUTION_DEPTH = 10 across mixed local/import chains, or separate counters?
2. Broken local reference — if reactor-op references operators and operators is later deleted, is that rejected at write time (referential integrity on PUT) or a graceful skip at enforcement (matching today's missing-root-policy posture)?
3. Self-reference — rejected at write time, I assume?
4. allowedImportAdditions scope — confirming it only constrains import references (not local ones), since local references live inside the same policy the author already controls.

Happy with the rest. Point 3 from my earlier comment can be closed as resolved by the design.

[thjaeckle]

@hu-ahmed Thanks for the thorough review of the updated design! Here are the answers to your follow-up questions:

1. Local-reference cycle detection — Local references resolve one level deep only (no transitive chaining), so cycles cannot occur. resolveReferences() iterates each entry's references once without recursion.

2. Broken local reference — Rejected at write time (referential integrity). DeletePolicyEntry returns 409 if another entry has a local reference to the entry being deleted. Same for ModifyPolicyEntries and ModifyPolicy — if the resulting entry set would leave a dangling local reference, the request is rejected.

3. Self-reference — Rejected at write time with 400. Added in a825463.

4. allowedImportAdditions scope — Confirmed: only constrains import references. Local references are unconstrained since the author controls both entries within the same policy.