From 556b2210a04a9b6c9cd2c306b17bc191ac9705d4 Mon Sep 17 00:00:00 2001 From: Keith Wall Date: Fri, 19 Sep 2025 14:20:41 +0100 Subject: [PATCH 1/5] Create 011-user-namespace.md Signed-off-by: Keith Wall --- proposals/011-user-namespace.md | 69 +++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 proposals/011-user-namespace.md diff --git a/proposals/011-user-namespace.md b/proposals/011-user-namespace.md new file mode 100644 index 0000000..5e2bcbc --- /dev/null +++ b/proposals/011-user-namespace.md @@ -0,0 +1,69 @@ + + +# User Namespace Filter + +Proposes the introduction of a "User Namespace Filter" to Kroxylicious's core filters. + +The role of the User Namespace Filter is to give the client a private space within the kafka cluster space that is isolated from other users sharing the cluster. Namespacing can be applied selectively to different resource types. This allows +the possibility for some resource types (probably topics) to be shared between users whereas only resource instances exist in the private space. + +## Current situation + +The project has a preview multi-tenancy filter, which is a similar idea to this one except that it applies unconditionally to all resource types. + +## Motivation + +We encountered a use-case where the desire was to share topics but maintain isolated spaces for groups and transactional-ids. + +## Proposal + +The role of the user namespace filter is to give the client the impression of a private kafka cluster space that is isolated from other clients sharing the cluster. Namespacing can be applied selectively to different resource types. + +The filter will use a pluggable API to determine how to map the name of each resource. Operations that retrieve lists of resources will see only those that fall within the namespace. + +For the initial release, the filter will need to support only namespacing for consumer group names and transactional ids. There will be scope for the filter to support prefixing of topic resources, but this won’t be supported in the initial release. + +This proposal will deliver a simple implementation of the API that simply uses the principal as the prefix. +The principal will be added to the resource as it is sent to the server, and removed as it is returned in the response. + +### APIs + +#### Filter Configuration + +```yaml +type: UserNamespaceFilter +config: +resourceNameMappers: +- resourceTypes: [TRANSACTIONAL_ID, GROUP_ID] + resourceNameMapper: SimplePrincipalPrefixingNameMapper +``` + +#### Resource Name Mapper API + +Asynchronous interface that lets the filter map from downstream names to upstream names and vice-versa. It also +has the responsibility to filter the upstream view, removing resources that don't belong in the view. + +```java +interface ResourceNameMapper { + /** Return a mapping of downstream names to upstream names. */ + CompletionStage> mapDownstreamResourceNames(String authorizationId, ResourceType resourceType, List downstreamResourceNames); + /** Return a filtered map of upstream resource names to downstream names. Any resources that do not form part of the view will be omitted from the map. */ + CompletionStage> mapUpstreamFilteredResourceNames(String authorizationId, ResourceType resourceType, List upstreamResourceNames); +} +``` + +## Affected/not affected projects + +Call out the projects in the Kroxylicious organisation that are/are not affected by this proposal. + +## Compatibility + +Call out any future or backwards compatibility considerations this proposal has accounted for. + +## Rejected alternatives + +Call out options that were considered while creating this proposal, but then later rejected, along with reasons why. + +## Limitations + +The name prefixing approach reduces the length of the name that the client may use for resource names. If the prefixed name exceeds the length permitted by Kafka the request must fail in an understandable way. From fdbcc98843f9aa63303d57e23c9f68d873fe6011 Mon Sep 17 00:00:00 2001 From: Keith Wall Date: Fri, 30 Jan 2026 14:08:37 +0000 Subject: [PATCH 2/5] Update 011-user-namespace.md Signed-off-by: Keith Wall --- proposals/011-user-namespace.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/proposals/011-user-namespace.md b/proposals/011-user-namespace.md index 5e2bcbc..11cca0e 100644 --- a/proposals/011-user-namespace.md +++ b/proposals/011-user-namespace.md @@ -1,10 +1,11 @@ -# User Namespace Filter +# Resource Isolation Filter -Proposes the introduction of a "User Namespace Filter" to Kroxylicious's core filters. -The role of the User Namespace Filter is to give the client a private space within the kafka cluster space that is isolated from other users sharing the cluster. Namespacing can be applied selectively to different resource types. This allows +Proposes the introduction of a "Resource Isolation Filter" to Kroxylicious's core filters. + +The role of the Resource Isolation Filter is to give the client a private space within the kafka cluster space that is isolated from other users sharing the cluster. Isolation can be applied selectively to different resource types. This allows the possibility for some resource types (probably topics) to be shared between users whereas only resource instances exist in the private space. ## Current situation @@ -17,7 +18,7 @@ We encountered a use-case where the desire was to share topics but maintain isol ## Proposal -The role of the user namespace filter is to give the client the impression of a private kafka cluster space that is isolated from other clients sharing the cluster. Namespacing can be applied selectively to different resource types. +The role of the Resource Isolation filter is to give the client the impression of a private kafka cluster space that is isolated from other clients sharing the cluster. Namespacing can be applied selectively to different resource types. The filter will use a pluggable API to determine how to map the name of each resource. Operations that retrieve lists of resources will see only those that fall within the namespace. @@ -31,7 +32,7 @@ The principal will be added to the resource as it is sent to the server, and rem #### Filter Configuration ```yaml -type: UserNamespaceFilter +type: ResourceIsolation config: resourceNameMappers: - resourceTypes: [TRANSACTIONAL_ID, GROUP_ID] From 1829a6f12dc2133efd04fa5d62af93d1a205a143 Mon Sep 17 00:00:00 2001 From: Keith Wall Date: Tue, 31 Mar 2026 11:15:09 +0100 Subject: [PATCH 3/5] updates to reflect implementation Signed-off-by: Keith Wall --- proposals/011-user-namespace.md | 70 ------------- proposals/015-entity-isolation.md | 161 ++++++++++++++++++++++++++++++ 2 files changed, 161 insertions(+), 70 deletions(-) delete mode 100644 proposals/011-user-namespace.md create mode 100644 proposals/015-entity-isolation.md diff --git a/proposals/011-user-namespace.md b/proposals/011-user-namespace.md deleted file mode 100644 index 11cca0e..0000000 --- a/proposals/011-user-namespace.md +++ /dev/null @@ -1,70 +0,0 @@ - - -# Resource Isolation Filter - - -Proposes the introduction of a "Resource Isolation Filter" to Kroxylicious's core filters. - -The role of the Resource Isolation Filter is to give the client a private space within the kafka cluster space that is isolated from other users sharing the cluster. Isolation can be applied selectively to different resource types. This allows -the possibility for some resource types (probably topics) to be shared between users whereas only resource instances exist in the private space. - -## Current situation - -The project has a preview multi-tenancy filter, which is a similar idea to this one except that it applies unconditionally to all resource types. - -## Motivation - -We encountered a use-case where the desire was to share topics but maintain isolated spaces for groups and transactional-ids. - -## Proposal - -The role of the Resource Isolation filter is to give the client the impression of a private kafka cluster space that is isolated from other clients sharing the cluster. Namespacing can be applied selectively to different resource types. - -The filter will use a pluggable API to determine how to map the name of each resource. Operations that retrieve lists of resources will see only those that fall within the namespace. - -For the initial release, the filter will need to support only namespacing for consumer group names and transactional ids. There will be scope for the filter to support prefixing of topic resources, but this won’t be supported in the initial release. - -This proposal will deliver a simple implementation of the API that simply uses the principal as the prefix. -The principal will be added to the resource as it is sent to the server, and removed as it is returned in the response. - -### APIs - -#### Filter Configuration - -```yaml -type: ResourceIsolation -config: -resourceNameMappers: -- resourceTypes: [TRANSACTIONAL_ID, GROUP_ID] - resourceNameMapper: SimplePrincipalPrefixingNameMapper -``` - -#### Resource Name Mapper API - -Asynchronous interface that lets the filter map from downstream names to upstream names and vice-versa. It also -has the responsibility to filter the upstream view, removing resources that don't belong in the view. - -```java -interface ResourceNameMapper { - /** Return a mapping of downstream names to upstream names. */ - CompletionStage> mapDownstreamResourceNames(String authorizationId, ResourceType resourceType, List downstreamResourceNames); - /** Return a filtered map of upstream resource names to downstream names. Any resources that do not form part of the view will be omitted from the map. */ - CompletionStage> mapUpstreamFilteredResourceNames(String authorizationId, ResourceType resourceType, List upstreamResourceNames); -} -``` - -## Affected/not affected projects - -Call out the projects in the Kroxylicious organisation that are/are not affected by this proposal. - -## Compatibility - -Call out any future or backwards compatibility considerations this proposal has accounted for. - -## Rejected alternatives - -Call out options that were considered while creating this proposal, but then later rejected, along with reasons why. - -## Limitations - -The name prefixing approach reduces the length of the name that the client may use for resource names. If the prefixed name exceeds the length permitted by Kafka the request must fail in an understandable way. diff --git a/proposals/015-entity-isolation.md b/proposals/015-entity-isolation.md new file mode 100644 index 0000000..5f77d03 --- /dev/null +++ b/proposals/015-entity-isolation.md @@ -0,0 +1,161 @@ + + +# Entity Isolation Filter + + +Proposes the introduction of a "Entity Isolation Filter" to Kroxylicious's core filters. + +This filter gives a connected client an isolated view of some or all of its entities (e.g. topics), segregating them entities of other clients that happen to use the same name. +Isolation can be applied selectively to some entity types and not others. For example, it's possible to have isolation for consumer groups, but not for topics. + +## Current situation + +The project has a preview multi-tenancy filter, which is a similar idea to this one except that it applies unconditionally to all resource types. + +## Motivation + +We encountered a use-case where the desire was to share topics but maintain isolated spaces for groups and transactional-ids. + +## Proposal + +It is proposed to add a new filter, the Entity Isolation filter, to set of core filters shipped as part of Kroxylicious. + +The filter will be capable of isolating the kafka entities of from one client from those of another client, even if they have to choose the same name. + +The filter will achieve isolation using name transformation and result filtering + +- Entity names known to the client will be transformed in some way to avoid collision with entities belonging to other clients. +- The client will have no knowledge of the entity's transformed name. +- When the client uses Kafka APIs that list entities, the result set will be filtered so that the client receives only entities that belong to them. + +## Configurable isolation + +The filter will expose configuration that allows the administrator to decide which entity types are isolated and which are not. +For instance, it will be possible to configure the filter so that Kafka group-ids and transactional-ids are isolated, but +topic are not. + +## Pluggable Mapper + +The filter will be delegate the job of transforming and filtering entity names to a `EntityNameMapper`. The mapper will be pluggable API +allowing alternative mapping strategies to be employed. + +For the first release there will be a single implementation of the mapper, PrincipalEntityNameMapper, that works by prefixing the entity name with the name of a unique principal from the authenticated subject and a configurable separator +To use this mapper, the proxy must be configured with a `SubjectBuilder` so that the filter is aware of the channel's authenticated subject (from TLS or SASL). + +On the request path, the isolated entity name will be prefixed with the principal name like so. + +``` +orders -> alice_orders +``` + +On the response path, for Kafka messages that list entities, the entities will be first filtered: + +``` +alice_orders -> alice_orders +bob_orders // filtered out +charlie_orders // filtered out +``` + +then the prefixing removed: + +``` +alice_orders -> orders +``` + + +## Restrictions scope for the first version + +In order to narrow the scope of the problem, the first version of the Entity Isolation Filter will be restricted +to the groupId and transactionId entity types. + +Topic isolation has some additional challenges which we want to defer to a future release. +https://github.com/kroxylicious/kroxylicious/issues/3504 + +### APIs + +#### Filter Configuration + +```yaml + +type: io.kroxylicious.filter.entityisolation.EntityIsolation +config: + entityTypes: + - GROUP_ID + - TRANSACTIONAL_ID + mapper: PrincipalEntityNameMapperService + mapperConfig: + separator: "-" + principalType: io.kroxylicious.proxy.authentication.User +``` + +## EntityNameMapper API + +Synchronous API that lets the filter map from downstream names to upstream names and vice versa. +It also has the responsibility to filter the upstream view, removing resources that don't belong in the view. + +Asynchronous interface that lets the filter map from downstream names to upstream names and vice-versa. It also +has the responsibility to filter the upstream view, removing resources that don't belong in the view. + +```java +interface EntityNameMapper { + String map(MapperContext mapperContext, + EntityType entityType, + String downstreamEntityName) + throws EntityMapperException; + + /** + * Maps an upstream kafka entity name to a downstream name. + * + * @param mapperContext mapper context. + * @param entityType entity type. + * @param upstreamEntityName upstream entity name. + * @return downstream entity name + * @throws EntityMapperException the mapped entity name violates one or more system constraints. + */ + String unmap(MapperContext mapperContext, + EntityType entityType, + String upstreamEntityName) + throws EntityMapperException; + + /** + * Tests whether the given upstreams entity name belongs to this context. + * + * @param mapperContext mapper context. + * @param entityType entity type. + * @param upstreamEntityName upstream entity name. + * @return true if the mapped entity name belongs to this context, false otherwise. + */ + boolean isOwnedByContext(MapperContext mapperContext, + EntityType entityType, + String upstreamEntityName); +} +``` + +```java +record MapperContext(Subject authenticatedSubject, @Nullable ClientTlsContext clientTlsContext, @Nullable ClientSaslContext clientSaslContext) { + public MapperContext { + Objects.requireNonNull(authenticatedSubject); + } +} +``` + + +## Affected/not affected projects + +This proposal introduces a new filter. It does not affect any existing projects with the Kroxylicious organisation. + +## Compatibility + +This proposal introduces a new filter. There are no changes to existing APIs. + +## Rejected alternatives + +### Build on the Multi Tenancy Filter + +The project has an existing 'proof-of-concept' Multi Tenancy Filter that dates back to the project's inception. +It isolates topic names, group ids and transactional ids by prefixing with entities with the virtual cluster name. +Its implementation is sufficient to cover basic produce/consume use-cases only but nothing more. +There are several areas where the filter will fail. + +Whilst it would have been possible to base the entity isolation work on the multi tenancy filter, +it was felt by the developers that a fresh start would give stronger foundations. From 5abb15aef35b7def09f28997585b146a18734cba Mon Sep 17 00:00:00 2001 From: Keith Wall Date: Tue, 31 Mar 2026 11:36:08 +0100 Subject: [PATCH 4/5] security - fail closed Signed-off-by: Keith Wall --- proposals/015-entity-isolation.md | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/proposals/015-entity-isolation.md b/proposals/015-entity-isolation.md index 5f77d03..736bcfa 100644 --- a/proposals/015-entity-isolation.md +++ b/proposals/015-entity-isolation.md @@ -62,14 +62,28 @@ then the prefixing removed: alice_orders -> orders ``` +## Security -## Restrictions scope for the first version +The role of the filter is to prevent the entities of one client being visible to another. In other words, prevent an information leak between clients. + +There's the possibility that a new version of the Kafka protocol could expose an entity in a new way. +There could be a new API key or API version of an existing API key that exposes an entity that ought to be isolated. + +If an old version of this filter (one that doesn't know about the new API key/versions) is used with Kafka a client/broker that supports it, an information leak is a possibility. + +In order to prevent this, the filter will employ a `fail-closed` methodology. + +This means it will: +- influence the `ApiVersions` negotiations so that client/broker selects only API key and API versions that are known to the filter. +- if the filter encounters a API key or API version that is unknown to it, it will close the connection with an error. + +## Restrictions in scope for the first version In order to narrow the scope of the problem, the first version of the Entity Isolation Filter will be restricted to the groupId and transactionId entity types. -Topic isolation has some additional challenges which we want to defer to a future release. -https://github.com/kroxylicious/kroxylicious/issues/3504 +Topic isolation has some additional challenges which we want to defer to a future release. These are described in the +following issue https://github.com/kroxylicious/kroxylicious/issues/3504. ### APIs From cd6df561a477a1f46d8bf8391bf09f2f3a47eba7 Mon Sep 17 00:00:00 2001 From: Keith Wall Date: Thu, 9 Apr 2026 10:54:25 +0100 Subject: [PATCH 5/5] Update proposals/015-entity-isolation.md Co-authored-by: Sam Barker Signed-off-by: Keith Wall --- proposals/015-entity-isolation.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/proposals/015-entity-isolation.md b/proposals/015-entity-isolation.md index 736bcfa..f091fa4 100644 --- a/proposals/015-entity-isolation.md +++ b/proposals/015-entity-isolation.md @@ -107,8 +107,6 @@ config: Synchronous API that lets the filter map from downstream names to upstream names and vice versa. It also has the responsibility to filter the upstream view, removing resources that don't belong in the view. -Asynchronous interface that lets the filter map from downstream names to upstream names and vice-versa. It also -has the responsibility to filter the upstream view, removing resources that don't belong in the view. ```java interface EntityNameMapper {