Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 29 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -286,8 +286,30 @@ fields. Runtime dictionary terms remain normalized literals; language
dictionaries are the supported boundary for maintained language-specific rule
data. Runtime calls such as `setStrict`, `setLoose`, `addStrict`, and `addLoose`
only mutate the returned filter instance; they do not change the compiled
dictionary object or other filters created from it. This release intentionally
does not add another built-in language or a separately published package.
dictionary object or other filters created from it.

The package also ships a reviewed, opt-in English language pack:

```ts
import {
createEnglishProfanityFilter,
englishProfanityDictionary,
englishProfanityFilter,
} from "@textfilters/profanity";

englishProfanityFilter.check("fucking");
englishProfanityFilter.censor("fucking");

const mutableEnglishFilter = createEnglishProfanityFilter();
mutableEnglishFilter.addStrict("tenant-only-term");
```

`englishProfanityFilter` is a shared read-only filter initialized on first use.
`createEnglishProfanityFilter()` returns an isolated mutable filter, while
`englishProfanityDictionary` exposes the validated source dictionary. Importing
the package does not add English rules to `filter`; the shared default remains
Russian-only. The English pack is intentionally limited to its reviewed audit
vocabulary and is not a broad toxicity or hate-speech classifier.

The Russian dictionary is maintained as split family data with an explicit rule
order. New high-risk family rules are expected to add nearby coverage and
Expand Down Expand Up @@ -355,13 +377,12 @@ For future external language pack guidance, see
source dictionary shape, stable ids, taxonomy metadata, strict and loose views,
human-maintained JSON, and conformance expectations.
The [external language pack policy](docs/external-language-pack-policy.md)
defines when the project is ready to create a real external package and keeps
the built-in Russian dictionary in this package for now.
defines when the project is ready to create a separately published language
package and keeps the built-in dictionaries in this package for now.

A reviewed, opt-in English dictionary is available as a private non-published
[English language pack example](examples/english-language-pack/README.md). It
does not change the Russian shared default and remains intentionally limited to
the reviewed audit vocabulary until English-language ownership is established.
The [English language pack example](examples/english-language-pack/README.md)
documents the reviewed vocabulary and re-exports the package-owned
implementation from `src/languages/en` for repository-level examples.

## Related Textfilters Packages

Expand Down
13 changes: 9 additions & 4 deletions docs/external-language-pack-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,12 @@ until ongoing maintenance is credible.

## Current Boundaries

The repository contains a real reviewed English dictionary as a private,
non-published example. It remains an opt-in test and authoring artifact until a
named maintainer or maintenance group satisfies the ownership criteria above.
It does not change the built-in Russian dictionary or the shared default filter.
The main package contains a reviewed English dictionary under
`src/languages/en` and exposes it through an opt-in public API. It is not a
separately published language package, does not change the built-in Russian
dictionary, and does not add English rules to the shared default filter.

A future separately published English package still requires a named maintainer
or maintenance group that satisfies the ownership criteria above. Shipping a
limited opt-in dictionary in the main package does not by itself satisfy those
external-package requirements.
4 changes: 2 additions & 2 deletions docs/language-pack-authoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,8 +320,8 @@ The following work is intentionally out of scope for this milestone:

- creating `@textfilters/profanity-ru` or any other external language package;
- moving the Russian dictionary out of this package;
- adding another real language dictionary beyond the private reviewed English
pack;
- adding another real language dictionary beyond the reviewed English pack in
`src/languages/en`;
- broadening profanity coverage beyond the reviewed English vocabulary;
- changing Russian runtime behavior;
- adding generated matcher metadata to source dictionaries;
Expand Down
25 changes: 16 additions & 9 deletions examples/english-language-pack/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Reviewed English Profanity Language Pack

This directory contains a real, opt-in English dictionary for the reviewed
audit vocabulary. It is deliberately non-published: the package is marked
`private`, has no release workflow, and remains an in-repository example until
an English-language maintainer or maintenance group is named.
The package-owned implementation of this reviewed, opt-in English dictionary
lives in `src/languages/en` and is included in the published distribution. This
private example re-exports that implementation so repository examples exercise
the same code as package consumers.

The pack covers only `fuck`, `fucking`, `fucked`, `shit`, `dick`,
`motherfucker` (including detection before a possessive suffix), `cock`,
Expand All @@ -12,20 +12,26 @@ or contextual moderation.

## Opt-in API

The example exports a validated dictionary and an isolated filter:
The public package exports a validated dictionary, a shared read-only filter,
and a factory for isolated mutable filters:

```ts
import {
createEnglishProfanityFilter,
englishProfanityDictionary,
englishProfanityFilter,
} from "./src/index.js";
} from "@textfilters/profanity";

englishProfanityFilter.check("reviewed English text");
englishProfanityFilter.analyze("reviewed English text");

const mutableEnglishFilter = createEnglishProfanityFilter();
mutableEnglishFilter.addStrict("tenant-only-term");
```

Importing the main `@textfilters/profanity` package does not enable these
rules. The built-in shared filter remains Russian-only.
Importing the main package does not enable these rules on the default `filter`.
The built-in shared default remains Russian-only, and the English filter is
initialized only when its methods are first called.

## Policy boundaries

Expand All @@ -41,5 +47,6 @@ rules. The built-in shared filter remains Russian-only.
retain the normal runtime matching contract.
- The possessive suffix in `motherfucker's` is not part of the reported range;
only the maintained profanity token is censored.
- Publication requires the ownership and release criteria in
- A separately published language package requires the ownership and release
criteria in
[the external language pack policy](../../docs/external-language-pack-policy.md).
3 changes: 0 additions & 3 deletions examples/english-language-pack/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@
"description": "Non-published reviewed English profanity language pack.",
"license": "UNLICENSED",
"sideEffects": false,
"dependencies": {
"@textfilters/core": "^0.3.1"
},
"files": [
"dist",
"README.md"
Expand Down
57 changes: 1 addition & 56 deletions examples/english-language-pack/src/dictionary.ts
Original file line number Diff line number Diff line change
@@ -1,56 +1 @@
import type { ProfanityLanguageDictionary } from "@textfilters/profanity";

export const englishProfanityDictionary = {
language: "en",
rules: [
{
id: "en.obscene.fuck.family",
category: "OBSCENE_MAT",
severity: "high",
source: "fuck(?:ed|ing)?",
match: { strict: {} },
},
{
id: "en.obscene.shit",
category: "OBSCENE_MAT",
severity: "high",
source: "shit",
match: { strict: {} },
},
{
id: "en.vulgar.dick",
category: "VULGAR",
severity: "medium",
source: "dick",
match: { strict: {} },
},
{
id: "en.insult.motherfucker",
category: "STRONG_INSULT",
severity: "high",
source: "motherfucker",
match: { strict: {} },
},
{
id: "en.vulgar.cock",
category: "VULGAR",
severity: "medium",
source: "cock",
match: { strict: {} },
},
{
id: "en.insult.bitch",
category: "STRONG_INSULT",
severity: "medium",
source: "bitch",
match: { strict: {} },
},
{
id: "en.insult.bastard",
category: "STRONG_INSULT",
severity: "medium",
source: "bastard",
match: { strict: {} },
},
],
} as const satisfies ProfanityLanguageDictionary;
export { englishProfanityDictionary } from "../../../src/languages/en/dictionary.js";
Loading