🇵🇱 pizza-analysis-stempel

Polish algorithmic stemming (Stempel) for INFINI Pizza

---

Overview

Algorithmic stemmer for Polish using the Stempel algorithm — a table-driven stemmer that applies learned suffix-removal patterns from a compiled automaton. Compared to Morfologik's dictionary lookup, Stempel is lighter weight and can handle out-of-vocabulary words, but may produce non-dictionary stems.

Components

Type	Name	Description
TokenFilter	stempel_stem	Polish Stempel algorithmic stemmer
TokenFilter	polish_stop	Polish stop words

Stempel vs Morfologik

Feature	Stempel	Morfologik
Approach	Algorithmic (learned rules)	Dictionary (FSA lookup)
OOV handling	Applies rules to unknown words	Falls through unchanged
Output	Stems (may not be real words)	Lemmas (dictionary forms)
Size	Smaller model	Larger dictionary

Choose Stempel when you need broader coverage; choose Morfologik when accuracy matters more.

Example

use pizza_engine::analysis::AnalysisFactory;

let mut factory = AnalysisFactory::new();
pizza_analysis_stempel::register_all(&mut factory);

let stem = factory.get_token_filter("stempel_stem").unwrap();

Installation

[dependencies]
pizza-analysis-stempel = "0.1"

Or via pizza-analysis-all:

[dependencies]
pizza-analysis-all = { version = "0.1", features = ["stempel"] }

License

MIT

---

_{Part of the INFINI Pizza ecosystem}