feat(analyzer): add recognizer-level threshold config

CHANGELOG.md

@@ -13,6 +13,7 @@ All notable changes to this project will be documented in this file.
 - Added Python 3.14 package support for `presidio-analyzer` by allowing Python `<3.15` and avoiding `spacy==3.8.14`, which does not provide compatible Python 3.14 wheels.
 
 #### Added
+- Analyzer YAML now accepts `recognizer_score_thresholds` for per-recognizer and per-entity score cutoffs, keyed by `RecognizerResult.RECOGNIZER_NAME_KEY`, while preserving the global `default_score_threshold` fallback when no override matches.
 - Optional `countries` filter on `RecognizerRegistry.load_predefined_recognizers()` to scope predefined country-specific recognizers to a subset of locales (e.g. `countries=["us", "uk"]`). The same filter is also exposed as a top-level `supported_countries` field in the recognizer-registry YAML, mirroring `supported_languages`, and as an advisory per-recognizer `country_code:` field on every predefined country-specific entry in `default_recognizers.yaml` (cross-checked against the class attribute at load time). Country tagging works via two reconciled paths: the class-level `EntityRecognizer.COUNTRY_CODE` ClassVar (canonical for predefined recognizers) and the new `country_code` constructor kwarg on `EntityRecognizer` / `PatternRecognizer` (the path for custom recognizers without a subclass — flows through `PatternRecognizer.from_dict` so YAML `type: custom` entries can declare `country_code:` directly). Conflicting values raise `ValueError` at construction time so a predefined country recognizer can never be silently re-tagged. The resolved tag is read via the `country_code()` and `is_country_specific()` instance methods, and serialized through `to_dict()` / `from_dict()` for round-tripping. Inputs to the `countries` filter are validated up front (rejects bare strings, non-iterables, non-string elements, and blank codes). Locale-agnostic recognizers and untagged custom recognizers are always loaded regardless of the filter, preserving backwards compatibility. Adds `RecognizerRegistry.get_country_codes()` for introspection and a `WARNING` log when a requested country has no matching recognizer. See `docs/analyzer/filtering_by_country.md`. Fixes #1328.
 - Canadian SIN (`CA_SIN`) recognizer for the Canadian Social Insurance Number, using regex pattern matching, context words (English and French), and Luhn checksum validation. Disabled by default.
 - South African ID number (`ZA_ID_NUMBER`) recognizer for the 13-digit national identity number, using pattern matching, context words, birth-date validation, and Luhn checksum validation. Disabled by default.

docs/analyzer/analyzer_engine_provider.md

@@ -70,10 +70,22 @@ recognizer_registry:
 
 The configuration file contains the following parameters:
 
-  - `supported_languages`: A list of supported languages that the analyzer will support.
-  - `default_score_threshold`: A score that determines the minimal threshold for detection.
-  - `nlp_configuration`: Configuration given to the NLP engine which will detect the PIIs and extract features for the downstream logic.
-  - `recognizer_registry`: All the recognizers that will be used by the analyzer. 
+- `supported_languages`: A list of supported languages that the analyzer will support.
+- `default_score_threshold`: A score that determines the minimal threshold for detection.
+- `recognizer_score_thresholds`: Optional per-recognizer thresholds keyed by the recognizer name from `RecognizerResult.RECOGNIZER_NAME_KEY`. Use `default` for a recognizer-wide fallback, then add entity names for overrides.
+- `nlp_configuration`: Configuration given to the NLP engine which will detect the PIIs and extract features for the downstream logic.
+- `recognizer_registry`: All the recognizers that will be used by the analyzer.
+
+Example:
+
+```yaml
+recognizer_score_thresholds:
+  CreditCardRecognizer:
+    default: 0.4
+    CREDIT_CARD: 0.7
+  SpacyRecognizer:
+    PERSON: 0.6
+```
 
 !!! note "Note"
 

docs/tutorial/08_no_code.md

@@ -39,9 +39,17 @@ supported_languages:
   - en
   - es
 default_score_threshold: 0.4
+recognizer_score_thresholds:
+  CreditCardRecognizer:
+    default: 0.4
+    CREDIT_CARD: 0.7
+  SpacyRecognizer:
+    PERSON: 0.6
 """
 ```
 
+Use this when a recognizer needs a lower or higher cutoff than the global `default_score_threshold`, with entity-specific overrides taking priority over the recognizer default.
+
 ### Recognizer Registry parameters
 
 ([default file](https://github.com/microsoft/presidio/blob/main/presidio-analyzer/presidio_analyzer/conf/default_recognizers.yaml))

presidio-analyzer/presidio_analyzer/analyzer_engine.py

@@ -2,7 +2,7 @@
 import logging
 import os
 from collections import Counter
-from typing import List, Optional
+from typing import Dict, List, Optional, Union
 
 import regex as re
 
@@ -15,6 +15,7 @@
     ContextAwareEnhancer,
     LemmaContextAwareEnhancer,
 )
+from presidio_analyzer.input_validation import ConfigurationValidator
 from presidio_analyzer.nlp_engine import NlpArtifacts, NlpEngine, NlpEngineProvider
 from presidio_analyzer.recognizer_registry import (
     RecognizerRegistry,
@@ -24,6 +25,7 @@
 logger = logging.getLogger("presidio-analyzer")
 
 REGEX_TIMEOUT_SECONDS = int(os.environ.get("REGEX_TIMEOUT_SECONDS", 60))
+RecognizerScoreThresholds = Dict[str, Union[float, Dict[str, float]]]
 
 class AnalyzerEngine:
     """
@@ -40,6 +42,8 @@ class AnalyzerEngine:
     defines whether the decision process within the analyzer should be logged or not.
     :param default_score_threshold: Minimum confidence value
     for detected entities to be returned
+    :param recognizer_score_thresholds: Optional recognizer and entity-specific
+    score thresholds applied when no request-level score threshold is provided.
     :param supported_languages: List of possible languages this engine could be run on.
     Used for loading the right NLP models and recognizers for these languages.
     :param context_aware_enhancer: instance of type ContextAwareEnhancer for enhancing
@@ -56,6 +60,7 @@ def __init__(
         default_score_threshold: float = 0,
         supported_languages: List[str] = None,
         context_aware_enhancer: Optional[ContextAwareEnhancer] = None,
+        recognizer_score_thresholds: Optional[RecognizerScoreThresholds] = None,
     ):
         if not supported_languages:
             supported_languages = ["en"]
@@ -102,6 +107,9 @@ def __init__(
 
         self.log_decision_process = log_decision_process
         self.default_score_threshold = default_score_threshold
+        self.recognizer_score_thresholds = (
+            self.__normalize_recognizer_score_thresholds(recognizer_score_thresholds)
+        )
 
         if not context_aware_enhancer:
             logger.debug(
@@ -254,9 +262,10 @@ def analyze(
                 json.dumps([str(result.to_dict()) for result in results]),
             )
 
-        # Remove duplicates or low score results
-        results = EntityRecognizer.remove_duplicates(results)
+        # Filter low-score results before deduplication so recognizer-specific
+        # thresholds do not get lost when duplicate spans collapse.
         results = self.__remove_low_scores(results, score_threshold)
+        results = EntityRecognizer.remove_duplicates(results)
 
         if allow_list:
             results = self._remove_allow_list(
@@ -340,11 +349,50 @@ def __remove_low_scores(
         :return: List[RecognizerResult]
         """
         if score_threshold is None:
-            score_threshold = self.default_score_threshold
+            return [
+                result
+                for result in results
+                if result.score >= self.__get_result_score_threshold(result)
+            ]
 
         new_results = [result for result in results if result.score >= score_threshold]
         return new_results
 
+    @staticmethod
+    def __normalize_recognizer_score_thresholds(
+        recognizer_score_thresholds: Optional[RecognizerScoreThresholds]
+    ) -> Dict[str, Dict[str, float]]:
+        """Normalize shorthand threshold values into the nested mapping shape."""
+        if recognizer_score_thresholds is None:
+            return {}
+        return ConfigurationValidator.validate_recognizer_score_thresholds(
+            recognizer_score_thresholds
+        )
+
+    def __get_result_score_threshold(self, result: RecognizerResult) -> float:
+        """Resolve the threshold to apply for a single recognizer result."""
+        if not self.recognizer_score_thresholds:
+            return self.default_score_threshold
+
+        metadata = result.recognition_metadata or {}
+        recognizer_name = metadata.get(RecognizerResult.RECOGNIZER_NAME_KEY)
+        if not recognizer_name:
+            return self.default_score_threshold
+
+        recognizer_thresholds = self.recognizer_score_thresholds.get(recognizer_name)
+        if not recognizer_thresholds:
+            return self.default_score_threshold
+
+        entity_threshold = recognizer_thresholds.get(result.entity_type)
+        if entity_threshold is not None:
+            return entity_threshold
+
+        recognizer_default_threshold = recognizer_thresholds.get("default")
+        if recognizer_default_threshold is not None:
+            return recognizer_default_threshold
+
+        return self.default_score_threshold
+
     @staticmethod
     def _remove_allow_list(
         results: List[RecognizerResult],

presidio-analyzer/presidio_analyzer/analyzer_engine_provider.py

@@ -88,6 +88,9 @@ def create_engine(self) -> AnalyzerEngine:
         nlp_engine = self._load_nlp_engine()
         supported_languages = self.configuration.get("supported_languages", ["en"])
         default_score_threshold = self.configuration.get("default_score_threshold", 0)
+        recognizer_score_thresholds = self.configuration.get(
+            "recognizer_score_thresholds", {}
+        )
 
         registry = self._load_recognizer_registry(
             supported_languages=supported_languages, nlp_engine=nlp_engine
@@ -98,6 +101,7 @@ def create_engine(self) -> AnalyzerEngine:
             registry=registry,
             supported_languages=supported_languages,
             default_score_threshold=default_score_threshold,
+            recognizer_score_thresholds=recognizer_score_thresholds,
         )
 
         return analyzer

presidio-analyzer/presidio_analyzer/conf/default_analyzer_full.yaml

@@ -1,6 +1,12 @@
 supported_languages: 
   - en
 default_score_threshold: 0
+# recognizer_score_thresholds:
+#   CreditCardRecognizer:
+#     default: 0.4
+#     CREDIT_CARD: 0.7
+#   SpacyRecognizer:
+#     PERSON: 0.6
 nlp_configuration:
   nlp_engine_name: spacy
   models:

presidio-analyzer/presidio_analyzer/input_validation/schemas.py

@@ -44,6 +44,88 @@ def validate_score_threshold(threshold: float) -> float:
             )
         return threshold
 
+    @staticmethod
+    def validate_recognizer_score_thresholds(
+        thresholds: Dict[str, Any],
+    ) -> Dict[str, Dict[str, float]]:
+        """Validate recognizer-specific score thresholds."""
+        if not isinstance(thresholds, dict):
+            raise ValueError("recognizer_score_thresholds must be a dictionary")
+
+        validated_thresholds: Dict[str, Dict[str, float]] = {}
+        for recognizer_name, recognizer_thresholds in thresholds.items():
+            if (
+                not isinstance(recognizer_name, str)
+                or recognizer_name.strip() != recognizer_name
+                or not recognizer_name
+            ):
+                raise ValueError(
+                    "recognizer_score_thresholds keys must be non-empty strings"
+                )
+
+            if isinstance(recognizer_thresholds, (int, float)) and not isinstance(
+                recognizer_thresholds, bool
+            ):
+                validated_thresholds[recognizer_name] = {
+                    "default": ConfigurationValidator._validate_named_threshold(
+                        recognizer_name=recognizer_name,
+                        threshold_name="default",
+                        threshold_value=recognizer_thresholds,
+                    )
+                }
+                continue
+
+            if not isinstance(recognizer_thresholds, dict):
+                raise ValueError(
+                    f"recognizer_score_thresholds for recognizer '{recognizer_name}' "
+                    f"must be a dictionary"
+                )
+
+            validated_recognizer_thresholds: Dict[str, float] = {}
+            for threshold_name, threshold_value in recognizer_thresholds.items():
+                if (
+                    not isinstance(threshold_name, str)
+                    or threshold_name.strip() != threshold_name
+                    or not threshold_name
+                ):
+                    raise ValueError(
+                        "recognizer_score_thresholds nested keys must be "
+                        "non-empty strings"
+                    )
+                if not isinstance(threshold_value, (int, float)) or isinstance(
+                    threshold_value, bool
+                ):
+                    raise ValueError(
+                        "recognizer_score_thresholds values must be numeric"
+                    )
+
+                validated_recognizer_thresholds[threshold_name] = (
+                    ConfigurationValidator._validate_named_threshold(
+                        recognizer_name=recognizer_name,
+                        threshold_name=threshold_name,
+                        threshold_value=threshold_value,
+                    )
+                )
+
+            validated_thresholds[recognizer_name] = validated_recognizer_thresholds
+
+        return validated_thresholds
+
+    @staticmethod
+    def _validate_named_threshold(
+        recognizer_name: str,
+        threshold_name: str,
+        threshold_value: float,
+    ) -> float:
+        """Validate one named recognizer threshold and keep config context."""
+        try:
+            return ConfigurationValidator.validate_score_threshold(threshold_value)
+        except ValueError as exc:
+            raise ValueError(
+                "recognizer_score_thresholds value for "
+                f"'{recognizer_name}.{threshold_name}' {exc}"
+            ) from exc
+
     @staticmethod
     def validate_nlp_configuration(config: Dict[str, Any]) -> Dict[str, Any]:
         """Validate NLP configuration structure.
@@ -112,6 +194,7 @@ def validate_analyzer_configuration(config: Dict[str, Any]) -> Dict[str, Any]:
         valid_keys = {
             "supported_languages",
             "default_score_threshold",
+            "recognizer_score_thresholds",
             "nlp_configuration",
             "recognizer_registry",
         }
@@ -135,6 +218,13 @@ def validate_analyzer_configuration(config: Dict[str, Any]) -> Dict[str, Any]:
                 config["default_score_threshold"]
             )
 
+        if "recognizer_score_thresholds" in config:
+            config["recognizer_score_thresholds"] = (
+                ConfigurationValidator.validate_recognizer_score_thresholds(
+                config["recognizer_score_thresholds"]
+            )
+            )
+
         # Validate nested configurations
         if "nlp_configuration" in config:
             ConfigurationValidator.validate_nlp_configuration(