Repository navigation
perf(kernel): borne la fuite de spans, -300 ms d'imports, YAML x24 - #314
Conversation
… échouaient en mode ouvert Les trois premiers points de #305 dans l'ordre que l'issue conseille, la moitié mesurable de #306, le fond de #310, et un défaut trouvé en route. #305.1 — FUITE : `Tracer._spans` était une liste que rien ne vidait, alors que `TracingMiddleware` enveloppe CHAQUE appel de plugin, tracing désactivé compris. Un Span fuyait donc par appel : ~562 octets, près de 5 Go par jour et par worker à 100 req/s. C'est un `deque(maxlen=…)`, réglable par `observability.tracing.max_local_spans` (défaut 1000, 0 pour ne rien garder) ; un réglage aberrant ne peut pas redonner `maxlen=None`, qui serait la fuite d'origine. `export()` rend les spans les plus RÉCENTS, ce qui est tout ce qu'un historique de débogage doit montrer. Non fait, et c'était un « considérer » de l'issue : ne pas construire le Span du tout quand personne ne le lit. `span()` doit yielder un objet, et rendre `trace_id` / `span_id` paresseux changerait la forme publique d'une dataclass pour ~2 µs sur un appel de 38 µs. #305.2 et #306 — IMPORTS : `xcore/sdk/__init__.py` importait `..services.xworker` (celery) et le package externe `xcoresdk` (qui tire SQLAlchemy par son module d'authentification). Les deux étaient payés par toute application qui fait `import xcore`, et par CHAQUE worker sandbox à chaque recyclage, même sans service de tâches ni base configurés. Résolus à la demande (PEP 562) : import xcore : 593,9 ms → 294,1 ms (celery et sqlalchemy : 0,0 ms) `find_spec` résout sans exécuter (0,05 ms), donc `__all__` ne change pas et `from xcore.sdk import *` expose les mêmes noms qu'avant. Le premier accès paie l'import, et lui seul. #305.4 — YAML : les trois lecteurs du noyau passaient par le loader PYTHON de PyYAML alors que libyaml est là. Un module feuille `xcore/kernel/yamlio.py` (aucun import xcore : le worker sandbox garde son jeu minimal) choisit `CSafeLoader`, avec repli silencieux sur `SafeLoader`. Même sûreté — un tag d'objet arbitraire reste refusé, c'est testé. Mesuré sur un manifeste : 33,97 ms → 1,39 ms (×24). #305.3 — DOC : le coût par appel de chaque couche d'observabilité, et le fait que l'exportateur console est synchrone, donc réservé au développement. #310 — ORDONNANCEUR : sans Redis il n'y a AUCUN verrou, et chaque processus exécute chaque job — deux workers, deux paies. Une bibliothèque ne peut pas savoir combien de processus tournent (uvicorn `--workers` est décidé dehors), donc pas de refus : un avertissement unique au démarrage, qui NOMME les jobs concernés. Deux replis silencieux deviennent en revanche des erreurs, parce qu'ils trompaient l'opérateur : `backend: redis` sans les paquets redis (il croyait ses jobs protégés) et un backend inconnu comme `database`, documenté mais pas implémenté. Les deux marquent le service DEGRADED au lieu de se taire. TROUVÉ EN ROUTE — le scan de sécurité échouait en mode OUVERT. `ASTScanner.scan` a deux phases d'imports : l'extension compilée `scanner_core` quand elle est là, l'AST Python sinon. Seule la seconde signalait une erreur de syntaxe. Avec le scanner compilé — la configuration optimisée, celle qui est livrée — un plugin dont une source ne se parse pas revenait donc `passed=True`, sans qu'aucun de ses imports, builtins ou attributs interdits n'ait pu être examiné. Ne pas savoir lire un fichier n'est pas l'avoir trouvé sain : `_check_builtins_and_attrs`, qui parse dans les DEUX phases, le dit maintenant, une fois et une seule. C'est ce qui faisait échouer `test_scan_syntax_error` dans deux fichiers de tests, avant comme après ce travail (vérifié sur l'arbre d'origine). `test_yaml_import_error` simulait l'absence de PyYAML en neutralisant `sys.modules["yaml"]` : il neutralise aussi `xcore.kernel.yamlio`, puisque c'est par là que passe désormais la lecture. Le contrat testé est le même (ManifestError « pyyaml non installé »). Tests : 1516 passent, 0 échec (dont 2 qui échouaient avant). black, isort et flake8 sont muets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reviewer's GuideData structuresBound the tracer’s in-memory span history and add the configurable limit that controls its size.
APIMake SDK exports for worker, SQL, and external SDK features lazy while preserving their public names and optional-dependency errors.
Provide a leaf YAML-loading interface that uses PyYAML’s safe C loader when available and safely falls back otherwise.
Uses of the APIRoute configuration, sandbox-manifest, and security-manifest YAML reads through the new shared loader.
ImplementationMake syntax and file-read failures fail the security scan rather than allowing unparsable files to pass.
Report missing or unsupported scheduler locks as degraded operation and warn once with the affected jobs when running unlocked.
DocumentationDocument measured per-call observability costs, console-exporter limitations, and the bounded local span history.
TestsCover bounded span retention, recent-span export, invalid limits, and span API behavior.
Check YAML loader parity, safe rejection, stream handling, and C-loader selection.
Verify lazy SDK imports and preserved exports in fresh interpreters.
Adapt the missing-PyYAML test to the new loader module and cover fail-closed scanning across scanner implementations.
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've found 9 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="xcore/services/scheduler/service.py" line_range="277" />
<code_context>
+ """
+ if _REDIS_LOCK_CLIENT is not None:
+ return
+ jobs = [str(j.get("id") or j.get("func") or "?") for j in self._config.jobs]
+ if not jobs and not _JOB_REGISTRY:
+ return # aucun job : rien à dire
</code_context>
<issue_to_address>
**Malformed job config aborts boot**
When `services.scheduler.jobs` contains a non-mapping entry, such as `null`, `_add_job_from_config` catches and logs invalid job entries, but `_warn_if_unlocked` then calls `.get()` on each entry without checking its type. A non-mapping entry raises `AttributeError` during scheduler initialization and aborts application boot instead of skipping the bad job.
Filter or safely normalize job entries before reading `id` and `func` in `_warn_if_unlocked`.
</issue_to_address>
### Comment 2
<location path="xcore/services/scheduler/service.py" line_range="254" />
<code_context>
self._scheduler.start()
- self._status = ServiceStatus.READY
+ self._status = ServiceStatus.DEGRADED if degraded else ServiceStatus.READY
logger.info(
"scheduler started",
</code_context>
<issue_to_address>
**Unavailable Redis still reports ready**
When redis packages are installed but the configured Redis service is down or unreachable, `SchedulerService.init` treats successful Redis client construction as proof that the distributed lock works, although `from_url` does not verify connectivity; it can set the service to `READY` and report `distributed_lock=True` while `_dispatch_job` later fails at `client.set` and the job does not run.
Validate Redis connectivity during initialization, mark the service degraded when it fails, and handle lock-acquisition failures explicitly so job failures are visible and do not masquerade as a healthy scheduler.
Also at `xcore/services/scheduler/service.py:259`.
</issue_to_address>
### Comment 3
<location path="xcore/configurations/sections.py" line_range="322" />
<code_context>
+ # récents. Ce n'est pas le pipeline d'export OTel : c'est un historique de débogage, et
+ # il doit être borné — une liste sans limite faisait ~562 octets par appel, soit près de
+ # 5 Go par jour et par worker à 100 req/s (#305). 0 = ne rien garder.
+ max_local_spans: int = 1000
</code_context>
<issue_to_address>
**Configured span history is ignored**
When `observability.tracing.max_local_spans` is set in application configuration, `ConfigLoader._parse_observability` does not pass `max_local_spans` into `TracingConfig`, so the default limit is used and the configured history size is ignored.
Pass the configured `max_local_spans` value into `TracingConfig` in `ConfigLoader._parse_observability`.
Also at `xcore/configurations/sections.py:323`.
</issue_to_address>
### Comment 4
<location path="xcore/kernel/observability/tracing.py" line_range="54" />
<code_context>
+ value = getattr(config, "max_local_spans", DEFAULT_MAX_LOCAL_SPANS)
+ if isinstance(value, bool) or not isinstance(value, int):
+ return DEFAULT_MAX_LOCAL_SPANS
+ return max(value, 0)
+
+
</code_context>
<issue_to_address>
**Tracer startup fails on large limits**
When `max_local_spans` exceeds the platform’s `Py_ssize_t` range, `_max_local_spans` returns the value unchanged, so `deque(maxlen=...)` raises `OverflowError` during `Tracer` construction and prevents startup.
Clamp or reject values that cannot be represented as a `Py_ssize_t` before passing them to `deque`.
</issue_to_address>
### Comment 5
<location path="xcore/sdk/__init__.py" line_range="196" />
<code_context>
+# `find_spec` RÉSOUT sans exécuter (mesuré : 0,05 ms) : `from xcore.sdk import *` ne doit
+# pas lever pour une dépendance que l'utilisateur n'a pas, et ne doit pas non plus importer
+# celery ou SQLAlchemy juste pour savoir s'ils sont là.
+__all__ += list(_WORKER)
+if importlib.util.find_spec("sqlalchemy") is not None:
+ __all__ += list(_SQL_ADAPTERS)
</code_context>
<issue_to_address>
**Wildcard imports require Celery**
When an application without the optional Celery dependency uses a wildcard import from `xcore.sdk`, `__all__` always includes the worker names, so `from xcore.sdk import *` resolves `WorkerService` and triggers the lazy Celery import. Without the optional worker dependency, the wildcard import raises `ImportError` even for callers that do not use worker functionality.
Only include worker names in `__all__` when Celery is available, or otherwise ensure wildcard import does not resolve unavailable optional names.
</issue_to_address>
### Comment 6
<location path="xcore/services/scheduler/service.py" line_range="279" />
<code_context>
+ return
+ jobs = [str(j.get("id") or j.get("func") or "?") for j in self._config.jobs]
+ if not jobs and not _JOB_REGISTRY:
+ return # aucun job : rien à dire
+ logger.warning(
+ "scheduler sans verrou : en multi-processus, CHAQUE processus exécutera ces jobs",
</code_context>
<issue_to_address>
**Unprotected jobs go unwarned**
When the scheduler starts with no jobs and jobs are registered afterward, `SchedulerService.init` calls `_warn_if_unlocked` before plugin or caller registrations, and the empty-job return suppresses the warning permanently. The later jobs run in every process without operators being warned about duplicate executions.
Emit the unlocked-scheduler warning when jobs are registered after initialization, or otherwise defer it until those registrations are available.
Also at `xcore/services/scheduler/service.py:261`, `xcore/services/scheduler/service.py:278`.
</issue_to_address>
### Comment 7
<location path="xcore/kernel/observability/tracing.py" line_range="155" />
<code_context>
+ # tout, même tracing désactivé). Une liste sans limite gardait donc un Span par appel
+ # pour toujours — une fuite mesurée à ~562 octets par appel (#305). Un deque garde les
+ # N plus récents, ce qui est tout ce que `export()` sert à montrer.
+ self._spans: deque[Span] = deque(maxlen=_max_local_spans(config))
self._otel_tracer = None
</code_context>
<issue_to_address>
**Concurrent exports fail**
When `Tracer.export()` runs while another thread completes a span, `Tracer.export()` iterates `self._spans` while span completion appends to the deque; the iterator raises `RuntimeError` and the caller gets no history.
Synchronize deque access so `Tracer.export()` iterates a stable snapshot of the spans.
</issue_to_address>
### Comment 8
<location path="tests/unit/security/test_scanner_fails_closed.py" line_range="42-44" />
<code_context>
+ ):
+ """The whole point: the C++ path and the Python path must agree on an unparsable file."""
+ plugin = _plugin(tmp_path, "def broken(\n pass")
+ with_cpp = ASTScanner()
+ without_cpp = ASTScanner()
+ monkeypatch.setattr(without_cpp, "_cpp", None)
+ assert with_cpp.scan(plugin).passed is without_cpp.scan(plugin).passed is False
+
</code_context>
<issue_to_address>
**Compiled scanner path goes untested**
When the test suite runs without the optional C++ scanner extension, both `ASTScanner` instances use the Python fallback, so the assertion compares that path with itself and the compiled scanner’s fail-closed behavior is not tested.
Require the compiled scanner for this test and verify that `with_cpp` uses it before asserting its behavior.
</issue_to_address>
### Comment 9
<location path="tests/unit/sdk/test_lazy_imports.py" line_range="61-63" />
<code_context>
+ """`__all__` must not shrink: a plugin doing `from xcore.sdk import *` kept working."""
+ out = _run(
+ "import xcore.sdk as s\n"
+ "names = set(s.__all__)\n"
+ "print(all(n in names for n in ('WorkerService', 'task', 'task_registry')))"
+ )
+ assert out == "True"
+
</code_context>
<issue_to_address>
**Star import can still fail**
When an export included in `__all__` beyond the three checked worker names is unavailable or mis-mapped, the test checks only that three worker names appear in `__all__`; it never executes `from xcore.sdk import *`. When an available conditional lazy export, such as a SQL adapter, is listed but cannot be resolved, the star import raises while this assertion still passes.
Run an actual star import in a fresh interpreter and verify the expected names are bound.
</issue_to_address>Sourcery assessment
Needs a human reviewer. 9 findings to address first, and the bounded deque permanently drops older spans from Tracer.export(); reverting restores future history collection, but cannot recover spans already evicted. That loss is limited to in-memory debugging data, while the scanner and scheduler changes alter reversible runtime behavior.
Blocking findings: xcore/services/scheduler/service.py:277, xcore/services/scheduler/service.py:254, xcore/configurations/sections.py:322, xcore/kernel/observability/tracing.py:54, xcore/sdk/__init__.py:196, and 4 more
| """ | ||
| if _REDIS_LOCK_CLIENT is not None: | ||
| return | ||
| jobs = [str(j.get("id") or j.get("func") or "?") for j in self._config.jobs] |
There was a problem hiding this comment.
🟡 Medium · Malformed job config aborts boot
When services.scheduler.jobs contains a non-mapping entry, such as null, _add_job_from_config catches and logs invalid job entries, but _warn_if_unlocked then calls .get() on each entry without checking its type. A non-mapping entry raises AttributeError during scheduler initialization and aborts application boot instead of skipping the bad job.
Filter or safely normalize job entries before reading id and func in _warn_if_unlocked.
Prompt for AI agents
In `xcore/services/scheduler/service.py` at line 277:
**Malformed job config aborts boot**
When `services.scheduler.jobs` contains a non-mapping entry, such as `null`, `_add_job_from_config` catches and logs invalid job entries, but `_warn_if_unlocked` then calls `.get()` on each entry without checking its type. A non-mapping entry raises `AttributeError` during scheduler initialization and aborts application boot instead of skipping the bad job.
Filter or safely normalize job entries before reading `id` and `func` in `_warn_if_unlocked`.|
|
||
| self._scheduler.start() | ||
| self._status = ServiceStatus.READY | ||
| self._status = ServiceStatus.DEGRADED if degraded else ServiceStatus.READY |
There was a problem hiding this comment.
🟠 High · Unavailable Redis still reports ready
When redis packages are installed but the configured Redis service is down or unreachable, SchedulerService.init treats successful Redis client construction as proof that the distributed lock works, although from_url does not verify connectivity; it can set the service to READY and report distributed_lock=True while _dispatch_job later fails at client.set and the job does not run.
Validate Redis connectivity during initialization, mark the service degraded when it fails, and handle lock-acquisition failures explicitly so job failures are visible and do not masquerade as a healthy scheduler.
Also at xcore/services/scheduler/service.py:259.
Prompt for AI agents
In `xcore/services/scheduler/service.py` at line 254:
**Unavailable Redis still reports ready**
When redis packages are installed but the configured Redis service is down or unreachable, `SchedulerService.init` treats successful Redis client construction as proof that the distributed lock works, although `from_url` does not verify connectivity; it can set the service to `READY` and report `distributed_lock=True` while `_dispatch_job` later fails at `client.set` and the job does not run.
Validate Redis connectivity during initialization, mark the service degraded when it fails, and handle lock-acquisition failures explicitly so job failures are visible and do not masquerade as a healthy scheduler.
Also at `xcore/services/scheduler/service.py:259`.| # récents. Ce n'est pas le pipeline d'export OTel : c'est un historique de débogage, et | ||
| # il doit être borné — une liste sans limite faisait ~562 octets par appel, soit près de | ||
| # 5 Go par jour et par worker à 100 req/s (#305). 0 = ne rien garder. | ||
| max_local_spans: int = 1000 |
There was a problem hiding this comment.
🟡 Medium · Configured span history is ignored
When observability.tracing.max_local_spans is set in application configuration, ConfigLoader._parse_observability does not pass max_local_spans into TracingConfig, so the default limit is used and the configured history size is ignored.
Pass the configured max_local_spans value into TracingConfig in ConfigLoader._parse_observability.
Also at xcore/configurations/sections.py:323.
Prompt for AI agents
In `xcore/configurations/sections.py` at line 322:
**Configured span history is ignored**
When `observability.tracing.max_local_spans` is set in application configuration, `ConfigLoader._parse_observability` does not pass `max_local_spans` into `TracingConfig`, so the default limit is used and the configured history size is ignored.
Pass the configured `max_local_spans` value into `TracingConfig` in `ConfigLoader._parse_observability`.
Also at `xcore/configurations/sections.py:323`.| value = getattr(config, "max_local_spans", DEFAULT_MAX_LOCAL_SPANS) | ||
| if isinstance(value, bool) or not isinstance(value, int): | ||
| return DEFAULT_MAX_LOCAL_SPANS | ||
| return max(value, 0) |
There was a problem hiding this comment.
🟡 Medium · Tracer startup fails on large limits
When max_local_spans exceeds the platform’s Py_ssize_t range, _max_local_spans returns the value unchanged, so deque(maxlen=...) raises OverflowError during Tracer construction and prevents startup.
Clamp or reject values that cannot be represented as a Py_ssize_t before passing them to deque.
Prompt for AI agents
In `xcore/kernel/observability/tracing.py` at line 54:
**Tracer startup fails on large limits**
When `max_local_spans` exceeds the platform’s `Py_ssize_t` range, `_max_local_spans` returns the value unchanged, so `deque(maxlen=...)` raises `OverflowError` during `Tracer` construction and prevents startup.
Clamp or reject values that cannot be represented as a `Py_ssize_t` before passing them to `deque`.| # `find_spec` RÉSOUT sans exécuter (mesuré : 0,05 ms) : `from xcore.sdk import *` ne doit | ||
| # pas lever pour une dépendance que l'utilisateur n'a pas, et ne doit pas non plus importer | ||
| # celery ou SQLAlchemy juste pour savoir s'ils sont là. | ||
| __all__ += list(_WORKER) |
There was a problem hiding this comment.
🟡 Medium · Wildcard imports require Celery
When an application without the optional Celery dependency uses a wildcard import from xcore.sdk, __all__ always includes the worker names, so from xcore.sdk import * resolves WorkerService and triggers the lazy Celery import. Without the optional worker dependency, the wildcard import raises ImportError even for callers that do not use worker functionality.
Only include worker names in __all__ when Celery is available, or otherwise ensure wildcard import does not resolve unavailable optional names.
Prompt for AI agents
In `xcore/sdk/__init__.py` at line 196:
**Wildcard imports require Celery**
When an application without the optional Celery dependency uses a wildcard import from `xcore.sdk`, `__all__` always includes the worker names, so `from xcore.sdk import *` resolves `WorkerService` and triggers the lazy Celery import. Without the optional worker dependency, the wildcard import raises `ImportError` even for callers that do not use worker functionality.
Only include worker names in `__all__` when Celery is available, or otherwise ensure wildcard import does not resolve unavailable optional names.| return | ||
| jobs = [str(j.get("id") or j.get("func") or "?") for j in self._config.jobs] | ||
| if not jobs and not _JOB_REGISTRY: | ||
| return # aucun job : rien à dire |
There was a problem hiding this comment.
🟡 Medium · Unprotected jobs go unwarned
When the scheduler starts with no jobs and jobs are registered afterward, SchedulerService.init calls _warn_if_unlocked before plugin or caller registrations, and the empty-job return suppresses the warning permanently. The later jobs run in every process without operators being warned about duplicate executions.
Emit the unlocked-scheduler warning when jobs are registered after initialization, or otherwise defer it until those registrations are available.
Also at xcore/services/scheduler/service.py:261, xcore/services/scheduler/service.py:278.
Prompt for AI agents
In `xcore/services/scheduler/service.py` at line 279:
**Unprotected jobs go unwarned**
When the scheduler starts with no jobs and jobs are registered afterward, `SchedulerService.init` calls `_warn_if_unlocked` before plugin or caller registrations, and the empty-job return suppresses the warning permanently. The later jobs run in every process without operators being warned about duplicate executions.
Emit the unlocked-scheduler warning when jobs are registered after initialization, or otherwise defer it until those registrations are available.
Also at `xcore/services/scheduler/service.py:261`, `xcore/services/scheduler/service.py:278`.| # tout, même tracing désactivé). Une liste sans limite gardait donc un Span par appel | ||
| # pour toujours — une fuite mesurée à ~562 octets par appel (#305). Un deque garde les | ||
| # N plus récents, ce qui est tout ce que `export()` sert à montrer. | ||
| self._spans: deque[Span] = deque(maxlen=_max_local_spans(config)) |
There was a problem hiding this comment.
🟡 Medium · Concurrent exports fail
When Tracer.export() runs while another thread completes a span, Tracer.export() iterates self._spans while span completion appends to the deque; the iterator raises RuntimeError and the caller gets no history.
Synchronize deque access so Tracer.export() iterates a stable snapshot of the spans.
Prompt for AI agents
In `xcore/kernel/observability/tracing.py` at line 155:
**Concurrent exports fail**
When `Tracer.export()` runs while another thread completes a span, `Tracer.export()` iterates `self._spans` while span completion appends to the deque; the iterator raises `RuntimeError` and the caller gets no history.
Synchronize deque access so `Tracer.export()` iterates a stable snapshot of the spans.| with_cpp = ASTScanner() | ||
| without_cpp = ASTScanner() | ||
| monkeypatch.setattr(without_cpp, "_cpp", None) |
There was a problem hiding this comment.
🟡 Medium · Compiled scanner path goes untested
When the test suite runs without the optional C++ scanner extension, both ASTScanner instances use the Python fallback, so the assertion compares that path with itself and the compiled scanner’s fail-closed behavior is not tested.
Require the compiled scanner for this test and verify that with_cpp uses it before asserting its behavior.
Prompt for AI agents
In `tests/unit/security/test_scanner_fails_closed.py` at lines 42-44:
**Compiled scanner path goes untested**
When the test suite runs without the optional C++ scanner extension, both `ASTScanner` instances use the Python fallback, so the assertion compares that path with itself and the compiled scanner’s fail-closed behavior is not tested.
Require the compiled scanner for this test and verify that `with_cpp` uses it before asserting its behavior.| "names = set(s.__all__)\n" | ||
| "print(all(n in names for n in ('WorkerService', 'task', 'task_registry')))" | ||
| ) |
There was a problem hiding this comment.
🟡 Medium · Star import can still fail
When an export included in __all__ beyond the three checked worker names is unavailable or mis-mapped, the test checks only that three worker names appear in __all__; it never executes from xcore.sdk import *. When an available conditional lazy export, such as a SQL adapter, is listed but cannot be resolved, the star import raises while this assertion still passes.
Run an actual star import in a fresh interpreter and verify the expected names are bound.
Prompt for AI agents
In `tests/unit/sdk/test_lazy_imports.py` at lines 61-63:
**Star import can still fail**
When an export included in `__all__` beyond the three checked worker names is unavailable or mis-mapped, the test checks only that three worker names appear in `__all__`; it never executes `from xcore.sdk import *`. When an available conditional lazy export, such as a SQL adapter, is listed but cannot be resolved, the star import raises while this assertion still passes.
Run an actual star import in a fresh interpreter and verify the expected names are bound.
Traite #305 points 1, 2, 3 et 4, la moitié mesurable de #306, le fond de #310, et un défaut de sécurité trouvé en route. Chaque gain est mesuré avant/après sur cette machine (Python 3.14.8, 8 cœurs).
#305.1 — La fuite (
bug)Tracer._spansétait unelistque rien ne vidait, alors queTracingMiddlewareenveloppe chaque appel de plugin, tracing désactivé compris : unSpanfuyait par appel (~562 o, près de 5 Go/jour/worker à 100 req/s). C'est maintenant undeque(maxlen=…), réglé parobservability.tracing.max_local_spans(défaut 1000,0pour ne rien garder).Critères d'acceptation de l'issue :
len(tracer._spans)reste borné après N ≫ maxlen appels ✅,export()fonctionne toujours ✅ (et rend les spans les plus récents). Un réglage aberrant (négatif,None, texte) ne peut pas redonnermaxlen=None, qui serait la fuite — c'est testé.Non fait, et l'issue le posait comme un « considérer » : ne pas construire le
Spandu tout quand personne ne le lit.span()doit yielder un objet, et rendretrace_id/span_idparesseux changerait la forme publique d'une dataclass pour ~2 µs sur un appel de 38 µs. À trancher par le mainteneur.#305.2 + #306 — Les imports
xcore/sdk/__init__.pyimportait..services.xworker(celery) et le package externexcoresdk, qui tire SQLAlchemy par son module d'authentification. Payé par toute application qui faitimport xcore, et par chaque worker sandbox à chaque recyclage, sans service de tâches ni base configurés.Critère d'acceptation :
python -X importtime -c "import xcore"ne montre plus celery ni sqlalchemy ✅.Résolus à la demande (PEP 562).
find_specrésout sans exécuter (0,05 ms), donc__all__ne bouge pas etfrom xcore.sdk import *expose les mêmes noms qu'avant — c'est testé dans un interpréteur neuf par cas, parce quesys.modulesest global et qu'un test qui aurait importé celery avant masquerait la régression.Le gain de RSS par worker annoncé dans #306 (79 Mo) n'est pas vérifié ici : il demande un hôte avec des plugins sandboxés en vrai. Seul le temps d'import est mesuré.
#305.4 — Le YAML
Les trois lecteurs du noyau passaient par le loader Python de PyYAML alors que libyaml est installé.
xcore/kernel/yamlio.pychoisitCSafeLoader, avec repli silencieux surSafeLoader(une installation sans libyaml est juste plus lente, ce n'est pas une anomalie). Module feuille, sans aucun import xcore : le worker sandbox garde son jeu d'imports minimal, comme #306 le demande.Mesuré sur un manifeste : 33,97 ms → 1,39 ms (×24). Même sûreté que
safe_load— un tag!!python/object/applyreste refusé, et c'est testé.#305.3 — La doc
doc/observability/observability.mddonne le coût par appel de chaque couche (le span OTel est le seul significatif : +57 µs, ×2,5) et dit que l'exportateur console est synchrone, donc réservé au développement.#310 — L'ordonnanceur
Sans Redis il n'y a aucun verrou et chaque processus exécute chaque job : deux workers, deux paies. Une bibliothèque ne peut pas savoir combien de processus tournent (
--workersest décidé dehors, et chaque worker se croit seul), donc pas de refus — un avertissement unique au démarrage, qui nomme les jobs concernés.En revanche, deux replis silencieux deviennent des erreurs, parce qu'ils trompaient l'opérateur :
backend: redissans les paquets redis → il croyait ses jobs protégés et n'avait rien ;databaseest documenté dans le commentaire du champ mais pas implémenté) → il recevait « mémoire » sans un mot.Les deux marquent le service
DEGRADED. Le vrai verrou sans Redis (un bail en base) reste à faire : c'était l'option 1 de l'issue et c'est une décision d'architecture, pas un correctif.Trouvé en route : le scan de sécurité échouait en mode OUVERT
ASTScanner.scana deux phases d'imports — l'extension compiléescanner_corequand elle est là, l'AST Python sinon. Seule la seconde signalait une erreur de syntaxe. Avec le scanner compilé — la configuration optimisée, celle qui est livrée — un plugin dont une source ne se parse pas revenait doncpassed=True, alors qu'aucun de ses imports, builtins ou attributs interdits n'avait pu être examiné. Ne pas savoir lire un fichier n'est pas l'avoir trouvé sain._check_builtins_and_attrsparse de toute façon dans les deux phases : c'est là que le défaut de parsing est signalé, une fois et une seule. Les deux chemins rendent maintenant le même verdict, ce qu'un test vérifie explicitement.C'est ce qui faisait échouer
test_scan_syntax_errordanstest_ast_scanner.pyettest_validation.py— avant comme après ce travail : vérifié en remisant mes modifications et en relançant sur l'arbre d'origine (2 échecs). Ils passent maintenant.Vérification
Un seul test a dû être adapté :
test_yaml_import_errorsimulait l'absence de PyYAML en neutralisantsys.modules["yaml"]; il neutralise aussixcore.kernel.yamlio, puisque c'est par là que passe désormais la lecture. Le contrat testé est identique (ManifestError« pyyaml non installé »).🤖 Generated with Claude Code
Summary by Sourcery
Reduce startup and YAML parsing overhead, bound tracing memory usage, and make scheduler and security failures explicit and safe.
Bug Fixes:
Enhancements:
Documentation:
Tests: