Skip to content

fix(intelligence): unify rule-based importance scoring - #1142

Open
knqiufan wants to merge 2 commits into
oceanbase:mainfrom
knqiufan:fix/importance-weighted-rule-eval-1141
Open

fix(intelligence): unify rule-based importance scoring#1142
knqiufan wants to merge 2 commits into
oceanbase:mainfrom
knqiufan:fix/importance-weighted-rule-eval-1141

Conversation

@knqiufan

@knqiufan knqiufan commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Test plan

  • python -m pytest tests/unit/intelligence/test_importance_evaluator.py
  • python -m pytest tests/unit/intelligence
  • git diff --check

@wayyoungboy wayyoungboy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because the current rule-based scoring path drops existing metadata/context importance signals.

The new rule-based path delegates to dimension scoring, but that scoring path does not appear to consume the existing metadata.priority, metadata.tags, or context.user_engagement signals. A direct probe on the current head with neutral content plus high-priority/profile/high-engagement metadata/context returns 0.0; the previous rule fallback assigned a positive score from those signals. The new tests patch the dimension scorer, so they do not exercise the real evaluator behavior.

Please carry the metadata/context boosts into the real dimension scoring path, or keep the old signal path covered by a test that calls the real evaluator instead of patching it.

@knqiufan

knqiufan commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

@wayyoungboy Thanks for the review! Fixed in 0d0da9a.

The rule-based path now carries the existing metadata/context signals into the real dimension scoring path:

  • metadata.priority and context.user_engagement contribute through relevance.
  • metadata.tags contributes through novelty.
  • Added a regression test that calls the real evaluate_importance path with neutral content plus high-priority/profile/high-engagement metadata/context and verifies the legacy positive score is preserved without patching the scorer.

Verified:

  • python -m pytest tests/unit/intelligence/test_importance_evaluator.py
  • python -m pytest tests/unit/intelligence
  • git diff --check

Copy link
Copy Markdown
Member

Thanks for the follow-up. This is still not ready to merge while the previous CHANGES_REQUESTED review is active.

Please request re-review after confirming that the rule-based path carries the existing metadata/context importance signals into the six-dimension scoring path, with regression coverage for those signals. Once that review is cleared, this can be reconsidered for merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement]: Unify rule-based importance evaluation with six-dimension weighted scoring

2 participants