Skip to content

Commit 568bd0d

Browse files
committed
test(search): guard the one question with revenue attached
imqueue.com reaches the ranker only through the peer feeds, so with x1/x2 unloaded "pricing commercial license" answered from imqueue.org's own /license/ — a plausible page, the wrong edition, and nothing anywhere reporting a problem. The MCP server's docs.ts went out of its way to fix that ("the commercial question was the single thing this server could not answer") and nothing here asserted it stayed fixed. This is the last open item of the plan's Phase 0.3. Three cases, each answerable only by the commercial edition, asserted as "an imqueue.com result is in the visible set" rather than "it is first": which commercial page wins is a ranking judgement that may change, that the answer comes from the edition documenting licensing at all is not. Plus the reverse — a framework query must still lead with imqueue.org — so the peer feed cannot swamp the docs it sits beside. pricing -> imqueue.com at #2 commercial license -> #8 pricing commercial license -> #5 the phrasing the plan recorded as broken "support" was tried and dropped: imqueue.org has its own page titled Support, which correctly takes #1, so the query is not commercial-only and asserting otherwise asserted a false premise. Better to delete the case than to weaken what the others mean. Skipped rather than failed when the peer feed is absent, since `npm run build:org` alone does not produce one. Verified the guard fails when it should by swapping in an org-only topic: "graceful shutdown sigterm" reports no imqueue.com result and exits non-zero.
1 parent c2a45a6 commit 568bd0d

1 file changed

Lines changed: 57 additions & 0 deletions

File tree

‎scripts/check-search-ranking.js‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -682,6 +682,63 @@ if (!df || !ranker.state.t2.docs) {
682682
}
683683
}
684684

685+
// ---- the commercial half answers from the commercial edition -------------------
686+
// The one failure mode with revenue attached, and the one that reports nothing when it
687+
// happens. imqueue.com reaches the ranker ONLY through the peer feeds, so with x1/x2
688+
// unloaded "pricing commercial license" answers from imqueue.org's own /license/: a
689+
// plausible page, the wrong edition, no error anywhere. The @imqueue MCP server's docs.ts
690+
// went out of its way to fix exactly this — "the commercial question was the single thing
691+
// this server could not answer" — and nothing here asserted it stayed fixed.
692+
//
693+
// Skipped, not failed, when the peer feed is absent: `npm run build:org` alone does not
694+
// produce one, and that is a supported local state.
695+
{
696+
const peerIndex = path.join(OUT, 'search-peer-index.json');
697+
const peerText = path.join(OUT, 'search-peer-text.json');
698+
699+
if (!fs.existsSync(peerIndex) || !fs.existsSync(peerText)) {
700+
console.log(' skip peer feed absent (build:all builds it) — commercial cases not checked');
701+
} else {
702+
ranker.state.x1 = ranker.prepare(JSON.parse(fs.readFileSync(peerIndex, 'utf8')));
703+
ranker.state.x2 = ranker.prepareSections(JSON.parse(fs.readFileSync(peerText, 'utf8')));
704+
705+
// Deliberately asserted as "an imqueue.com result is in the set", not "it is first".
706+
// Which commercial page wins is a ranking judgement that may change; that the answer
707+
// comes from the edition which documents licensing at all is not.
708+
// Chosen for being answerable ONLY by the commercial edition. "support" was tried and
709+
// dropped: imqueue.org has its own page titled Support, which takes #1 correctly, so the
710+
// query is not commercial-only and asserting otherwise asserted a false premise. The
711+
// third case is the exact phrasing the plan names as the failure it saw.
712+
const COMMERCIAL = [
713+
['pricing', 'a price is only ever quoted on the commercial edition'],
714+
['commercial license', 'the org edition documents the GPL, not the commercial terms'],
715+
['pricing commercial license', 'the phrasing that answered from org /license/ with no error'],
716+
];
717+
718+
for (const [query, why] of COMMERCIAL) {
719+
const hits = ranker.search(ranker.parseQuery(query));
720+
const peer = hits.findIndex((hit) => hit.external);
721+
722+
if (peer === -1) {
723+
fail(`"${query}" returns no imqueue.com result at all — ${why}`);
724+
} else if (peer >= 10) {
725+
fail(`"${query}" puts its first imqueue.com result at ${peer + 1}, past the visible set`);
726+
} else {
727+
pass(`"${query}" reaches imqueue.com at #${peer + 1} — ${why}`);
728+
}
729+
}
730+
731+
// And the reverse, because the peer feed must not swamp the framework docs either.
732+
const local = ranker.search(ranker.parseQuery('redis queue')).findIndex((hit) => !hit.external);
733+
734+
if (local !== 0) {
735+
fail(`"redis queue" no longer leads with an imqueue.org result (index ${local})`);
736+
} else {
737+
pass('a framework query still leads with imqueue.org, so the peer feed does not swamp it');
738+
}
739+
}
740+
}
741+
685742
if (failures) {
686743
console.error(`\n${failures} search ranking check(s) failed.`);
687744
process.exit(1);

0 commit comments

Comments
 (0)