feat(search): index the 16 package front pages, and pin the page-focus ranker

TWO CHANGES, both from building a real app against the MCP server and recording where
search failed.

1. THE PACKAGE FRONT PAGES WERE INVISIBLE. search-corpus.js excluded everything under
   /api/, correctly for the 1,152 symbol pages — the feed already carries each symbol's
   name, kind, package and summary, and deriving that twice once produced 52% junk. But
   the rule matches on URL, and /api/<pkg>/latest/ is a different kind of page: an
   overview naming the entry points, a Remarks section stating the behaviours no
   signature shows, and worked examples.

   Those 16 URLs got ONE record whose entire searchable text was a title and a
   one-liner — 98 characters for @imqueue/http-protect, against 123 for a single one of
   its properties. So "protect an HTTP gateway from too many requests per IP" returned
   four HttpProtect PROPERTY pages, each with a summary mentioning requests and IPs,
   while the page holding `app.use(new HttpProtect().jsonMiddleware())` never appeared.
   A property page outscored the front page of the package because it had more indexed
   text than the front page did. The `hardening` topic of the question KPI scored 0% on
   BOTH rankers for the same reason: what answers it was not in the corpus.

   Sections only, not a page record — the feed already contributes a `package` record
   for each of those exact URLs, and a second record would leave the ranker's URL dedupe
   choosing between two spellings of one page.

   Cost: 719 -> 836 sections, search-text.json 206.9 -> 241.4 KB gz, on a feed fetched in
   the background after the first query rather than on load. check:search-index now
   validates 700 section ranges against the real mirrors, up from 599, and those pages
   are dense with code fences.

2. THE RANKER NOW JUDGES A SECTION BY ITS PAGE (submodule -> 3a43ea0). Natural micro
   94.1 -> 94.2 and macro 88.9 -> 89.0 with 19 better against 7 worse; question accuracy
   64.0 -> 64.5, macro 60.9 -> 61.5, #1-exactly 47.8 -> 48.7; artificial 91.1 -> 90.8.
   See that commit for the mechanism and for the two experiments rejected on the way.

Honest note on (1): measured alone it moved no aggregate — natural, artificial and
question all within 0.2 either way. It is kept because the content genuinely belongs in
the index and individual answers do improve (/api/validation/latest/#remarks went from
absent to #2 for "how do I reject bad input before it reaches my method?"), and because
nothing can rank a page whose text was never indexed. The aggregate needs the weights,
which is what (2) starts on.

Author: Mikhus

scripts/lib/search-corpus.js

@@ -559,7 +559,30 @@ function buildCorpus(outputDir) {
       // BELOW it is generated reference and comes from /api/search-index.json.
       // Excluding the whole prefix left the /api/ landing page — a sitemap URL —
       // unsearchable.
-      if (!mirror || /^\/api\/.+/.test(mirror.url)) {
+      //
+      // WITH ONE EXCEPTION, and it was measured: the 16 PACKAGE INDEX pages,
+      // /api/<pkg>/latest/. They are not symbol pages. Each one carries an
+      // overview naming the entry points, a Remarks section stating the
+      // behaviours no signature shows, and worked code examples — the best page
+      // in the reference for anyone who does not already know what to call.
+      //
+      // All of that was invisible to search. The feed gives these URLs ONE record
+      // whose entire searchable text is a title and a one-line summary, so
+      // "protect an HTTP gateway from too many requests per IP" returned four
+      // HttpProtect *property* pages — each with its own summary mentioning
+      // requests and IPs — while the page holding `app.use(new
+      // HttpProtect().jsonMiddleware())` never appeared at all. Measured over a
+      // scratch build against 14 packages, this was the single largest source of
+      // unanswerable questions: the `hardening` topic of the question KPI scored
+      // 0% on BOTH rankers, because what answers it was not in the corpus.
+      //
+      // Sections only, NOT a page record: the feed already contributes a `package`
+      // record for this exact URL with the right title, kind and package name.
+      // Pushing a second record for the same URL would put two spellings of one
+      // page in the index and leave the ranker's URL dedupe to pick between them.
+      const isPackageIndex = mirror && /^\/api\/[^/]+\/latest\/$/.test(mirror.url);
+
+      if (!mirror || (/^\/api\/.+/.test(mirror.url) && !isPackageIndex)) {
         continue;
       }
 
@@ -588,8 +611,14 @@ function buildCorpus(outputDir) {
         record.w = meta.k;
       }
 
-      docs.push(record);
+      if (!isPackageIndex) {
+        docs.push(record);
+      }
 
+      // faqRecords is driven by heading SHAPE, so it contributes nothing from a
+      // package index today (its headings are Remarks, Example 1, Classes). Left
+      // running rather than skipped: if one of those pages ever grows a
+      // question-shaped heading, that heading is a good answer.
       faq.push(...faqRecords(parts, mirror));
 
       const pageIdx = pages.length;