Decide sooner that a region is too slow to scan

DESIGN.md

@@ -333,8 +333,10 @@ to scan when, over a trailing window of chunks, either:
 - three quarters of the last 64 MiB were over budget, or
 - scanning them ran below the `--min-rate` floor, the same floor that calls a
   whole drive too slow to test. That is measured on I/O time, with every
-  drill-down, retry and cache-busting seek counted, once there are at least 8
-  chunks and a minute of I/O to judge.
+  drill-down, retry and cache-busting seek counted, once there are at least 3
+  chunks and 20 s of I/O to judge. A healthy drive never spends 20 s on three
+  chunks; that is a twentieth of the floor. Looser than this, a drive covering
+  34 KiB/s spent four minutes before anything could be decided.
 
 Then it stops reading. It probes 64 MiB 1 GiB further on, and again, until a
 probe comes back fast, then bisects between the last slow probe and that one
@@ -354,9 +356,11 @@ as over. The steps are fixed rather than doubling, so a healthy island more
 than a gigabyte wide is still found.
 
 And whatever the window says, one chunk may not eat the scan. A drill-down
-that has spent 30 s of I/O on one chunk stops, and the whole chunk is
-condemned: the sectors drilled so far stay in the findings, and the rest is
-recorded as unread. A run of such chunks becomes one region.
+that has spent 10 s of I/O on one chunk, about three times what drilling a
+healthy 1 MiB chunk costs, stops, and the whole chunk is condemned. The
+sectors drilled so far stay in the findings, and the rest is recorded as
+unread. A run of such chunks becomes one region, and so does a cut chunk
+followed by a skipped region.
 
 I/O time rather than wall-clock time is what gets judged, because it counts
 only what the drive did. It is also what `HDDSCAN_SLOW` can make exact for the

docs/guide.html

@@ -507,7 +507,7 @@ <h3>Test mode</h3>
 <dt id="opt-second-pass"><code>--second-pass</code></dt>
 <dd>after <a href="#opt-mode"><code>--mode</code></a> write, re-read the whole device and re-verify the pattern (catches slow data decay)</dd>
 <dt id="opt-skip-slow"><code>--skip-slow</code></dt>
-<dd>when a region is too slow to scan -- most chunks over the last 64 MiB missed their budget, or the scan there, drill-downs and retries included, ran under the <a href="#opt-min-rate"><code>--min-rate</code></a> floor -- stop reading: probe 1 GiB on, and on, until a probe is fast, bisect back to the edge, carry on there.  A chunk whose drill-down costs over 30 s is condemned whole. What was jumped over is condemned unread; the report says so and <a href="#opt-hide-bad"><code>--hide-bad</code></a> cuts it out. Sequential or reverse order only.  On under repair</dd>
+<dd>when a region is too slow to scan -- most chunks over the last 64 MiB missed their budget, or the scan there, drill-downs and retries included, ran under the <a href="#opt-min-rate"><code>--min-rate</code></a> floor -- stop reading: probe 1 GiB on, and on, until a probe is fast, bisect back to the edge, carry on there.  A chunk whose drill-down costs over 10 s is condemned whole. What was jumped over is condemned unread; the report says so and <a href="#opt-hide-bad"><code>--hide-bad</code></a> cuts it out. Sequential or reverse order only.  On under repair</dd>
 <dt id="opt-no-skip-slow"><code>--no-skip-slow</code></dt>
 <dd>read every region, however slow, under repair too</dd>
 <dt id="opt-rewrite-weak"><code>--rewrite-weak</code></dt>

hddscan.8

@@ -382,13 +382,13 @@ apiece. A region counts as too slow once three quarters of the chunks over
 the last 64 MiB have missed their budget, or once scanning it \(em every
 drill-down, retry and seek included \(em has run below the
 .B \-\-min\-rate
-floor for a minute of I/O. Then stop reading: probe 64 MiB 1 GiB further on,
+floor for 20 s of I/O over at least three chunks. Then stop reading: probe 64 MiB 1 GiB further on,
 and again, until a probe comes back fast, then bisect between the last slow
 probe and that one down to 16 MiB, with 8 MiB probes once the gap is narrower
 than a wide one, and carry on scanning there. Every
 probe is judged against the same calibrated budget as the scan; what drilling
 a probe's over-budget chunks would cost is taken from what drilling has cost
-on this drive so far. Separately, a chunk whose drill-down has spent 30 s of
+on this drive so far. Separately, a chunk whose drill-down has spent 10 s of
 I/O stops being drilled and is condemned whole. What was jumped over is
 .B condemned without being read:
 the report says so on its Coverage line and under the verdict, draws it as

hddscan.c

@@ -5375,7 +5375,7 @@ static void check_temperature(ctx_t *c)
  * budget, and it decides nothing about what is over.
  *
  * And whatever the window says, one chunk may not eat the scan: a drill-down
- * that has spent 30 s of I/O on one chunk stops, and the whole chunk is
+ * that has spent 10 s of I/O on one chunk stops, and the whole chunk is
  * condemned.  No window can help while a single chunk takes five minutes.
  *
  * Only for an order that walks the drive: sequential or reverse, unsampled.
@@ -5392,9 +5392,17 @@ static void check_temperature(ctx_t *c)
 #define SKIP_WIDE_BYTES (64ull << 20)    /* to find damage: 4% of it, 94% */
 #define SKIP_PROBE_MIN 4
 #define SKIP_PROBE_MAX 512
-#define SKIP_RATE_MIN_CHUNKS 8          /* the rate rule, once it has these */
-#define SKIP_RATE_MIN_US (60ull * 1000000)      /* and a minute of I/O */
-#define SKIP_DRILL_CAP_US (30ull * 1000000)     /* one chunk's drill-down */
+/*
+ * How soon.  Drilling a healthy 1 MiB chunk costs 256 single reads, about
+ * 3.5 s on a drive with a 13.6 ms sector median, so a drill-down past 10 s
+ * of I/O is a chunk the drive is fighting, and three chunks costing 20 s
+ * between them is a scan running at a twentieth of the floor.  Looser than
+ * this, a drive covering 34 KiB/s spent four minutes before anything could
+ * be decided -- and a probe that finds nothing costs a second or two.
+ */
+#define SKIP_RATE_MIN_CHUNKS 3          /* the rate rule, once it has these */
+#define SKIP_RATE_MIN_US (20ull * 1000000)      /* and this much I/O */
+#define SKIP_DRILL_CAP_US (10ull * 1000000)     /* one chunk's drill-down */
 #define MAX_SKIPS 4096
 
 /* the bytes [a, b) covers in the scan order, which only ever walks */
@@ -5537,7 +5545,7 @@ static int skip_cut(ctx_t *c, uint64_t off, size_t len, uint64_t from)
 /*
  * One chunk's verdict into the trailing window; 1 when the region it covers
  * is slow: the window is full and three quarters of it was over budget, or
- * scanning it has run below the throughput floor for a minute of I/O.
+ * scanning it has run below the throughput floor for 20 s of I/O.
  */
 static int skip_note(ctx_t *c, uint64_t step, int over)
 {
@@ -5698,18 +5706,33 @@ static uint64_t skip_ahead(ctx_t *c, void *buf)
 			s0 = c->sw_step[i];
 	if (c->nskip && s0 < c->skips[c->nskip - 1].s1)
 		s0 = c->skips[c->nskip - 1].s1;
-	k = skip_add(c);
-	if (!k)
-		return t + 1;
-	k->s0 = s0;
-	k->k0 = t + 1;
-	k->s1 = s1;
-	steps_extent(c, s0, s1, &k->off, &k->len);
 	steps_extent(c, t + 1, s1, &off, &unread);
-	k->unread = unread;
+	/*
+	 * Right after a stretch already condemned -- a chunk whose drill-down
+	 * was cut short, most often, which is what set this off -- it is the
+	 * same region, and is written down as one.
+	 */
+	if (c->nskip && s0 == c->skips[c->nskip - 1].s1) {
+		uint64_t was;
+
+		k = &c->skips[c->nskip - 1];
+		was = k->len;
+		k->s1 = s1;
+		steps_extent(c, k->s0, s1, &k->off, &k->len);
+		c->skip_cond += k->len - was;
+	} else {
+		k = skip_add(c);
+		if (!k)
+			return t + 1;
+		k->s0 = s0;
+		k->k0 = t + 1;
+		k->s1 = s1;
+		steps_extent(c, s0, s1, &k->off, &k->len);
+		c->skip_cond += k->len;
+	}
+	k->unread += unread;
 	skip_band_add(c, off, unread);
 	c->skip_bytes += unread;
-	c->skip_cond += k->len;
 	memset(c->sw_over, 0, sizeof(c->sw_over));
 	msg(PROG ": %s: %s%s from %s was too slow to scan; skipped %s "
 	    "of it without reading%s (--skip-slow) and carried on %s\n",
@@ -7985,7 +8008,7 @@ static void usage(void)
 "                         under the --min-rate floor -- stop reading: probe\n"
 "                         1 GiB on, and on, until a probe is fast, bisect\n"
 "                         back to the edge, carry on there.  A chunk whose\n"
-"                         drill-down costs over 30 s is condemned whole.\n"
+"                         drill-down costs over 10 s is condemned whole.\n"
 "                         What was jumped over is condemned unread; the\n"
 "                         report says so and --hide-bad cuts it out.\n"
 "                         Sequential or reverse order only.  On under repair\n"

tests/cli.sh

@@ -646,13 +646,13 @@ assert_hasnt "without --skip-slow the rate rule does nothing" \
 
 # And a chunk so full of weak sectors that drilling it alone would take
 # minutes: every 4 KiB of it reads slow enough to be weak, and retried.
-# The drill-down stops once it has cost 30 s, and the chunk is condemned.
+# The drill-down stops once it has cost 10 s, and the chunk is condemned.
 f=$(slowimg skipcut.bin 64M)
 out=$(HDDSCAN_SLOW=8M:1M HDDSCAN_SLOW_MS=512000 run --chunk 1M \
 	--chunk-slow-ms 1000 --retries 2 --skip-slow \
 	--json "$TMP/skipcut.json" "$f")
 assert_has "a drill-down that costs too much is cut short" "$out" \
-	"drilling the chunk at 8.00 MiB has cost over 30 s"
+	"drilling the chunk at 8.00 MiB has cost over 10 s"
 read -r roff rend <<< "$(json_region "$TMP/skipcut.json")"
 assert_eq "and exactly that chunk is condemned" "$roff $rend" \
 	"8388608 9437184"