Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,10 @@ cmake --build build --clean-first -j # and the same for asan/ if that t
`test/g1freshcheck.sh` catches the ordinary stale binary (binary older than source) and is worth
believing when it fires — it is not noise. It cannot catch this variant, because here the binary is
*newer* than the source and only its contents are stale. Nothing in CMake can repair a source that
changed mid-compile; the discipline is the fix.
changed mid-compile; the discipline is the fix. When this variant is suspected, `--doctor`'s `layout`
row reports the cross-translation-unit `sizeof`/`alignof` evidence; treat `state="disagree"` as a
clean-rebuild requirement. `state="agree"` compares only the `types=` registered in `src/model.h`; a
same-size layout change or a stale constant is invisible, so `agree` does not rule out a mixed binary.

## Verify

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1797,9 +1797,9 @@ wrong, and it has. These are the results that say so, all in-tree, all published
### In the tests

<details>
<summary><b>614 gate scripts</b>, five contracts no unit test can hold, and the house rule: write the gate before the code it measures</summary> <!-- gatecount -->
<summary><b>615 gate scripts</b>, five contracts no unit test can hold, and the house rule: write the gate before the code it measures</summary> <!-- gatecount -->

`test/regression.sh` names **614 gate scripts** and is the authoritative list; <!-- gatecount -->
`test/regression.sh` names **615 gate scripts** and is the authoritative list; <!-- gatecount -->
`python3 test/pargates.py . ./build/ripwire -j 6` runs the same set in parallel. On top of them sit the
contracts that do not fit a unit test: two runs byte-identical, warm output identical to cold, output
that pipes clean through `xmllint --noout`, a sanitizer build with `-fno-sanitize-recover=all`, and a
Expand Down
2 changes: 2 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ sections in call order: the lazy tags.scm prewarm (`ingest_prewarm.h`), the para
(`ingest_parsepool.h`), the document post-pass (`ingest_docpass.h`), and the build-model tail —
dedup, symbol assignment, span attribution, ordered emit (`ingest_model.h`).

`--doctor`'s `layout` row compares the shared model's recorded `sizeof`/`alignof` facts across translation units and reports evidence of a possible mixed binary instead of guessing which object is stale; agreement covers only the registered facts.

**Crawl order is deterministic, and that is load-bearing.** The walk *collects every candidate path
first*, sorts them lexicographically by byte, and only then assigns node IDs and parses. Node IDs are
indices into that sorted list, so they are stable across runs of the same tree; if IDs followed
Expand Down
6 changes: 3 additions & 3 deletions docs/EVALS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ section, and it is not an afterthought.
| **Co-change / known-item evals** | `--eval`, `--eval-retrieval` (see `bench/ANSWERQUALITY.md`) | Whether the tool surfaces the other files a real historical commit touched; and known-item retrieval across four rankers. |
| **Ensemble calibration harness** | `bench/ensemblecal/` | Whether `--ensemble`'s four evidence families are actually orthogonal, how often each fires, how stable each is across commits — and the preset ladder derived from that (§9). |
| **Differential argv harness** | `test/argvdiffcheck.sh` | That a refactor changed *nothing observable*: two binaries, every argv vector, stdout + stderr + exit code byte-identical. |
| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 614 gate scripts plus the determinism, cache-transparency and golden contracts. <!-- gatecount --> |
| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 615 gate scripts plus the determinism, cache-transparency and golden contracts. <!-- gatecount --> |
| **`--quality-delta`** | `src/quality.h` | Ten measured code-quality failure modes, reported only where a change made them worse. |

### The labeling protocol (why the held-out eval is allowed to disagree with the ranker)
Expand Down Expand Up @@ -5834,7 +5834,7 @@ copy here would be exactly the dialect divergence that gate exists to catch. Com
tags, wrap, stable-order defaults), seven individually invoked standalone gates (`g1freshcheck`,
`skillscan`, `htmlexport`, `compresscheck`, `handoffcheck`, `releaseinstallcheck`,
`taskroutecheck`), and a single loop
naming **614 gate scripts**, all of which exist on disk. <!-- gatecount -->
naming **615 gate scripts**, all of which exist on disk. <!-- gatecount -->

`python3 test/pargates.py . ./build/ripwire -j 6` runs the same scripts in parallel so a full
verification fits in one sitting. It does not modify `regression.sh`.
Expand Down Expand Up @@ -6846,7 +6846,7 @@ Listed because the reason is more useful than the silence.
shipped**. See `bench/locbench/anchorhop_calib.json`. The mention anchor's reproducible numbers are
the ablations in §4.
- **A single round gate-count.** Two in-tree numbers disagree (`test/pargates.py`'s docstring says
~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 614. The <!-- gatecount -->
~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 615. The <!-- gatecount -->
loop is the authority; the stale docstrings are a known drift. Since 2026-09-10 the number is not
written by hand anywhere: `docs/gatecount_build.py` derives it from the loop and rewrites every
published site, `test/gatecountcheck.sh` fails if any of them drifts, and `test/manifestcheck.sh`
Expand Down
6 changes: 3 additions & 3 deletions present/deck5_ripwire_build.js
Original file line number Diff line number Diff line change
Expand Up @@ -1068,7 +1068,7 @@ function storyCards(s, { kick, head, stories, footText }){
kicker(s, "// how it stays true", AMBER);
title(s, "Proven, not promised");
const cards = [
["614 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount
["615 gate scripts", "the suite runs on every push — plus determinism, cache-transparency and golden contracts; the gate count itself is gated against the runner's own loop"], // gatecount
["byte-identical, always", "two runs over the same tree produce the same bytes; warm equals cold. Enforced in CI, twice — Release AND a plain flavour, because NDEBUG once blinded a whole class of checks"],
["differential refactoring", "a refactor must prove it changed nothing observable: two binaries, hundreds of argv vectors, stdout + stderr + exit codes byte-identical"],
["held-out labels, authored blind", "eval labels were written by reading source before the ranker ever ran on them — so the eval is allowed to say the ranker is wrong. It has."],
Expand All @@ -1092,7 +1092,7 @@ function storyCards(s, { kick, head, stories, footText }){
title(s, "Claims you can trust, because we publish what failed", { size: 32 });

card(s, MX, 1.72, 3.86, 1.72);
stat(s, "614", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount
stat(s, "615", "gate scripts named by test/regression.sh — and the COUNT itself is gated against the runner's own loop, so it cannot go stale quietly", // gatecount
MX+0.15, 1.86, 3.56, CYAN, { bsize: 42, bh: 0.66, lsize: 9.5 });
card(s, 4.68, 1.72, 3.86, 1.72, CARD2);
stat(s, "8", "registered NEGATIVES — changes built, gated green, measured against a band written before the code, and reverted rather than tuned",
Expand Down Expand Up @@ -1342,7 +1342,7 @@ function storyCards(s, { kick, head, stories, footText }){
["179 long flags · 33 slides", "bash test/deckclaimcheck.sh"],
["every --flag named here exists", "bash test/deckcheck.sh"],
["74.7% fewer element bytes", "bash test/showcasecapturecheck.sh"],
["614 gate scripts", "bash test/manifestcheck.sh"], // gatecount
["615 gate scripts", "bash test/manifestcheck.sh"], // gatecount
["49 repos · 70 papers · 237 surveyed","bash test/readmedriftcheck.sh"],
["the ten moments, any row", "ripwire . --callers=SYM | wc -c"],
["the head-to-head table", "bench/headtohead/r4-2026-08-06/"],
Expand Down
17 changes: 17 additions & 0 deletions src/model.h
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

#include "infra/profileScope.h"
#include "smallvec.h" // rw::SmallVec — THE ONE ALIAS; the per-key span lists and per-file id buckets below
#include "structlayout.h" // cross-translation-unit sizeof/alignof tripwire for the shared model

#include <algorithm> // std::sort — symbolsByFile below
#include <tuple> // std::tie — lessUnindexedExt's mixed-direction compare
Expand Down Expand Up @@ -1016,6 +1017,22 @@ struct IngestResult
std::size_t reparsedFiles = 0;
};

// These aggregates cross the ingest/main translation-unit boundary. The record is emitted once per TU by
// structlayout.h, so a header change that leaves a mixed binary can be diagnosed after linking instead of
// passing a self-consistent static_assert in each half.
RIPWIRE_LAYOUT_REGISTER_TYPES( RIPWIRE_LAYOUT_TYPE_ENTRY( std::string ),
RIPWIRE_LAYOUT_TYPE_ENTRY( Symbol ),
RIPWIRE_LAYOUT_TYPE_ENTRY( Reference ),
RIPWIRE_LAYOUT_TYPE_ENTRY( Include ),
RIPWIRE_LAYOUT_TYPE_ENTRY( ConstOpen ),
RIPWIRE_LAYOUT_TYPE_ENTRY( Binding ),
RIPWIRE_LAYOUT_TYPE_ENTRY( BindingAlias ),
RIPWIRE_LAYOUT_TYPE_ENTRY( RouteDef ),
RIPWIRE_LAYOUT_TYPE_ENTRY( RouteUse ),
RIPWIRE_LAYOUT_TYPE_ENTRY( FileHealth ),
RIPWIRE_LAYOUT_TYPE_ENTRY( SkippedOversize ),
RIPWIRE_LAYOUT_TYPE_ENTRY( IngestResult ) );

// multi-root workspace cap: a sane bound on N crawl roots — an agent joining a
// handful of checkouts is the use case; hundreds of roots is a mis-glued path list, refused loudly.
inline constexpr std::size_t kMaxWorkspaceRoots = 16;
Expand Down
253 changes: 253 additions & 0 deletions src/structlayout.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,253 @@
// SPDX-License-Identifier: Apache-2.0
// Copyright 2026 David Brewster

//
// structlayout.h
//
// Cross-translation-unit layout tripwire. Every translation unit that includes
// this header registers the sizeof/alignof facts it compiled with. The registry
// is intentionally separate from src/layout.h: that file models source text for
// repositories ripwire analyzes, while this file records this binary's ABI view.
//
// The records have internal linkage. Only the registry and its registrar are
// shared, so an inline record cannot make the linker discard one TU's evidence.
//
#pragma once

#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <cstring>
#include <vector>

#if defined( __GNUC__ ) || defined( __clang__ )
#define RIPWIRE_LAYOUT_USED __attribute__( ( used ) )
#else
#define RIPWIRE_LAYOUT_USED
#endif

// __FILE__ expands to this header at the registration site. GCC and Clang's
// __BASE_FILE__ is the primary source file passed to the compiler, which is the
// identity needed here. A build system can override it for a compiler without
// __BASE_FILE__ (and for fixture tests that compile one source twice).
#if !defined( RIPWIRE_LAYOUT_TU )
#if defined( __BASE_FILE__ )
#define RIPWIRE_LAYOUT_TU __BASE_FILE__
#else
#error "RIPWIRE_LAYOUT_TU must be supplied when __BASE_FILE__ is unavailable"
#endif
#endif

namespace rw
{
namespace layout_registry
{

struct TypeLayout
{
const char* name = nullptr;
std::size_t size = 0;
std::size_t align = 0;
};

struct LayoutRecord
{
const char* unit = nullptr;
const TypeLayout* types = nullptr;
std::size_t typeCount = 0;
};

class Registry
{
public:
void add( const LayoutRecord* record )
{
records_.push_back( record );
}

const std::vector<const LayoutRecord*>& records() const noexcept
{
return records_;
}

private:
std::vector<const LayoutRecord*> records_;
};

// Function-local construction keeps registration independent of the order in
// which static initializers from different translation units run.
inline Registry& registry()
{
static Registry instance;
return instance;
}

class Registrar
{
public:
explicit Registrar( const LayoutRecord& record )
{
registry().add( &record );
}
};

enum class CheckState : std::uint8_t
{
NoRecords,
NotChecked,
Agree,
Disagree,
};

struct LayoutMismatch
{
const char* typeName = nullptr;
const char* unit0 = nullptr;
const char* unit1 = nullptr;
std::size_t size0 = 0;
std::size_t size1 = 0;
std::size_t align0 = 0;
std::size_t align1 = 0;
bool present0 = false;
bool present1 = false;
};

struct LayoutCheck
{
CheckState state = CheckState::NoRecords;
std::size_t unitCount = 0;
std::size_t typeCount = 0;
LayoutMismatch mismatch;
};

inline const TypeLayout* findType( const LayoutRecord& record, const char* name ) noexcept
{
for( std::size_t i = 0; i < record.typeCount; ++i )
{
if( std::strcmp( record.types[ i ].name, name ) == 0 )
{
return &record.types[ i ];
}
}
return nullptr;
}

inline std::vector<const LayoutRecord*> sortedRecords()
{
std::vector<const LayoutRecord*> records = registry().records();
std::sort( records.begin(), records.end(), []( const LayoutRecord* left, const LayoutRecord* right )
{
return std::strcmp( left->unit, right->unit ) < 0;
} );
return records;
}

inline LayoutCheck disagreement( const LayoutRecord& first, const TypeLayout* firstType,
const LayoutRecord& second, const TypeLayout* secondType,
const char* typeName ) noexcept
{
LayoutCheck result;
result.state = CheckState::Disagree;
result.unitCount = 2;
result.typeCount = first.typeCount;
result.mismatch = { typeName,
first.unit,
second.unit,
firstType != nullptr ? firstType->size : 0,
secondType != nullptr ? secondType->size : 0,
firstType != nullptr ? firstType->align : 0,
secondType != nullptr ? secondType->align : 0,
firstType != nullptr,
secondType != nullptr };
return result;
}

inline LayoutCheck compare()
{
const std::vector<const LayoutRecord*> records = sortedRecords();
LayoutCheck result;
result.unitCount = records.size();

if( records.empty() )
{
return result;
}
result.typeCount = records.front()->typeCount;
if( records.size() == 1 )
{
result.state = CheckState::NotChecked;
return result;
}

bool hasType = false;
for( const LayoutRecord* record : records )
{
hasType = hasType || record->typeCount != 0;
}
if( !hasType )
{
result.state = CheckState::NotChecked;
return result;
}

const LayoutRecord& first = *records.front();
for( std::size_t typeIndex = 0; typeIndex < first.typeCount; ++typeIndex )
{
const TypeLayout& expected = first.types[ typeIndex ];
for( std::size_t unitIndex = 1; unitIndex < records.size(); ++unitIndex )
{
const LayoutRecord& actualRecord = *records[ unitIndex ];
const TypeLayout* actual = findType( actualRecord, expected.name );
if( actual == nullptr || actual->size != expected.size || actual->align != expected.align )
{
LayoutCheck mismatch = disagreement( first, &expected, actualRecord, actual, expected.name );
mismatch.unitCount = records.size();
return mismatch;
}
}
}

// A record with an extra type is also a disagreement. The fixed model list
// should never take this path, but it keeps the registry honest for tests
// and for future additions to the recorded aggregate set.
for( std::size_t unitIndex = 1; unitIndex < records.size(); ++unitIndex )
{
const LayoutRecord& actualRecord = *records[ unitIndex ];
for( std::size_t typeIndex = 0; typeIndex < actualRecord.typeCount; ++typeIndex )
{
const TypeLayout& actual = actualRecord.types[ typeIndex ];
if( findType( first, actual.name ) == nullptr )
{
LayoutCheck mismatch = disagreement( first, nullptr, actualRecord, &actual, actual.name );
mismatch.unitCount = records.size();
return mismatch;
}
}
}

result.state = CheckState::Agree;
return result;
}

} // namespace layout_registry
} // namespace rw

#define RIPWIRE_LAYOUT_TYPE_ENTRY( Type ) { #Type, sizeof( Type ), alignof( Type ) }
#define RIPWIRE_LAYOUT_DETAIL_JOIN_IMPL( left, right ) left##right
#define RIPWIRE_LAYOUT_DETAIL_JOIN( left, right ) RIPWIRE_LAYOUT_DETAIL_JOIN_IMPL( left, right )
#define RIPWIRE_LAYOUT_DETAIL_REGISTER( id, ... ) \
namespace \
{ \
RIPWIRE_LAYOUT_USED static const ::rw::layout_registry::TypeLayout \
RIPWIRE_LAYOUT_DETAIL_JOIN( kRipwireLayoutTypes, id )[] = { __VA_ARGS__ }; \
RIPWIRE_LAYOUT_USED static const ::rw::layout_registry::LayoutRecord \
RIPWIRE_LAYOUT_DETAIL_JOIN( kRipwireLayoutRecord, id ) = { \
RIPWIRE_LAYOUT_TU, \
RIPWIRE_LAYOUT_DETAIL_JOIN( kRipwireLayoutTypes, id ), \
sizeof( RIPWIRE_LAYOUT_DETAIL_JOIN( kRipwireLayoutTypes, id ) ) \
/ sizeof( ::rw::layout_registry::TypeLayout ) }; \
RIPWIRE_LAYOUT_USED static const ::rw::layout_registry::Registrar \
RIPWIRE_LAYOUT_DETAIL_JOIN( kRipwireLayoutRegistrar, id )( \
RIPWIRE_LAYOUT_DETAIL_JOIN( kRipwireLayoutRecord, id ) ); \
}
#define RIPWIRE_LAYOUT_REGISTER_TYPES( ... ) RIPWIRE_LAYOUT_DETAIL_REGISTER( __COUNTER__, __VA_ARGS__ )
Loading