diff --git a/notes/issue-633-glift-exact-lab-20260908.md b/notes/issue-633-glift-exact-lab-20260908.md new file mode 100644 index 000000000..8ad43a969 --- /dev/null +++ b/notes/issue-633-glift-exact-lab-20260908.md @@ -0,0 +1,101 @@ +# #661 — Compatible g-lift vs exact-lab ΔZ (after the #633/#653 repairs) + +Leftover of #633, not a new production, not #622. One comparison, one notes +file. No N725 rerun, no new Monte Carlo, no STATUS, draft PR, no merge. + +**Objects used (all existing):** + +| object | source | +|---|---| +| frozen g (9 deciles, 725 not in the freeze) | `results/type582-residual/latest.json` `consensus_g`, cross-checked identical to `results/p612-n725-score/latest.json` spin0 consensus | +| repaired exact laboratory | `results/probe-invariant-shape/census-exact.json` (PR #653; bond `dual_fail = 0`; PR #628's broken bond Z **not** copied) | +| PR #655 lineage | `results/probe-invariant-shape/n725-zflow-corrected.json`, `scripts/threshold_quantile_lineage.py` (brought here unchanged from `analysis/633-n725-spin0`) | +| withdrawn historical reading | `results/probe-invariant-shape/blindness-and-glift.json` (93.9° raw / 159.6° affine-removed — withdrawn in #633) | + +**Engine:** `scripts/probe_invariant_shape/issue661_glift_exact_lab.py` → +`results/probe-invariant-shape/issue661-glift-exact-lab.json`. + +--- + +## Formula and anchors + +```text +DZ_Q[g](u) = (g(u)−g(a))/W − (Q(u)−Q(a))(g(b)−g(a))/W² +W = Q(b)−Q(a) +``` + +Shared anchors **a = 0.2, b = 0.8** (both exist on g's 9-decile grid; no +interpolation of g, so the 1/4–3/4 sensitivity is not even computed). +Angles are **unoriented** (|cosine|), per the #633 retraction of the +159.6° oriented reading. + +**Chart exactness.** The census stores Q only at the 1/4, 3/4 anchors and +Z on the 9 deciles; Q on the deciles is recovered as `Q(u) = Q(1/4) + (Q(3/4)−Q(1/4))·Z(u)`, +i.e. exactly up to an affine map of Q. The lift direction is invariant +under `Q ↦ cQ + d` (because `DZ_{cQ+d}[g] = DZ_Q[g]/c`), so the angle +below is chart-exact, not an interpolation. Nothing about the anchor pair +0.2/0.8 is missing on either side, so no angle is left undetermined. + +**Comparison object.** ΔZ = Z_{L=4} − Z_{L=3} on the same deciles, +recomputed from the repaired census — the same object named in #633 +(`blindness-and-glift.json` direction 6). The exact laboratory is +deterministic (a census), so it carries no sampling covariance; the N725 +jackknife covariance lives in `n725-zflow-corrected.json` and is cited, +not recomputed. + +--- + +## Result: g-lift vs exact-lab ΔZ, unoriented + +| Q used for the lift | unoriented angle | \|cosine\| | +|---|---:|---:| +| site L=3 (primary) | **24.87°** | 0.9073 | +| site L=4 | 24.72° | 0.9083 | +| bond L=3 (dual_fail = 0) | 25.81° | 0.9003 | + +Primary reading (spin0 g, Q = site L=3, ΔZ = Z_{L=4}−Z_{L=3}): + +```text +DZ_Q[g]: [-0.0459, 0.0000, 0.2328, 0.4543, 0.6099, 0.6550, 0.5133, 0.0000, -1.5582] +ΔZ_exact: [-0.0068, 0.0002, -0.0010, -0.0035, -0.0055, -0.0057, -0.0031, 0.0049, 0.0258] +unoriented angle 24.87° (|cos| = 0.9073) +``` + +**What this means.** The g-lift with anchors 0.2/0.8 is a *chart-compatible* +version of the #633 direction-6 question: g is lifted into the affine chart +of Q before the angle is taken, instead of comparing raw 9-vectors. On the +exact laboratory the lifted g and the lab tangent are close to parallel +(|cos| ≈ 0.91, unoriented ≈ 24.9°) — a materially different picture from +the withdrawn raw 93.9°/159.6° reading, and from the raw tiny-torus +unoriented 20.4°. + +**Scope limits, stated not hidden:** + +* The exact-lab/tiny-torus ΔZ is a toy-size tangent. **It cannot prove an + asymptotic mismatch**, and the ~25° agreement here does not prove a lift + in the N-regime either. +* On N725 there is still no second production size, so N725 g-vs-ΔZ + remains **undetermined** — unchanged from #655. The 81.4° number in + `n725-zflow-corrected.json` is g-lift vs Z_725 in the 0.2/0.8 chart (a + same-size parallelism check), **not** the g-vs-ΔZ test and not used as + an incompatibility claim. +* The `equal` consensus g (a different freeze vector, not just a + reweighting) gives 25.10° / 24.97° / 25.95° across the same three Q's — + sensitivity only, not a second vote (Δ < 0.3° on the primary). + +--- + +## What was NOT done + +* No N725 rerun; `n725_100m.hist.csv` untouched. +* No new production, no new S theory (#622 territory). +* No STATUS edit, no merge, #633/#622 stay open. +* The PR-#628 broken bond Z (dual_fail = 118133) is asserted against: the + script refuses to run on a census whose bond `dual_fail ≠ 0`. + +Reproduce: + +```bash +python3 scripts/probe_invariant_shape/issue661_glift_exact_lab.py +python3 -m unittest discover -s tests -p "test_issue661*" +``` diff --git a/results/probe-invariant-shape/census-exact.json b/results/probe-invariant-shape/census-exact.json index 47ab0e22a..ecc61153a 100644 --- a/results/probe-invariant-shape/census-exact.json +++ b/results/probe-invariant-shape/census-exact.json @@ -97,31 +97,26 @@ "3": { "bonds": 18, "configs": 262144, - "dual_fail": 118133, + "dual_fail": 0, "rank_pair_counts": { - "0,2": 51704, - "1,2": 49263, - "0,1": 4397, - "1,1": 41839, - "1,0": 5633, - "2,2": 12183, - "2,1": 46657, - "2,0": 50468 + "0,2": 75460, + "1,1": 111224, + "2,0": 75460 }, - "M_half": "-309/65536", - "M_half_float": -0.0047149658203125, - "Q_quarter": 0.06398471681373033, - "Q_threequarters": 0.0756899475848746, + "M_half": "0", + "M_half_float": 0.0, + "Q_quarter": 0.3882947252165274, + "Q_threequarters": 0.6117052747834726, "Z_levels_float": [ - -0.7416617264257065, - -0.1881488746822507, - 0.15741237889336662, - 0.41262227554113784, - 0.6164158893134861, - 0.7867630751111613, - 0.9335085336027731, - 1.062655985722729, - 1.1781480664979216 + -0.4351929790516855, + -0.12188426707886556, + 0.11038357169268018, + 0.31127822617880646, + 0.4999999999999995, + 0.688721773821193, + 0.8896164283073188, + 1.1218842670788656, + 1.435192979051685 ], "self_dual_p_half": 0.5 } diff --git a/results/probe-invariant-shape/issue661-glift-exact-lab.json b/results/probe-invariant-shape/issue661-glift-exact-lab.json new file mode 100644 index 000000000..fa51f4ed2 --- /dev/null +++ b/results/probe-invariant-shape/issue661-glift-exact-lab.json @@ -0,0 +1,228 @@ +{ + "schema": "matching-one.probe-invariant-shape.issue661-glift-exact-lab.v1", + "issue": 661, + "leftover_of": 633, + "not_a_new_production": true, + "not_622": true, + "n725_rerun": false, + "levels": [ + 0.1, + 0.2, + 0.3, + 0.4, + 0.5, + 0.6, + 0.7, + 0.8, + 0.9 + ], + "g_anchors": [ + 0.2, + 0.8 + ], + "g": [ + -0.2887126766006569, + -0.12066783122729056, + 0.06921759100370817, + 0.2385347464569182, + 0.3780737808016512, + 0.47827324271813404, + 0.518589378735454, + 0.444347665976014, + 0.048264834818264817 + ], + "g_provenance": [ + "results/type582-residual/latest.json consensus_g", + "results/p612-n725-score/latest.json weightings.spin0.consensus_direction_frozen_from_five_committed_transitions" + ], + "g_freeze_excludes_725": true, + "inputs": { + "census": "results/probe-invariant-shape/census-exact.json (PR #653 repaired; bond dual_fail = 0)", + "n725_corrected": "results/probe-invariant-shape/n725-zflow-corrected.json (PR #655, cited not recomputed)", + "historical_withdrawn": "results/probe-invariant-shape/blindness-and-glift.json" + }, + "formula": "DZ_Q[g](u) = (g(u)-g(a))/W - (Q(u)-Q(a))(g(b)-g(a))/W^2, W = Q(b)-Q(a), a=0.2 b=0.8", + "chart_note": "Q on the deciles is recovered from the census Z exactly up to affine; the lift direction is invariant under Q -> cQ + d (DZ_{cQ+d}[g] = DZ_Q[g]/c), so the angle is chart-exact.", + "exact_lab_deltaZ_site_L4_minus_L3": [ + -0.006797699629817133, + 0.0002479930160355637, + -0.0009907372721496138, + -0.003537707044666105, + -0.0054824715154933035, + -0.005748146216893901, + -0.0030877168533604005, + 0.004938246020020465, + 0.02583032703854138 + ], + "angles": { + "spin0": { + "DZ_Q3_vs_deltaZ_site": { + "DZ_Q[g]": [ + -0.04590685144626999, + 0.0, + 0.23275668196968086, + 0.45426388884666424, + 0.6099303197327367, + 0.6549966226111241, + 0.5133120513829883, + 0.0, + -1.5582388669503862 + ], + "Q_used": "site L=3 quantile on deciles (from census Z, affine-exact)", + "angle": { + "undetermined": false, + "abs_cosine": 0.9072881745290168, + "unoriented_angle_deg": 24.866753181486924, + "oriented_angle_deg": 155.13324681851307, + "supplement_deg": 24.866753181486928 + } + }, + "DZ_Q4_vs_deltaZ_site": { + "DZ_Q[g]": [ + -0.0467072732003041, + 0.0, + 0.2906501536787948, + 0.5694317237611014, + 0.7659051017146012, + 0.823125854548467, + 0.6451526874124143, + 0.0, + -1.9549459013266768 + ], + "Q_used": "site L=4 quantile on deciles (from census Z, affine-exact)", + "angle": { + "undetermined": false, + "abs_cosine": 0.9083441109890034, + "unoriented_angle_deg": 24.722486365157316, + "oriented_angle_deg": 155.27751363484268, + "supplement_deg": 24.722486365157323 + } + }, + "DZ_Qbond_vs_deltaZ_site": { + "DZ_Q[g]": [ + -0.0925461373162797, + 0.0, + 0.30363539443401505, + 0.5845402808640013, + 0.7781807179715399, + 0.830246212876729, + 0.6469032505536589, + 0.0, + -1.9376320845061075 + ], + "Q_used": "bond L=3 quantile on deciles (from census Z, affine-exact)", + "angle": { + "undetermined": false, + "abs_cosine": 0.9002696949407579, + "unoriented_angle_deg": 25.806459892823096, + "oriented_angle_deg": 154.19354010717691, + "supplement_deg": 25.806459892823085 + } + } + }, + "equal": { + "DZ_Q3_vs_deltaZ_site": { + "DZ_Q[g]": [ + -0.06074480777126601, + 0.0, + 0.2690998907679317, + 0.5231614255047904, + 0.7007382235322221, + 0.7511455365479518, + 0.5877518445553369, + 0.0, + -1.7786716284121584 + ], + "Q_used": "site L=3 quantile on deciles (from census Z, affine-exact)", + "angle": { + "undetermined": false, + "abs_cosine": 0.9055334394141108, + "unoriented_angle_deg": 25.10477552619196, + "oriented_angle_deg": 154.89522447380804, + "supplement_deg": 25.104775526191958 + } + }, + "DZ_Q4_vs_deltaZ_site": { + "DZ_Q[g]": [ + -0.0645774503946358, + 0.0, + 0.3356090283629875, + 0.6547639839121768, + 0.8784412084587006, + 0.9423167591680894, + 0.737441641740441, + 0.0, + -2.2281236532821502 + ], + "Q_used": "site L=4 quantile on deciles (from census Z, affine-exact)", + "angle": { + "undetermined": false, + "abs_cosine": 0.906540049437126, + "unoriented_angle_deg": 24.968492825628104, + "oriented_angle_deg": 155.0315071743719, + "supplement_deg": 24.96849282562809 + } + }, + "DZ_Qbond_vs_deltaZ_site": { + "DZ_Q[g]": [ + -0.11168753664293107, + 0.0, + 0.34775293253809186, + 0.6679021943687657, + 0.8878200232915316, + 0.9461404725444196, + 0.7364937344340572, + 0.0, + -2.2019474775399637 + ], + "Q_used": "bond L=3 quantile on deciles (from census Z, affine-exact)", + "angle": { + "undetermined": false, + "abs_cosine": 0.8991400297273758, + "unoriented_angle_deg": 25.954742784699015, + "oriented_angle_deg": 154.04525721530098, + "supplement_deg": 25.95474278469902 + } + } + } + }, + "sensitivity_equal_weighting": { + "note": "Equal weighting is a sensitivity, not a second vote.", + "delta_unoriented_deg_primary": -0.23802234470503691 + }, + "historical_withdrawn_reading": { + "source": "results/probe-invariant-shape/blindness-and-glift.json", + "raw_deg": 93.91923544902247, + "affine_removed_deg": 159.62484413088862, + "unoriented_deg": 20.37515586911138, + "status": "withdrawn in #633 (raw oriented angle and affine-removed reading replaced by unoriented 20.4)" + }, + "scope_limits": [ + "The exact laboratory is deterministic: no sampling covariance for Delta Z_exact; the N725 jackknife covariance lives in n725-zflow-corrected.json and is cited, not recomputed.", + "Tiny-torus / exact-lab Delta Z cannot prove an asymptotic mismatch.", + "Equal weighting is a sensitivity, not a second vote." + ], + "n725_cited_not_recomputed": { + "file": "results/probe-invariant-shape/n725-zflow-corrected.json", + "g_lift_DZ_Q_anchors_0.2_0.8": [ + -0.21272027493304035, + 0.0, + 1.2412422534636602, + 2.428553083731784, + 3.263644590503705, + 3.50488701715073, + 2.7449924738210703, + 0.0, + -8.298053476346686 + ], + "angle_g_lift_vs_Z_in_0.2_0.8_chart": { + "undetermined": false, + "abs_cosine": 0.1489195537575465, + "unoriented_angle_deg": 81.43568168217404, + "oriented_angle_deg": 98.56431831782596, + "supplement_deg": 81.43568168217404 + }, + "tiny_torus_unoriented_deg": 20.37515586911138, + "status": "no second production size; N725 g-vs-DeltaZ remains undetermined" + } +} diff --git a/results/probe-invariant-shape/n725-zflow-corrected.json b/results/probe-invariant-shape/n725-zflow-corrected.json new file mode 100644 index 000000000..096c56259 --- /dev/null +++ b/results/probe-invariant-shape/n725-zflow-corrected.json @@ -0,0 +1,274 @@ +{ + "schema": "matching-one.probe-invariant-shape.n725-zflow-corrected.v1", + "issue": 633, + "histogram": "results/server-20260907/P612-n725-fullcurve/raw/n725_100m.hist.csv", + "histogram_git_commit": "8b5f9d1a", + "n": 725, + "levels_11": [ + 0.1, + 0.2, + 0.25, + 0.3, + 0.4, + 0.5, + 0.6, + 0.7, + 0.75, + 0.8, + 0.9 + ], + "deciles": [ + 0.1, + 0.2, + 0.3, + 0.4, + 0.5, + 0.6, + 0.7, + 0.8, + 0.9 + ], + "z_anchors": [ + 0.25, + 0.75 + ], + "g_anchors": [ + 0.2, + 0.8 + ], + "orientation_representative": { + "first": [ + 26, + 7 + ], + "second": [ + 23, + 14 + ] + }, + "orientation_cos4theta": { + "first": 0.4958535077288942, + "second": -0.5780680142687277 + }, + "spin0_weights": { + "first": 0.5382777069160998, + "second": 0.4617222930839002 + }, + "spin0_is_interpolation": true, + "weight_checks": { + "sum_w": 1.0, + "sum_w_cos4": 0.0, + "agrees_with_p612_weights": true, + "agrees_with_p612_cos4": true + }, + "withdrawn_n725_zflow": { + "file": "results/probe-invariant-shape/n725-zflow.json", + "formula": "Re(w^2)/|w|^2 = cos(2 theta), misnamed cos_four_theta", + "reported_cos4": { + "first": 0.8648275862068966, + "second": 0.4593103448275862 + }, + "reported_weights": { + "first": -1.1326530612244896, + "second": 2.13265306122449 + }, + "true_spin4_leakage_of_those_weights": -1.7944485136741972, + "cdf_then_invert": true, + "replaced_0.4_and_0.6_with_quarter_anchors": true, + "mean_of_batchwise_Z_was_reported_as_shape": true + }, + "retractions": { + "full_rank_batch_noise_does_not_kill_one_parameter_mean": "Counterexample: y_N = m + f(N) g + epsilon with epsilon uniform on {\u00b1e_i}. Mean lies on a line; noise covariance is I/d. Withdrawn from #628 \u00a7W3 / #622.", + "159.6_deg_is_not_almost_orthogonal": "Affine-removed angle 159.6248\u00b0 between g and \u0394Z_{L=4-L=3} is 20.375\u00b0 between unoriented lines. Use abs cosine. Tiny-torus \u0394Z cannot prove an asymptotic mismatch." + }, + "weightings": { + "spin0": { + "weights": { + "first": 0.5382777069160998, + "second": 0.4617222930839002 + }, + "Q_pooled": [ + 0.5418336218910869, + 0.5597068065994498, + 0.5663656171761691, + 0.5722960076752488, + 0.582909585882762, + 0.5927464521537054, + 0.6025341207843034, + 0.6129854231453626, + 0.6187764349360221, + 0.6252375697608342, + 0.642367299065417 + ], + "Z_pooled_quarter_anchors": [ + -0.4680712176155713, + -0.1270503087211879, + 0.0, + 0.11315203144230356, + 0.31565942707472294, + 0.5033471352882452, + 0.6900961510247521, + 0.8895073185617152, + 1.0, + 1.1232786493509248, + 1.4501144064854807 + ], + "Q_se": [ + 2.73644662281879e-06, + 2.3514597269879632e-06, + 2.265774742800073e-06, + 2.2023824269102545e-06, + 2.1161444359437407e-06, + 2.0754989975560686e-06, + 2.0731311521363245e-06, + 2.099145547137414e-06, + 2.1259320823291476e-06, + 2.1734829641163383e-06, + 2.4458593294712768e-06 + ], + "Z_se": [ + 2.3760189168853775e-05, + 4.912613057784862e-06, + 0.0, + 3.4009789295155714e-06, + 7.322475908201013e-06, + 8.601512372777377e-06, + 7.539225330619552e-06, + 3.5337233425040574e-06, + 0.0, + 4.938731967303258e-06, + 2.175659724323453e-05 + ], + "Q_deciles": [ + 0.5418336218910869, + 0.5597068065994498, + 0.5722960076752488, + 0.582909585882762, + 0.5927464521537054, + 0.6025341207843034, + 0.6129854231453626, + 0.6252375697608342, + 0.642367299065417 + ], + "g_lift_DZ_Q_anchors_0.2_0.8": [ + -0.21272027493304035, + 0.0, + 1.2412422534636602, + 2.428553083731784, + 3.263644590503705, + 3.50488701715073, + 2.7449924738210703, + 0.0, + -8.298053476346686 + ], + "angle_g_lift_vs_Z_in_0.2_0.8_chart": { + "undetermined": false, + "abs_cosine": 0.1489195537575465, + "unoriented_angle_deg": 81.43568168217404, + "oriented_angle_deg": 98.56431831782596, + "supplement_deg": 81.43568168217404 + }, + "batches": 100, + "note": "Z_pooled is from the pooled histogram, not the mean of batchwise Z" + }, + "equal": { + "weights": { + "first": 0.5, + "second": 0.5 + }, + "Q_pooled": [ + 0.5418342329710413, + 0.5597071641375351, + 0.5663659151254876, + 0.572296259747219, + 0.5829097663412028, + 0.5927465782512797, + 0.6025342095906012, + 0.612985496943196, + 0.6187765129902232, + 0.6252376637635848, + 0.6423674978344991 + ], + "Z_pooled_quarter_anchors": [ + -0.46806720689886405, + -0.12704970481614902, + 0.0, + 0.11315163084070815, + 0.3156585097237889, + 0.503345968192863, + 0.6900950559361854, + 0.8895067737640974, + 1.0, + 1.1232794708817662, + 1.4501185982491738 + ], + "Q_se": [ + 2.7197749916869554e-06, + 2.321661378254843e-06, + 2.2346791468703784e-06, + 2.1718708531769862e-06, + 2.0890560822564635e-06, + 2.0517412012278993e-06, + 2.0507203615553514e-06, + 2.0760822464724183e-06, + 2.10257963819614e-06, + 2.150874948842966e-06, + 2.433992792523544e-06 + ], + "Z_se": [ + 2.3623206077040243e-05, + 4.916990823104154e-06, + 0.0, + 3.411636919986221e-06, + 7.352113507333939e-06, + 8.642322757060646e-06, + 7.579395838977281e-06, + 3.5537193800012226e-06, + 0.0, + 4.967359154525975e-06, + 2.1908291176929045e-05 + ], + "Q_deciles": [ + 0.5418342329710413, + 0.5597071641375351, + 0.572296259747219, + 0.5829097663412028, + 0.5927465782512797, + 0.6025342095906012, + 0.612985496943196, + 0.6252376637635848, + 0.6423674978344991 + ], + "g_lift_DZ_Q_anchors_0.2_0.8": [ + -0.21274503283584156, + 0.0, + 1.2412544605129239, + 2.428573872159751, + 3.2636706848033024, + 3.5049138092185386, + 2.7450126545036646, + 0.0, + -8.298109696111137 + ], + "angle_g_lift_vs_Z_in_0.2_0.8_chart": { + "undetermined": false, + "abs_cosine": 0.1489207304335024, + "unoriented_angle_deg": 81.43561350336135, + "oriented_angle_deg": 98.56438649663865, + "supplement_deg": 81.43561350336135 + }, + "batches": 100, + "note": "Z_pooled is from the pooled histogram, not the mean of batchwise Z" + } + }, + "tiny_torus_g_vs_deltaZ": { + "source": "results/probe-invariant-shape/blindness-and-glift.json", + "raw_deg": 93.91923544902247, + "affine_removed_deg": 159.62484413088862, + "unoriented_deg": 20.37515586911138, + "claim": "report unoriented 20.4\u00b0, not 159.6\u00b0 as almost-orthogonal" + }, + "exact_lab_census_present": true, + "exact_lab_dual_fail_site": null, + "exact_lab_note": "g vs exact-lab \u0394Z uses this branch's repaired census; do not copy PR #628's broken bond Z." +} \ No newline at end of file diff --git a/scripts/probe_invariant_shape/issue661_glift_exact_lab.py b/scripts/probe_invariant_shape/issue661_glift_exact_lab.py new file mode 100644 index 000000000..f0f06a51f --- /dev/null +++ b/scripts/probe_invariant_shape/issue661_glift_exact_lab.py @@ -0,0 +1,224 @@ +#!/usr/bin/env python3 +"""#661: compatible g-lift vs exact-lab ΔZ, on existing objects only. + +Leftover of #633/#655. Not a new production, not #622. Reads: + +* frozen g — ``results/type582-residual/latest.json`` ``consensus_g`` + (cross-checked against ``results/p612-n725-score/latest.json`` spin0 + consensus; 725 is not in the freeze), +* the repaired exact laboratory — ``results/probe-invariant-shape/census-exact.json`` + (PR #653, bond ``dual_fail = 0``; PR #628's broken bond Z is NOT copied), +* the historical withdrawn reading — ``blindness-and-glift.json``. + +Formula (issue #661): + + DZ_Q[g](u) = (g(u) - g(a))/W - (Q(u) - Q(a)) (g(b) - g(a))/W^2 + W = Q(b) - Q(a) + +Shared anchors a = 0.2, b = 0.8 (both on g's 9-decile grid). Angles are +UNORIENTED (|cosine|). The exact-lab comparison object is the site tangent +ΔZ = Z_{L=4} − Z_{L=3} on the same deciles, the object named in #633 and +recomputed here from the repaired census. The lift is invariant under +affine reparametrisation of Q (Q ↦ cQ + d scales DZ_Q[g] by 1/c), so +recovering Q on the deciles from the census Z (which fixes Q up to affine) +is exact for the direction. + +Covariance: the exact laboratory is deterministic (census), so there is no +sampling covariance for ΔZ_exact; the N725 covariance lives in +``n725-zflow-corrected.json`` and is cited, not recomputed (no rerun). +No N725 Monte Carlo is run. Equal weighting is a labelled sensitivity. +""" +from __future__ import annotations + +import json +import math +from fractions import Fraction +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[2] + +CENSUS = ROOT / "results/probe-invariant-shape/census-exact.json" +TYPE582 = ROOT / "results/type582-residual/latest.json" +P612 = ROOT / "results/p612-n725-score/latest.json" +GLIFT_HIST = ROOT / "results/probe-invariant-shape/blindness-and-glift.json" +N725_CORR = ROOT / "results/probe-invariant-shape/n725-zflow-corrected.json" +OUT = ROOT / "results/probe-invariant-shape/issue661-glift-exact-lab.json" + +LEVELS = (0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9) +A_G, B_G = 0.2, 0.8 + + +def unoriented_angle_deg(u: list[float], v: list[float]) -> dict: + nu = math.sqrt(sum(x * x for x in u)) + nv = math.sqrt(sum(x * x for x in v)) + if nu == 0.0 or nv == 0.0: + return {"undetermined": True, "reason": "zero vector"} + cos = sum(x * y for x, y in zip(u, v)) / (nu * nv) + cos = max(-1.0, min(1.0, cos)) + oriented = math.degrees(math.acos(cos)) + return { + "undetermined": False, + "abs_cosine": abs(cos), + "unoriented_angle_deg": math.degrees(math.acos(abs(cos))), + "oriented_angle_deg": oriented, + "supplement_deg": 180.0 - oriented, + } + + +def dz_g_lift(Q: list[float], g: list[float], a: float = A_G, b: float = B_G) -> list[float]: + """DZ_Q[g](u) = (g(u)-g(a))/W - (Q(u)-Q(a))(g(b)-g(a))/W^2.""" + idx = {u: i for i, u in enumerate(LEVELS)} + W = Q[idx[b]] - Q[idx[a]] + if W == 0.0: + raise ValueError("degenerate anchor width W = Q(0.8) - Q(0.2)") + ga, gb = g[idx[a]], g[idx[b]] + return [ + (g[idx[u]] - ga) / W - (Q[idx[u]] - Q[idx[a]]) * (gb - ga) / (W * W) + for u in LEVELS + ] + + +def q_on_deciles_from_census(zdec: list[float], q_quarter: str | float, + q_threequarters: str | float) -> list[float]: + """Recover Q on the deciles from the census Z, exactly up to affine. + + Z(u) = (Q(u) - Q(1/4)) / (Q(3/4) - Q(1/4)), so + Q(u) = Q(1/4) + W_z Z(u). The lift direction is invariant under + Q ↦ cQ + d (verified: DZ_{cQ+d}[g] = DZ_Q[g]/c), so this is exact + for the angle, not an approximation. + """ + q4 = float(Fraction(q_quarter)) if isinstance(q_quarter, str) else q_quarter + q34 = float(Fraction(q_threequarters)) if isinstance(q_threequarters, str) else q_threequarters + wz = q34 - q4 + return [q4 + wz * z for z in zdec] + + +def main() -> None: + census = json.loads(CENSUS.read_text()) + type582 = json.loads(TYPE582.read_text()) + p612 = json.loads(P612.read_text()) + + g = type582["consensus_g"] + assert type582["levels"] == list(LEVELS) + g612 = p612["weightings"]["spin0"][ + "consensus_direction_frozen_from_five_committed_transitions"] + assert g == g612, "type582 consensus_g and p612 spin0 consensus disagree" + g_equal = p612["weightings"]["equal"][ + "consensus_direction_frozen_from_five_committed_transitions"] + + # repaired exact laboratory (PR #653): bond dual_fail must be 0 + assert census["bond"]["3"]["dual_fail"] == 0, ( + "census-exact.json is not the repaired #653 version" + ) + + z3 = census["site"]["3"]["Z_levels_float"] + z4 = census["site"]["4"]["Z_levels_float"] + zb = census["bond"]["3"]["Z_levels_float"] + dz_site = [b - a for a, b in zip(z3, z4)] + + q3 = q_on_deciles_from_census( + z3, census["site"]["3"]["Q_quarter"], census["site"]["3"]["Q_threequarters"]) + q4 = q_on_deciles_from_census( + z4, census["site"]["4"]["Q_quarter"], census["site"]["4"]["Q_threequarters"]) + qb = q_on_deciles_from_census( + zb, census["bond"]["3"]["Q_quarter"], census["bond"]["3"]["Q_threequarters"]) + + out = { + "schema": "matching-one.probe-invariant-shape.issue661-glift-exact-lab.v1", + "issue": 661, + "leftover_of": 633, + "not_a_new_production": True, + "not_622": True, + "n725_rerun": False, + "levels": list(LEVELS), + "g_anchors": [A_G, B_G], + "g": g, + "g_provenance": [ + "results/type582-residual/latest.json consensus_g", + "results/p612-n725-score/latest.json weightings.spin0." + "consensus_direction_frozen_from_five_committed_transitions", + ], + "g_freeze_excludes_725": True, + "inputs": { + "census": "results/probe-invariant-shape/census-exact.json (PR #653 repaired; bond dual_fail = 0)", + "n725_corrected": "results/probe-invariant-shape/n725-zflow-corrected.json (PR #655, cited not recomputed)", + "historical_withdrawn": "results/probe-invariant-shape/blindness-and-glift.json", + }, + "formula": "DZ_Q[g](u) = (g(u)-g(a))/W - (Q(u)-Q(a))(g(b)-g(a))/W^2, W = Q(b)-Q(a), a=0.2 b=0.8", + "chart_note": ( + "Q on the deciles is recovered from the census Z exactly up to affine; " + "the lift direction is invariant under Q -> cQ + d " + "(DZ_{cQ+d}[g] = DZ_Q[g]/c), so the angle is chart-exact." + ), + "exact_lab_deltaZ_site_L4_minus_L3": dz_site, + "angles": {}, + "sensitivity_equal_weighting": {}, + "historical_withdrawn_reading": None, + "scope_limits": [ + "The exact laboratory is deterministic: no sampling covariance for " + "Delta Z_exact; the N725 jackknife covariance lives in " + "n725-zflow-corrected.json and is cited, not recomputed.", + "Tiny-torus / exact-lab Delta Z cannot prove an asymptotic mismatch.", + "Equal weighting is a sensitivity, not a second vote.", + ], + } + + for gname, gv in (("spin0", g), ("equal", g_equal)): + lift3 = dz_g_lift(q3, gv) + lift4 = dz_g_lift(q4, gv) + liftb = dz_g_lift(qb, gv) + out["angles"][gname] = { + "DZ_Q3_vs_deltaZ_site": { + "DZ_Q[g]": lift3, + "Q_used": "site L=3 quantile on deciles (from census Z, affine-exact)", + "angle": unoriented_angle_deg(lift3, dz_site), + }, + "DZ_Q4_vs_deltaZ_site": { + "DZ_Q[g]": lift4, + "Q_used": "site L=4 quantile on deciles (from census Z, affine-exact)", + "angle": unoriented_angle_deg(lift4, dz_site), + }, + "DZ_Qbond_vs_deltaZ_site": { + "DZ_Q[g]": liftb, + "Q_used": "bond L=3 quantile on deciles (from census Z, affine-exact)", + "angle": unoriented_angle_deg(liftb, dz_site), + }, + } + + # labelled sensitivity: spin0 vs equal, primary comparison only + out["sensitivity_equal_weighting"] = { + "note": "Equal weighting is a sensitivity, not a second vote.", + "delta_unoriented_deg_primary": out["angles"]["spin0"]["DZ_Q3_vs_deltaZ_site"]["angle"]["unoriented_angle_deg"] + - out["angles"]["equal"]["DZ_Q3_vs_deltaZ_site"]["angle"]["unoriented_angle_deg"], + } + + if GLIFT_HIST.exists(): + gl = json.loads(GLIFT_HIST.read_text()) + d6 = gl["direction6"] + out["historical_withdrawn_reading"] = { + "source": "results/probe-invariant-shape/blindness-and-glift.json", + "raw_deg": d6["angle_raw_deg"], + "affine_removed_deg": d6["angle_after_affine_removal_deg"], + "unoriented_deg": min(d6["angle_after_affine_removal_deg"] % 180.0, + 180.0 - (d6["angle_after_affine_removal_deg"] % 180.0)), + "status": "withdrawn in #633 (raw oriented angle and affine-removed reading replaced by unoriented 20.4)", + } + + if N725_CORR.exists(): + n725 = json.loads(N725_CORR.read_text()) + w = n725["weightings"]["spin0"] + out["n725_cited_not_recomputed"] = { + "file": "results/probe-invariant-shape/n725-zflow-corrected.json", + "g_lift_DZ_Q_anchors_0.2_0.8": w["g_lift_DZ_Q_anchors_0.2_0.8"], + "angle_g_lift_vs_Z_in_0.2_0.8_chart": w["angle_g_lift_vs_Z_in_0.2_0.8_chart"], + "tiny_torus_unoriented_deg": n725["tiny_torus_g_vs_deltaZ"]["unoriented_deg"], + "status": "no second production size; N725 g-vs-DeltaZ remains undetermined", + } + + OUT.write_text(json.dumps(out, indent=1) + "\n") + print(json.dumps(out["angles"]["spin0"]["DZ_Q3_vs_deltaZ_site"]["angle"], indent=1)) + print("wrote", OUT) + + +if __name__ == "__main__": + main() diff --git a/tests/test_issue661_glift_exact_lab.py b/tests/test_issue661_glift_exact_lab.py new file mode 100644 index 000000000..f0e62fea1 --- /dev/null +++ b/tests/test_issue661_glift_exact_lab.py @@ -0,0 +1,87 @@ +"""#661: checks for the compatible g-lift vs exact-lab DeltaZ comparison.""" +from __future__ import annotations + +import json +import math +import sys +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(ROOT / "scripts" / "probe_invariant_shape")) + +from issue661_glift_exact_lab import ( # noqa: E402 + A_G, + B_G, + LEVELS, + dz_g_lift, + main, + q_on_deciles_from_census, + unoriented_angle_deg, +) + + +class TestIssue661GliftExactLab(unittest.TestCase): + def test_g_vectors_agree_between_the_two_freeze_files(self) -> None: + t582 = json.loads((ROOT / "results/type582-residual/latest.json").read_text()) + p612 = json.loads((ROOT / "results/p612-n725-score/latest.json").read_text()) + self.assertEqual(t582["levels"], list(LEVELS)) + self.assertEqual( + t582["consensus_g"], + p612["weightings"]["spin0"][ + "consensus_direction_frozen_from_five_committed_transitions"]) + + def test_repaired_census_in_use(self) -> None: + census = json.loads( + (ROOT / "results/probe-invariant-shape/census-exact.json").read_text()) + self.assertEqual(census["bond"]["3"]["dual_fail"], 0) + + def test_lift_is_affine_gauge_invariant(self) -> None: + census = json.loads( + (ROOT / "results/probe-invariant-shape/census-exact.json").read_text()) + t582 = json.loads((ROOT / "results/type582-residual/latest.json").read_text()) + g = t582["consensus_g"] + z3 = census["site"]["3"]["Z_levels_float"] + Q = q_on_deciles_from_census( + z3, census["site"]["3"]["Q_quarter"], census["site"]["3"]["Q_threequarters"]) + base = dz_g_lift(Q, g) + scaled = dz_g_lift([3.7 * q - 2.1 for q in Q], g) + n1 = math.sqrt(sum(x * x for x in base)) + n2 = math.sqrt(sum(x * x for x in scaled)) + cos = sum(a * b for a, b in zip(base, scaled)) / (n1 * n2) + self.assertAlmostEqual(abs(cos), 1.0, places=12) + + def test_lift_is_zero_at_anchors(self) -> None: + t582 = json.loads((ROOT / "results/type582-residual/latest.json").read_text()) + g = t582["consensus_g"] + Q = [0.1 * (i + 1) for i in range(len(LEVELS))] + lifted = dz_g_lift(Q, g) + ia, ib = LEVELS.index(A_G), LEVELS.index(B_G) + self.assertEqual(lifted[ia], 0.0) + self.assertEqual(lifted[ib], 0.0) + + def test_unoriented_angle_properties(self) -> None: + a = unoriented_angle_deg([1.0, 0.0], [0.0, 1.0]) + self.assertEqual(a["unoriented_angle_deg"], 90.0) + b = unoriented_angle_deg([1.0, 1.0], [-1.0, -1.0]) + self.assertAlmostEqual(b["unoriented_angle_deg"], 0.0, places=5) # same line, opposite orientation + z = unoriented_angle_deg([1.0], [0.0]) + self.assertTrue(z["undetermined"]) + + def test_main_writes_expected_angles(self) -> None: + main() + out = json.loads( + (ROOT / "results/probe-invariant-shape/issue661-glift-exact-lab.json").read_text()) + ang = out["angles"]["spin0"]["DZ_Q3_vs_deltaZ_site"]["angle"] + self.assertFalse(ang["undetermined"]) + # 0 < angle < 90: the g-lift and the lab tangent are NOT the same line + self.assertGreater(ang["unoriented_angle_deg"], 0.0) + self.assertLess(ang["unoriented_angle_deg"], 90.0) + self.assertAlmostEqual(ang["abs_cosine"], 0.9072881745290168, places=12) + # equal-weighting sensitivity moves the angle by well under a degree + self.assertLess( + abs(out["sensitivity_equal_weighting"]["delta_unoriented_deg_primary"]), 1.0) + + +if __name__ == "__main__": + unittest.main()