Functions
tuple[list, set]	observables_for_mode (str equations, blocked_override=None)

str\|None	equations_mode_of (dict recs)

bool	_same_design (row_sha, design_sha)

dict[int, dict]	load_results (Path res_dir, str\|None design_sha=None)

None	check_provenance (dict[int, dict] recs)

	grid_ok (dict r)

	obs_value (dict r, str observable, at_dose=None)

	dose_ladder_coverage (recs, design_row_ids)

bool	usable (dict r, float dfp_tol=DFP_TOL_DEFAULT, bool require_converged=False, at_dose=None)

	sobol_indices (recs, design_rows, p_keys, observable, n_boot=200, seed=0, dfp_tol=None, require_converged=False, at_dose=None)

	main (argv=None)

Variables
	HERE = Path(__file__).resolve().parent

list	OBSERVABLES

dict	BIN_MOMENT_BLOCKED

int	DFP_TOL_DEFAULT = 1e-2

float	PILE_TOL = 0.05

float	TOPBIN_TOL = 0.02

Detailed Description

merge_and_sobol - concatenate the machines' results and estimate S_i, S_i^T.

    python merge_and_sobol.py --design design/T2_design_v1.csv \
                              --results results/ --out report/

Merging is plain concatenation keyed on row_id, so it is order-independent and
idempotent: re-running after a machine catches up simply adds the new rows.

THE SUBTLETY THAT MAKES THIS FILE WORTH READING
-----------------------------------------------
Saltelli estimators are PAIRWISE.  For parameter j and base index i they need
the triple ( f(A_i), f(B_i), f(AB_i^(j)) ).  The plan says "discard and flag"
failed runs, which is right for the physics but incomplete here:

  * dropping a row globally biases EVERY index, because the A_i and B_i rows are
    shared across all p parameters;
  * dropping nothing and imputing is worse.

The correct handling is PAIRWISE DELETION: if AB_i^(j) is unusable, base index i
is excluded from parameter j's estimator ONLY.  If A_i or B_i is unusable, base
index i is excluded from all p estimators (they genuinely share those rows).
Every index therefore has its own effective sample size, reported as n_eff --
an index computed from 3 of 16 base points is not comparable to one computed
from 16 and must not be read as a screening result.

"Unusable" means failed OR inadmissible.  An inadmissible row (grid-limited or
dose-starved) ran to completion and conserves, but its observables are an
artefact of the numerics; feeding it to the estimator measures the grid, not
the physics.  See run_ensemble.py for why delta_FP cannot detect this.

Function Documentation

◆ _same_design()

bool merge_and_sobol._same_design	(	row_sha,
		design_sha
	)

protected

Compare design hashes that may be stored at different lengths.

The rows carry a 16-char prefix (run_ensemble truncates for compactness)
while design/<name>.meta.json holds the full 64-char digest.  A naive ==
is therefore always False, which silently rejects EVERY row -- the
filter's failure mode is to discard the whole campaign rather than to let
a stale one through, so it must be prefix-aware.

Definition at line 105 of file merge_and_sobol.py.

◆ check_provenance()

None merge_and_sobol.check_provenance ( dict[int, dict] recs )

Four machines must have run the SAME code against the SAME design.

Definition at line 160 of file merge_and_sobol.py.

◆ dose_ladder_coverage()

merge_and_sobol.dose_ladder_coverage	(	recs,
		design_row_ids
	)

rung -> how many of the design's rows reached it. Picks the common dose.

Definition at line 296 of file merge_and_sobol.py.

◆ equations_mode_of()

str | None merge_and_sobol.equations_mode_of ( dict recs )

What mode produced these rows?  Read from the manifest-backed rows.

Definition at line 97 of file merge_and_sobol.py.

◆ grid_ok()

merge_and_sobol.grid_ok ( dict r )

Re-derive the grid verdict from the recorded pile / top-bin quantities.

Deliberately does NOT trust the row's stored `grid_limited`.  Rows written
before 2026-08-03 had the withdrawn `occ > 0.10` rule folded into that flag,
and on the stopped v2 campaign it alone condemned 71 of 275 rows that show
no measurable truncation (plan S11(c)-2).  Recomputing here is what lets a
superseded scoring rule be undone without re-running anything.

Returns (ok, reason).  reason is None when ok.

Definition at line 249 of file merge_and_sobol.py.

◆ load_results()

dict[int, dict] merge_and_sobol.load_results	(	Path	res_dir,
		str \| None	design_sha = `None`
	)

Rows from results/*.jsonl, restricted to ONE design when design_sha is
given -- and it always should be from a CLI that knows the design.

row_id is unique within a design, NOT across designs.  A superseded
campaign's file left in results/ therefore does not merely add noise: its
row 25 overwrites the current design's row 25, and the Sobol estimator
then attributes that value to a completely different theta.  check_provenance
would report the design_sha256 split, but only as a printed warning after
the rows had already been merged.  Filtering here makes it structural.

Definition at line 120 of file merge_and_sobol.py.

◆ main()

merge_and_sobol.main ( argv = None )

Definition at line 414 of file merge_and_sobol.py.

◆ obs_value()

merge_and_sobol.obs_value	(	dict	r,
		str	observable,
			at_dose = `None`
	)

The observable, read either at end-of-run or off the dose ladder.

at_dose=None reads the end of the trajectory -- correct only when every row
ran to the same dose.  Passing a rung reads `at_dose[rung]`, which is what
makes a timed-out row comparable with a complete one instead of merely
present alongside it.

Definition at line 281 of file merge_and_sobol.py.

◆ observables_for_mode()

tuple[list, set] merge_and_sobol.observables_for_mode	(	str	equations,
			blocked_override = `None`
	)

Observables screenable in this equations mode, and those withheld.

Definition at line 88 of file merge_and_sobol.py.

◆ sobol_indices()

merge_and_sobol.sobol_indices	(	recs,
		design_rows,
		p_keys,
		observable,
		n_boot = `200`,
		seed = `0`,
		dfp_tol = `None`,
		require_converged = `False`,
		at_dose = `None`
	)

Saltelli/Jansen estimators with pairwise deletion.

S_j    = mean_i f(B_i) [ f(AB_i^j) - f(A_i) ] / V        (Saltelli 2010)
S_T_j  = mean_i [ f(A_i) - f(AB_i^j) ]^2 / (2 V)         (Jansen 1999)

Definition at line 348 of file merge_and_sobol.py.

◆ usable()

bool merge_and_sobol.usable	(	dict	r,
		float	dfp_tol = `DFP_TOL_DEFAULT`,
		bool	require_converged = `False`,
			at_dose = `None`
	)

Does this row enter a Sobol index?

DEFAULT POLICY since 2026-08-05 (author, plan S11(q)): a row is usable if it
ran and reached the dose.  Truncation of the size tail does not disqualify
it, because the campaign estimates a RANKING of parameters, not an absolute
observable, and the ranking tolerates a truncated tail.

require_converged=True restores the pre-2026-08-05 rule (grid_ok + delta_FP).
Keep it working: re-estimating the indices both ways is the cheap empirical
answer to "did truncation change the ranking", and it costs no CPU because
every gate quantity is still recorded per row.  See --require-converged.

Definition at line 308 of file merge_and_sobol.py.

Variable Documentation

◆ BIN_MOMENT_BLOCKED

dict merge_and_sobol.BIN_MOMENT_BLOCKED

Initial value:

=  {"N_loops_100", "d_100_nm", "f_100_number", "f_100_content",
                      "f_100_tem_0p8", "f_100_tem_1", "f_100_tem_1p25",
                      "f_100_tem_1p5"}

Definition at line 83 of file merge_and_sobol.py.

◆ DFP_TOL_DEFAULT

int merge_and_sobol.DFP_TOL_DEFAULT = 1e-2

Definition at line 242 of file merge_and_sobol.py.

◆ HERE

merge_and_sobol.HERE = Path(__file__).resolve().parent

Definition at line 41 of file merge_and_sobol.py.

◆ OBSERVABLES

list merge_and_sobol.OBSERVABLES