Phase 03 · Sub-layer 02 — Regenerate CF11 (engineering metrics) from real data
Recommended model: DeepSeek v4 Pro (opencode) — effort
highRouted:
carter route -c complex -r subagent -n coding,scientific→deepseek-v4-pro/high(bar 83, capability 83.4)Real storage-efficiency and graph-op scaling measurements off live tables, plus re-render; complex coding+analysis with a correctness bar on the reported ratios.
Working tree
Repository root. Cluster access required for the storage
and graph-op measurements. Does not edit supplementary_figures.tex.
Goal
CF11’s convertible panels are produced from real measurements: UUID-vs-UInt64
compressed storage (bytes/row) from actual ClickHouse column statistics, and
graph-operation wall-clock + memory from real connectome-graph runs on
ingested connectomes — replacing the synthetic graph-op benchmark artifact.
Why this matters now
CF11 caption: “Panels use the retained controlled synthetic benchmark bundle and synthetic graph-operation benchmark artifact.” Storage efficiency and graph-op scaling are measurable on data already in the cluster — these need not be synthetic.
Out of scope
- CF04 and CF14/16 (sibling sub-layers).
- Federation panels (CF15 — handled in merge; not convertible).
- Caption edits (Phase 03 merge).
Plan
- Storage (UUID vs UInt64): query ClickHouse
system.columns/system.partsfor compressed bytes/row on real ID columns across column classes; compute the ratio. (Theid_storage.parquetbenchmark already models this — replace its synthetic basis with livesystem.*numbers.) - Graph-op scaling + memory: run the
connectome-graphRust ops (metrics/motifs/paths/topology) across the ingested connectomes at their real edge counts; record wall-clock and peak memory. This reuses the graph-precompute path that already runs in ETL. - Write real artifacts + provenance under
article/reproducibility/. - Regenerate CF11 panels A–H from the real artifacts; recompute the Mann–Whitney annotations on storage panels.
- Note for the merge which panels became real and which (if any) remain modelled, so the caption is precise.
Acceptance criteria
- Storage panels (bytes/row, compression ratio) derive from live ClickHouse
system.*statistics, not synthetic rows. - Graph-op wall-clock + memory panels derive from real
connectome-graphruns on ingested connectomes at their actual edge counts. - Provenance file under
article/reproducibility/names the source tables / connectomes and run date. - CF11 regenerated; build clean; any panel that remains modelled is explicitly flagged for the caption.
Files likely touched
packages/plot/src/syndb_plot/manuscript/composite/fig11_engineering.py.crates/core/connectome-graph/(only if a measurement harness/bench entry is needed; do not change algorithms).benchmark/results/id_storage.parquet,graph_bench.parquet→ real regenerations.article/reproducibility/— provenance;article/figures/composite/CF11-*.
Pitfalls
- Symptom: storage ratio looks identical to synthetic. Cause: still reading the synthetic parquet. Recovery: point the loader at
system.columns; assert non-synthetic source. - Symptom: graph-op run OOMs on the largest connectome. Cause: in-memory graph for FlyWire/MICrONS scale. Recovery: cap at the connectomes that fit (document the cap in provenance and caption — no silent truncation).
- Symptom: memory numbers non-reproducible. Cause: allocator/》machine variance. Recovery: report on a fixed machine, median of N runs, state the machine in provenance.
Reference
crates/core/connectome-graph/(metrics/motifs/paths/topology).- Memory: “Graph precompute in ETL”.
- Gap inventory item 2.