Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Phase 03 · Sub-layer 02 — Regenerate CF11 (engineering metrics) from real data

Recommended model: DeepSeek v4 Pro (opencode) — effort high

Routed: carter route -c complex -r subagent -n coding,scientific → deepseek-v4-pro / high (bar 83, capability 83.4)

Real storage-efficiency and graph-op scaling measurements off live tables, plus re-render; complex coding+analysis with a correctness bar on the reported ratios.

Working tree

Repository root. Cluster access required for the storage and graph-op measurements. Does not edit supplementary_figures.tex.

Goal

CF11’s convertible panels are produced from real measurements: UUID-vs-UInt64 compressed storage (bytes/row) from actual ClickHouse column statistics, and graph-operation wall-clock + memory from real connectome-graph runs on ingested connectomes — replacing the synthetic graph-op benchmark artifact.

Why this matters now

CF11 caption: “Panels use the retained controlled synthetic benchmark bundle and synthetic graph-operation benchmark artifact.” Storage efficiency and graph-op scaling are measurable on data already in the cluster — these need not be synthetic.

Out of scope

  • CF04 and CF14/16 (sibling sub-layers).
  • Federation panels (CF15 — handled in merge; not convertible).
  • Caption edits (Phase 03 merge).

Plan

  1. Storage (UUID vs UInt64): query ClickHouse system.columns / system.parts for compressed bytes/row on real ID columns across column classes; compute the ratio. (The id_storage.parquet benchmark already models this — replace its synthetic basis with live system.* numbers.)
  2. Graph-op scaling + memory: run the connectome-graph Rust ops (metrics/motifs/paths/topology) across the ingested connectomes at their real edge counts; record wall-clock and peak memory. This reuses the graph-precompute path that already runs in ETL.
  3. Write real artifacts + provenance under article/reproducibility/.
  4. Regenerate CF11 panels A–H from the real artifacts; recompute the Mann–Whitney annotations on storage panels.
  5. Note for the merge which panels became real and which (if any) remain modelled, so the caption is precise.

Acceptance criteria

  • Storage panels (bytes/row, compression ratio) derive from live ClickHouse system.* statistics, not synthetic rows.
  • Graph-op wall-clock + memory panels derive from real connectome-graph runs on ingested connectomes at their actual edge counts.
  • Provenance file under article/reproducibility/ names the source tables / connectomes and run date.
  • CF11 regenerated; build clean; any panel that remains modelled is explicitly flagged for the caption.

Files likely touched

  • packages/plot/src/syndb_plot/manuscript/composite/fig11_engineering.py.
  • crates/core/connectome-graph/ (only if a measurement harness/bench entry is needed; do not change algorithms).
  • benchmark/results/id_storage.parquet, graph_bench.parquet → real regenerations.
  • article/reproducibility/ — provenance; article/figures/composite/CF11-*.

Pitfalls

  • Symptom: storage ratio looks identical to synthetic. Cause: still reading the synthetic parquet. Recovery: point the loader at system.columns; assert non-synthetic source.
  • Symptom: graph-op run OOMs on the largest connectome. Cause: in-memory graph for FlyWire/MICrONS scale. Recovery: cap at the connectomes that fit (document the cap in provenance and caption — no silent truncation).
  • Symptom: memory numbers non-reproducible. Cause: allocator/》machine variance. Recovery: report on a fixed machine, median of N runs, state the machine in provenance.

Reference

  • crates/core/connectome-graph/ (metrics/motifs/paths/topology).
  • Memory: “Graph precompute in ETL”.
  • Gap inventory item 2.