Phase 03 · Sub-layer 01 — Regenerate CF04 (materialised-view engine) from live cluster
Recommended model: DeepSeek v4 Pro (opencode) — effort
highRouted:
carter route -c complex -r subagent -n coding,scientific→deepseek-v4-pro/high(bar 83, capability 83.4)Running a real MV benchmark on the cluster, validating the numbers are sane, and re-rendering a multi-panel composite is complex coding+analysis; a weaker model risks shipping a mislabelled or mis-scaled panel.
Working tree
Repository root. Requires Nautilus cluster access
(~/.config/syndb/auth.json / $SYNDB_AUTH_TOKEN). Does not edit
supplementary_figures.tex — caption changes happen in the Phase 03 merge.
Goal
CF04’s panels (MV freshness vs raw latency, consolidation gains, concurrent reader latency) are produced from real ClickHouse materialised-view measurements on the production cluster, with a provenance file, replacing the synthetic artifact.
Why this matters now
CF04 caption: “Benchmark panels use the retained controlled synthetic benchmark artifact; they measure the manuscript benchmark workflow rather than production cluster traffic.” The MV engine is real and deployed — measuring it live removes a synthetic label from a capability the platform genuinely has.
Out of scope
- The other synthetic figures (CF11, CF14/16 — sibling sub-layers).
- Editing the caption text (Phase 03 merge owns
supplementary_figures.tex). - Changing the MV schema or the figure’s panel layout/TOML.
Plan
- Find the MV benchmark path:
fig4_analytics.pyand its loaders (_query/materialized_views.py,_query/templates.py) — identify which MVs back the freshness/consolidation/concurrent panels. - Run the benchmark against the production cluster (mirror how the live CF05 capture was produced; reuse the manuscript benchmark controller pattern). Capture: raw-vs-MV latency, pre/post-merge consolidation, concurrent reader latency — real measurements, repeated for percentiles.
- Write the reduced dataframe + provenance (run date, dataset, Helm
revision, benchmark image) under
article/reproducibility/and the manuscript cache. - Regenerate CF04 via the figure pipeline (
just article-panels/article-composites, or the documented composite build) so the panels render from the real artifact. Confirm the statistical annotations (Mann–Whitney/Wilcoxon/Kruskal–Wallis) recompute from real data. - Sanity-check magnitudes against CF05’s live latencies — MV speedups should be directionally consistent (MV faster than raw).
- Hand the merge a one-line note: “CF04 converted; provenance at
<path>; drop ‘synthetic’ from caption.”
Acceptance criteria
- CF04 SVG/PNG regenerated from a real cluster artifact; the synthetic artifact is no longer an input.
- A provenance file under
article/reproducibility/names the run (date, dataset, Helm revision, image), matching the CF05 provenance style. - Panel statistics (U/W/H) are recomputed from the real data (values differ from the synthetic version unless coincidentally equal).
- MV-vs-raw direction is physically sane (MV ≤ raw latency).
Files likely touched
packages/plot/src/syndb_plot/manuscript/composite/fig4_analytics.py(and_query/loaders).packages/plot/src/syndb_plot/manuscript/cache/— new real artifact.article/reproducibility/— provenance file.article/figures/composite/CF04-*.{svg,png}(regenerated).
Pitfalls
- Symptom: figure falls back to synthetic silently. Cause: no cluster auth → loaders use the synthetic generator. Recovery: confirm auth resolves before the run; assert the loader hit the live path (log the source).
- Symptom: empty/degenerate MV result. Cause: the MV isn’t populated for the chosen dataset. Recovery: pick a dataset with materialised views present (per the cache coverage audit, CF08).
- Symptom: composite build fails. Cause: panel size/TOML mismatch after data change. Recovery: re-run the layout solver for CF04’s TOML before compositing.
Reference
- Live-benchmark provenance pattern:
documentation/manuscript/benchmarks/live-cluster-2026-06-06/. .claude/skills/manuscript/build pipeline.- Gap inventory item 1.