Brief: SynDB Home Manager module with profiles, and a CLI restructure
1. Ask
The operator (2026-11-17): “Let’s create a home-manager module for SynDB that includes everything. Profiles: developer (data provider + admin tooling); data provider (includes data consumer); data consumer.”
Operator decisions already made (binding):
| Question | Answer |
|---|---|
| Where the module lives | The SynDB flake exports it (homeManagerModules.syndb), so any Nix user or lab can use it; canix imports it from a syndb flake input |
| Consumer CLI | Restructure the CLI code so consumer commands compile without ETL, federation and the other heavy code. This is a real refactor of the crate’s module and command layout, not just feature flags |
| Python libraries | No. Profiles carry only the CLI and MCP (no syndb_client, plot or nwb Python packaging) |
| Canix hosts | Developer profile on all dev hosts (atlas, nomad and others; canix topology decides which hosts count as dev) |
2. Deliverable
plan.md beside this brief, plus a reply of at most 10 lines with operator-only questions.
- Executors: the long-coding route (
opencode-go/deepseek-v4.1-flash, max) implements one slice at a time; Sol (openai/gpt-6.1-sol, xhigh) reviews each slice (sol-loop). Give PR-sized slices; slice 1 ships alone. - Each slice: goal, files, Cargo features, Nix outputs, module options (name, type, default), exact test and check names, done criteria. Cite
path:lineat SynDBorigin/trunk(68876577) and canixorigin/trunk. - One opinionated design, with rejected alternatives as one line each, in tables.
- Profile matrix: for each profile, which commands, features, packages, config files, MCP servers and secrets.
- Migration table: every current consumer of
syndb-cli/syndb-cli-tools/ CLI features (Nix packages, release bundles, cross builds, OCI images, CI, docs, scripts) and what happens to each.
3. Angles worth weighing (not mandates)
- Feature layout:
consumer(auth, api, query, search, download, completions),provider(= consumer + dataset upload, etl export, review assessor MCP),developer(= provider + ops, k8s, dev, stack, bench, ci, article, evidence, test, all MCP modes). Map each to a Nix package withmeta.mainProgram = "syndb"; profiles pick exactly one, since they collide on thesyndbbinary. - Restructure: move ETL-, federation- and article-only code behind module boundaries so the consumer build compiles fewer crates (time it). Consider a separate
syndb-cli-corecrate vs feature-gated modules. A consumer build must not pull polars, arrow-flight, ClickHouse or ETL GPU code if avoidable. - HM module options:
programs.syndb.enable,.profile(enum),.package(overridable),.serverUrl(null means the CLI default fromendpoints.nix),.credentialsFile(path, e.g. an agenix-delivered file, linked to~/.config/syndb/credentials.json; null means none),.mcp.enable, and an exposed.mcp.serversattrset (command, args, env) that harness modules (Claude Code, OpenCode, canix fleetix.mcp.servers) can consume. The review MCP’sSYNDB_REVIEW_SERVER_URLis set explicitly, never by default. - Shell completions (
syndb completions <shell>exists) installed by the module. - A HM module check: an evaluation test (home-manager
lib.homeManagerConfigurationper profile) asserting the right package and files, cheap and with no build. - Canix consumption: the pin needs every profile package prebuilt into the canix-fleet cache first. Uncached SynDB builds blocked the canix pin train for hours on 2026-11-15, and the operator then required caching first. The pin URL has no
submodules=1(vendor/queryfabric and vendor/topoxide are flake inputs; submodules break crane’s dummy-src) and no--follows. Canix already delivers~/.config/syndb/credentials.json(canixhome/user/can/repositories/syndb_owner_credentials.nix). Decide whether the module owns that link or canix keeps it.
4. Hard constraints
- SynDB AGENTS.md: ETL logic in Rust; no compiler env overrides; versions in
versions.nix; public URLs come only fromendpoints.nix→shared::endpoints(generated, drift-tested); no hard-coded origins anywhere (a lint test enforces it). - The review MCP stays assessor-only with no default server (see
crates/tools/cli/src/mcp/review.rs,documentation/src/reference/mcp.md). - Lazy senior dev: no abstractions beyond what’s needed; no new dependencies unless unavoidable.
- Qualification: local and bounded (
systemd-run --user -p MemoryMax=… --pipe --wait … </dev/null, from the main checkout’s direnv, with a private target dir); SynDB merges need sol-loop review, not hosted Actions. Nix package builds are heavy: say which builds each slice needs and keep them to the changed outputs. - Deploy: the 0.11.1 rollout freeze is Tue 2026-11-17 22:00Z for API-visible changes. This work is CLI and Nix only; say so, or flag anything API-visible.
- Canix: never edit
flake.nix/flake.lockby hand; pins go through the pin train (pin_request), and only after the prebuild into canix-fleet (canix cache binary push --cache canix-fleet). Canix Home Manager modules live underhome/. Ask before any host switch. - A known red baseline:
cargo build -p cli --no-default-featuresfails on trunk (~99 errors in k8s/etl_jobs/flight.rs etc., seen 2026-11-14). The restructure must make the consumer feature set build, and add a check that keeps it building.
5. Repo map (don’t re-survey beyond it; read the cited files)
| Path | What |
|---|---|
SynDB crates/tools/cli/Cargo.toml [features] | default=dev; prod=dataset+federation+etl-full; etl, graph-precompute, etl-full, etl-gpu, bench, ci, article-tools, mcp, dev=prod+article-tools, full=dev+bench+ci+mcp |
SynDB crates/tools/cli/src/main.rs:17-64,120-271 | command enum and gates. Ungated today: Auth, Api, Query, Review, Article, Evidence, Test, Completions. Gated: Ops (etl-full), Dev, Data (dataset), Etl/Migrate (etl), Stack (dev), Federation, Bench, GraphPrecompute, Ci, Mcp |
SynDB crates/tools/cli-common/ | auth, config, flight client, urls.rs (defaults from shared::endpoints) |
SynDB nix/packages.nix:224-340 | depsLayerFor; syndb-cli (-p cli --no-default-features --features prod, :261); syndb-cli-tools (:272, full); syndb-export-provider (:284, Python CAVE export app); syndb-cli-etl-image, syndb-cli-bench-image; cross builds syndb-cli-<target> (:188-222) |
SynDB nix/dev-shells.nix | the devshell builds syndb-cli-tools in the background |
SynDB flake.nix | flake-parts; perSystem at :299; no homeManagerModules or nixosModules yet |
SynDB scripts/syndb-mcp.sh, .mcp.json, documentation/src/reference/mcp.md | 4 MCP modes (figures, check, ops, review) via syndb mcp <mode>; .#syndb-cli-tools provides them |
SynDB crates/core/shared/src/endpoints.rs, endpoints.nix | public origins |
canix home/user/can/repositories/syndb_owner_credentials.nix | agenix → ~/.config/syndb/credentials.json |
canix docs/src/operations/workspace-update.md (pin train), docs/src/operations/attic-cache.md (canix-fleet push) | the pin and cache procedure |
canix lib/topology/ (Pkl) and fleetix_show | which hosts are dev hosts |
6. Facts
No scout: everything is in-repo. Earlier evidence: syndb-cli at SynDB c3d7e67f is in canix-fleet (/nix/store/0bvfbys0sgv7xm8frzsjwms97kagkm8f-syndb-cli-0.1.0); building it took about 12 minutes on atlas with the deps layer warm. syndb-cli-tools and syndb-export-provider were never pushed to canix-fleet.
7. References
None (no UI).