Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Brief: SynDB Home Manager module with profiles, and a CLI restructure

1. Ask

The operator (2026-11-17): “Let’s create a home-manager module for SynDB that includes everything. Profiles: developer (data provider + admin tooling); data provider (includes data consumer); data consumer.”

Operator decisions already made (binding):

QuestionAnswer
Where the module livesThe SynDB flake exports it (homeManagerModules.syndb), so any Nix user or lab can use it; canix imports it from a syndb flake input
Consumer CLIRestructure the CLI code so consumer commands compile without ETL, federation and the other heavy code. This is a real refactor of the crate’s module and command layout, not just feature flags
Python librariesNo. Profiles carry only the CLI and MCP (no syndb_client, plot or nwb Python packaging)
Canix hostsDeveloper profile on all dev hosts (atlas, nomad and others; canix topology decides which hosts count as dev)

2. Deliverable

plan.md beside this brief, plus a reply of at most 10 lines with operator-only questions.

  • Executors: the long-coding route (opencode-go/deepseek-v4.1-flash, max) implements one slice at a time; Sol (openai/gpt-6.1-sol, xhigh) reviews each slice (sol-loop). Give PR-sized slices; slice 1 ships alone.
  • Each slice: goal, files, Cargo features, Nix outputs, module options (name, type, default), exact test and check names, done criteria. Cite path:line at SynDB origin/trunk (68876577) and canix origin/trunk.
  • One opinionated design, with rejected alternatives as one line each, in tables.
  • Profile matrix: for each profile, which commands, features, packages, config files, MCP servers and secrets.
  • Migration table: every current consumer of syndb-cli / syndb-cli-tools / CLI features (Nix packages, release bundles, cross builds, OCI images, CI, docs, scripts) and what happens to each.

3. Angles worth weighing (not mandates)

  • Feature layout: consumer (auth, api, query, search, download, completions), provider (= consumer + dataset upload, etl export, review assessor MCP), developer (= provider + ops, k8s, dev, stack, bench, ci, article, evidence, test, all MCP modes). Map each to a Nix package with meta.mainProgram = "syndb"; profiles pick exactly one, since they collide on the syndb binary.
  • Restructure: move ETL-, federation- and article-only code behind module boundaries so the consumer build compiles fewer crates (time it). Consider a separate syndb-cli-core crate vs feature-gated modules. A consumer build must not pull polars, arrow-flight, ClickHouse or ETL GPU code if avoidable.
  • HM module options: programs.syndb.enable, .profile (enum), .package (overridable), .serverUrl (null means the CLI default from endpoints.nix), .credentialsFile (path, e.g. an agenix-delivered file, linked to ~/.config/syndb/credentials.json; null means none), .mcp.enable, and an exposed .mcp.servers attrset (command, args, env) that harness modules (Claude Code, OpenCode, canix fleetix.mcp.servers) can consume. The review MCP’s SYNDB_REVIEW_SERVER_URL is set explicitly, never by default.
  • Shell completions (syndb completions <shell> exists) installed by the module.
  • A HM module check: an evaluation test (home-manager lib.homeManagerConfiguration per profile) asserting the right package and files, cheap and with no build.
  • Canix consumption: the pin needs every profile package prebuilt into the canix-fleet cache first. Uncached SynDB builds blocked the canix pin train for hours on 2026-11-15, and the operator then required caching first. The pin URL has no submodules=1 (vendor/queryfabric and vendor/topoxide are flake inputs; submodules break crane’s dummy-src) and no --follows. Canix already delivers ~/.config/syndb/credentials.json (canix home/user/can/repositories/syndb_owner_credentials.nix). Decide whether the module owns that link or canix keeps it.

4. Hard constraints

  • SynDB AGENTS.md: ETL logic in Rust; no compiler env overrides; versions in versions.nix; public URLs come only from endpoints.nix → shared::endpoints (generated, drift-tested); no hard-coded origins anywhere (a lint test enforces it).
  • The review MCP stays assessor-only with no default server (see crates/tools/cli/src/mcp/review.rs, documentation/src/reference/mcp.md).
  • Lazy senior dev: no abstractions beyond what’s needed; no new dependencies unless unavoidable.
  • Qualification: local and bounded (systemd-run --user -p MemoryMax=… --pipe --wait … </dev/null, from the main checkout’s direnv, with a private target dir); SynDB merges need sol-loop review, not hosted Actions. Nix package builds are heavy: say which builds each slice needs and keep them to the changed outputs.
  • Deploy: the 0.11.1 rollout freeze is Tue 2026-11-17 22:00Z for API-visible changes. This work is CLI and Nix only; say so, or flag anything API-visible.
  • Canix: never edit flake.nix/flake.lock by hand; pins go through the pin train (pin_request), and only after the prebuild into canix-fleet (canix cache binary push --cache canix-fleet). Canix Home Manager modules live under home/. Ask before any host switch.
  • A known red baseline: cargo build -p cli --no-default-features fails on trunk (~99 errors in k8s/etl_jobs/flight.rs etc., seen 2026-11-14). The restructure must make the consumer feature set build, and add a check that keeps it building.

5. Repo map (don’t re-survey beyond it; read the cited files)

PathWhat
SynDB crates/tools/cli/Cargo.toml [features]default=dev; prod=dataset+federation+etl-full; etl, graph-precompute, etl-full, etl-gpu, bench, ci, article-tools, mcp, dev=prod+article-tools, full=dev+bench+ci+mcp
SynDB crates/tools/cli/src/main.rs:17-64,120-271command enum and gates. Ungated today: Auth, Api, Query, Review, Article, Evidence, Test, Completions. Gated: Ops (etl-full), Dev, Data (dataset), Etl/Migrate (etl), Stack (dev), Federation, Bench, GraphPrecompute, Ci, Mcp
SynDB crates/tools/cli-common/auth, config, flight client, urls.rs (defaults from shared::endpoints)
SynDB nix/packages.nix:224-340depsLayerFor; syndb-cli (-p cli --no-default-features --features prod, :261); syndb-cli-tools (:272, full); syndb-export-provider (:284, Python CAVE export app); syndb-cli-etl-image, syndb-cli-bench-image; cross builds syndb-cli-<target> (:188-222)
SynDB nix/dev-shells.nixthe devshell builds syndb-cli-tools in the background
SynDB flake.nixflake-parts; perSystem at :299; no homeManagerModules or nixosModules yet
SynDB scripts/syndb-mcp.sh, .mcp.json, documentation/src/reference/mcp.md4 MCP modes (figures, check, ops, review) via syndb mcp <mode>; .#syndb-cli-tools provides them
SynDB crates/core/shared/src/endpoints.rs, endpoints.nixpublic origins
canix home/user/can/repositories/syndb_owner_credentials.nixagenix → ~/.config/syndb/credentials.json
canix docs/src/operations/workspace-update.md (pin train), docs/src/operations/attic-cache.md (canix-fleet push)the pin and cache procedure
canix lib/topology/ (Pkl) and fleetix_showwhich hosts are dev hosts

6. Facts

No scout: everything is in-repo. Earlier evidence: syndb-cli at SynDB c3d7e67f is in canix-fleet (/nix/store/0bvfbys0sgv7xm8frzsjwms97kagkm8f-syndb-cli-0.1.0); building it took about 12 minutes on atlas with the deps layer warm. syndb-cli-tools and syndb-export-provider were never pushed to canix-fleet.

7. References

None (no UI).