Dashboard Agent V1 — chat, reports, Investigate, Watch - #4418
Draft
kathiekiwi wants to merge 270 commits into
Draft
Dashboard Agent V1 — chat, reports, Investigate, Watch#4418kathiekiwi wants to merge 270 commits into
kathiekiwi wants to merge 270 commits into
Conversation
Queries using deltaSumTimestampMerge failed with an unknown function error, which broke the queue detail stats and the started counts on the built in Queues dashboard.
The queues list header tiles now render the same line chart, grid, and tooltip as the rest of the metrics charts instead of a row sparkline, with the headline value in the tile header. The env saturation tile draws the environment concurrency limit and burst limit as labeled reference lines. Chart tooltips keep a gap between the series label and the value, and the shared line chart gains showDots and referenceLines options.
Adds an Allocation tab to the Queues page (behind the queue metrics UI flag): overview cards, a burst-aware capacity bar showing each queue allocation and its live usage in a distinct color, an inline-editable limits table with per-queue locks, load-weighted auto-balance, and a review dialog that bulk-applies limits as overrides through the existing concurrency system. The queue list now defaults to Busiest ordering (with Backlog and Name options). ClickHouse ranks queues by activity over the last 15 minutes and returns just the requested page of names, so the cost per page is one small aggregate regardless of environment size; idle queues follow in name order and any failure falls back to name ordering. The classic page keeps plain name order.
The fallback WHERE injection only targeted the top-level SELECT, so a query shaped as an outer aggregation over a FROM subquery failed to compile: the time column only exists inside the subquery. Descend into the subquery so the fallback lands next to the table reference.
Adds two rollups fed from the raw landing table: a per-queue 5-minute tier and an environment-level 1-minute tier (gauges plus TDigest wait quantiles). Ranking now reads the 5m tier and returns the page and the ranked total in one windowed query instead of two scans. The 5m materialized view reads raw rather than cascading off the 10s table: deltaSumTimestamp states hold a single first/last segment, so merging states in an MV's hash-ordered GROUP BY double-counts bridging spans. For the same reason the env tier carries no counter columns, and env-wide counter totals must group by queue before summing.
The built-in queues dashboard's enqueued vs started chart merged counter states across queues, which mixes unrelated cumulative counters and returns wrong totals; it now merges per queue and sums outside. Env header tiles and saturation charts read the environment rollup, so their cost no longer scales with queue count, and coarse-bucket ranges are served from the 5m rollup automatically. Queue list ranking runs as one query, time bounds are aligned to the bucket grid, and repeated auto-refresh reads share ClickHouse query-cache entries.
… rollup The env rollup's win comes from dropping the queue dimension, not from coarser buckets: row count is queue-independent (~8640/day/env), so full 10-second granularity stays cheap at any range. Env header tiles and saturation charts now resolve short-range detail exactly like the per-queue charts, and the current-value tiles read the latest 10-second bucket instead of a minute-wide one.
The simulator's --reset only cleared the raw and 10s tables, leaving stale rows in the 5m and env rollups. It also force-merges the rollups after seeding so current-value widgets read cleanly.
Counter events now emit per queue and op odometer readings with a seeded zero baseline, matching the production emitter, so throughput and started counts reconstruct from simulated data instead of reading zero. Scenario switches prune the previous scenario's queues, a --project flag seeds each scenario into its own project for side-by-side design review, and a new many-queues scenario covers pagination and relevance ranking with one runaway queue, a busy head, a bursty middle, and a sparse tail. Adds --help.
A --usage flag stages plausible running counts in the local run-queue Redis for the seeded queues, so the list's Running column and the Allocation tab's usage bars have data without the run engine. Staged state is reconciled on every run: present with --usage, cleared without. Local Redis hosts only.
The tail query's exclusion list overwrote the search's name filter via object spread, so searching while sorted by activity showed unrelated queues past the ranked head. Combine the conditions with AND instead.
…ot ready Without a readiness guard, every fire-and-forget emit during a metrics Redis outage queued a command in ioredis's in-memory offline queue until rejection. Metrics are loss-tolerant by design, so drop instead; waitUntilReady() lets embedders await the initial connect.
The allocation view keeps manual limit edits, the review dialog, and bulk apply. The one-shot auto-balance button is removed (and the row locks whose only purpose was protecting queues from it); a policy-driven approach can replace it if rebalancing returns.
deltaSumTimestamp states are kept per queue, and merging them across queues silently returns wrong totals, on the dashboard and the public query API alike. Columns can now declare a mergeGroupKey, and the compiler rejects queries that merge such a column without grouping by that key or pinning it to a single value. The error names the column, explains the failure, and includes a corrected example query.
…e calls Short parameter lists on quantilesMerge and quantilesTDigestMerge do execute (the state layout is parameter independent, verified on both ClickHouse versions we run), but they rely on undocumented leniency and make the result-array indexes mean different quantiles per call site. Every merge now uses the stored four-quantile list with indexes re-pointed accordingly; returned values are unchanged.
The consumer retries a failed batch with the same insert deduplication token, and deduplicate_blocks_in_dependent_materialized_views re-runs the materialized views on a source-deduplicated insert, relying on each target table's dedup window to drop the duplicate. Only the raw table had one, so retries appended extra copies into every aggregate tier, silently inflating sums and quantiles. All three targets now set non_replicated_deduplication_window, with a regression test inserting the same batch three times.
… cluster slots, and stream caps Counter readings for names past the per-env cardinality cap are dropped instead of merging unrelated odometers under the overflow label (gauges still flow). The odometer key now shares the stream's shard hash tag so the INCR plus XADD script stays in one Cluster slot. The counter stream cap defaults lower when the stream shares the run-queue Redis. The per-bucket counter boundary undercount is documented on the delta columns.
…d live per-key breakdown CK queues now emit two extra gauge fields from the CK-path Lua scripts: the number of concurrency keys with queued runs (ZCARD of the ckIndex) and the head-of-line wait of the most-starved key (now minus the oldest ckIndex score). Both flow through the existing stream into new max-aggregated columns on the 10s and 5m tiers, and non-CK scripts keep the 7-field gauge shape. The queue detail page grows a concurrency-keys section for queues with CK activity: charts for backlogged keys and most-starved wait, plus a live per-key table (queued, running, oldest wait) read from the ckIndex zset, most starved first. Per-key history is intentionally not stored: key values are user-controlled, so the per-key dimension stays in Redis where it is bounded by the live backlog. The queue simulator gains a tenant-hotspot scenario that stages the CK gauge columns and a live ckIndex so the per-key table and charts render with data.
…trics history Queues that shard work with concurrencyKey get a per-key history tier. Counter events for CK runs advance a second per-key odometer and carry both readings on ONE stream entry (cum + ck/ckcum), so per-key attribution adds no stream volume; the consumer expands the entry into a base row and a per-key row. A new 10s AggregatingMergeTree tier keyed by (queue, concurrency_key) holds per-key enqueue/started/ack deltas, backlog/running maxes, and wait sums. Rows are activity-bound: a (queue, key, bucket) row exists only when that key had events, so user-controlled key cardinality cannot inflate the table (benchmarked at ~19 bytes per event with reads under 100ms in the worst shapes). The per-queue tiers stay exact: their counter and wait aggregations now consume only base rows, so per-key odometers never merge under one queue_name and waits are never double counted. A per-queue key limiter (default 10k) acts as a safety valve; on overflow the per-key row is dropped while the base row keeps per-queue counts exact. Per-key odometers use a short TTL, which cumulative counters make loss-free. The queue detail page gains a top-keys-by-backlog chart, a key table merging live state (queued, running, oldest wait) with range stats (started, peak backlog, mean delay), and click-through per-key drill-down charts. The new queue_metrics_by_key table is also queryable directly; its delta columns require grouping or pinning BOTH queue and concurrency_key, enforced by the compile-time merge guard which now supports compound keys.
…il page The queue detail page splits into Overview (the existing concurrency, backlog, delay, and throttle charts) and a Concurrency keys tab that holds all per-key content. The tab only appears for queues with key activity. Inside it, the grouped per-key charts now lead (backlog by key plus a new throughput-by-key chart that makes fair-share visible), followed by the two whole-queue health charts retitled to say what they aggregate (keys with queued runs count, most-starved key wait), then the key table and drill-down.
…ery surfaces TableSchema gains a hidden flag: hidden tables stay fully queryable through the engine (the concurrency-keys tab keeps working, tenancy and the merge guard still apply) but are excluded from the query editor autocomplete, the Query page schema and examples panels, the AI query generator context, and the schema API. queue_metrics_by_key is hidden until the per-key surface is settled.
…n re-enqueue The wait metric measured dequeue time minus the original trigger time even when a run re-entered the queue after a waitpoint, checkpoint, or pending version, so the whole wait or checkpoint duration showed up as scheduling delay. Re-enqueues now anchor to the re-enqueue time while first enqueues keep the trigger or delay anchor; queue ordering is unchanged, so re-enqueued runs keep their original position. Nacked runs never left the queue stint and keep the original anchor. Also replaces the :ck: suffix regexes on user-controlled queue names with indexOf slicing (identical semantics) to remove a polynomial regex flagged by code scanning.
Behind the per-org queue metrics UI flag: the task detail page links its queue to the queue detail view and shows live queued/running counts, delay p95, peak backlog, and a queue backlog chart; the run inspector links the queue and concurrency key, and queued, delayed, and pending-version runs get a "Waiting in queue" block with an at-limit callout and per-key counts. Adds optional concurrency-key params to the run engine queue reads.
…ask detail queue chart The run inspector's "Waiting in queue" section now leads with two borderless stat + last-30-minutes sparklines (backlog and scheduling delay), matching the inspector's divider-and-rows layout instead of nested cards. On the task detail page the queue backlog moves behind a tab in the activity card rather than sitting beside the runs-by-status chart, so it does not crowd the agent-task hero charts.
…ersion collision Main added 035_fix_error_display_derivation.sql, which collides with the queue-metrics migration that also landed as 035. goose versions by numeric prefix, so two 035 files break the migrator. Renumber the queue-metrics migration to 036 (it sorts after the error-display fix on a fresh database).
## Queue pages design overhaul (frontend for the queue-metrics feature) Builds the dashboards on top of the queue-metrics pipeline (base branch). ### Queues list <img width="1440" height="788" alt="Screenshot 2026-07-20 at 01 02 55" src="https://github.com/user-attachments/assets/0b97dd6c-d05b-4dba-99c9-04266ec22cb3" /> ### Queue detail <img width="1440" height="1272" alt="tab-overview" src="https://github.com/user-attachments/assets/22b33f82-522d-4410-a477-fe5d2c2f7adf" /> <img width="1440" height="1532" alt="tab-keys" src="https://github.com/user-attachments/assets/b05619fa-93a8-43ef-8838-a6d7fc5d789b" /> ### Under the hood - Percent-based concurrency overrides (schema + migration + recalc on env limit change, capped at the env limit) — covered by unit tests - All recurring refresh hits ClickHouse only (15s blocks / 60s charts, paused in hidden tabs); Redis/PG serve first paint - Added `MetricsLayout` that is currently used for `/queues + $id`, `/agents + $id`, `/settings/usage` <img width="500" height="auto" alt="Screenshot 2026-07-20 at 01 07 31" src="https://github.com/user-attachments/assets/e8a798e7-deee-455d-be96-414f7d09a06c" />
A chart block's TRQL query used to run only in the panel, after the turn, so a bad query left a broken chart the model never learned about. render_view now runs each chart query through the query API first and fails by name with the query error, so the model fixes it in the same turn. The rows are discarded — the panel stays the runner. Skipped when the turn has no delegated token or the validation request itself fails.
Markdown renderers won't link an unknown scheme, so a cited trigger:// target rendered dead. Prose links now rewrite through the panel's resolver; while unresolved they degrade to their plain label.
…s at something worth monitoring
…headline is an unresolved recurring error
Contributor
Observability mapAs of 20/100 over 422 measured of 437 entry points (base 18, up 2) What this PR changed FIX FIRST
AUDIT 3 of 49 sensitive mutations record an actor. 46 without one. What the score is made ofReport only, nothing here gates the merge. The rules and their reasons: internal-packages/observability-map/README.md. |
The Badge primitive's small variant paints a blue tinted chip on system themes, which overrode the severity/confidence tones — Degraded and Medium confidence rendered blue instead of amber.
New "actions" view block: a row of 1-3 buttons the model may emit. A watch action opens the watch configuration card pre-filled; ask sends the labelled question as the user's next message; a navigate target that doesn't parse is dropped at render time, as on chart actions.
…ger button Agent logo instead of the indigo bubble, the button's own surface (charcoal on dark, white on light) and softened green border.
Third ToastUI variant: success's layout with the agent glyph and the Ask Trigger border. The wake toast drops its Callout composition for it.
…allery section id The wake toast moved to the standard toast's agent status, leaving the Callout variant with no consumer.
The root already mounts one; a fired toast rendered in both and the two copies stacked.
Sonner stamps data-theme="light" (its default) on the toast list; since the theme system remaps tokens by that attribute, every custom toast rendered light regardless of the page's theme.
…ays the side panel
…ast surface matches Ask Trigger on dark themes A wake read on screen before the next poll never toasted. The toast list is now recent deliveries (15 min, id-deduped client-side); the dot still counts unread only.
The wake seeded the card and said it had started looking, then nothing ran it — the findings were left to a turn only the user could start. The watcher now reports a delivered consented wake to the webapp, which mints the same delegated user-actor token a turn gets and sends a `watch.investigate` action into the chat; the agent conducts a real investigating turn on that card and delivers the findings as its own message. Best-effort throughout: nothing here can retry or invalidate the wake.
Chats belong to (organization, user); several queries enforced only the user, so a user's own chat from another org could be opened, renamed, pinned or appended to through a different org's route.
Crossing orgs re-renders the layout without remounting, so the previous org's open chat and history lingered in the panel.
…ates The card catalogs move out of Chat UI onto their own storybook pages (view blocks, report view, investigation card, watch card), all sharing the manifest and the page shell. Cuts near-duplicate states: 97 sections down to 52, section ids unchanged. The screenshot script now walks every page.
…1095) The env JWT exchange now stamps a signed `act` claim (acting user + client kind), auth surfaces it as `actor`, and the tenant context prefers it over `orgMember`, which only exists on dev environments. Identity only — the JWT still authorizes as the environment.
The toast source is recent deliveries, so an in-memory dedupe re-toasted every wake younger than the window on every refresh.
…aths The routes this branch adds logged their failures without saying whose request they were, and several catch clauses answered every error the same way. - every failure-path log now carries the tenant ids the route has in hand (userId / organizationId / projectId / environmentId) - the body-parse try blocks guard only the parse; the shape check moved out - the boundary catches rethrow a thrown Response instead of turning it into a 500 - routes that relied on the central handler get a log-and-rethrow boundary, so the response is unchanged and the failure is named - the advisory watch email-alert state moved to the alerts service, where the rest of that logic lives
The heartbeat kept a review stand from ageing by appending a minute of fresh telemetry every 30s. It was a crutch for demos, not something a developer needs, and it carried a state file, a tag-and-prune cycle and a per-tick Redis top-up with it. All of it goes; --degrade / --recover / --showcase and the base seed stay. The Redis key shapes and the depth staging move to seed-agent-examples-redis.mts so the scenario kit can stage the same keys the same way.
Every Watch condition was proven by hand-run Redis and ClickHouse surgery. Now each one is a verb: pnpm --filter webapp run scenarios:watch -- queue:fill email-sends 400 pnpm --filter webapp run scenarios:watch -- queue:drain email-sends pnpm --filter webapp run scenarios:watch -- error:recur pnpm --filter webapp run scenarios:watch -- run:fail 90 pnpm --filter webapp run scenarios:watch -- health:degrade | health:recover Each verb is idempotent, runs on top of the seeded agent-examples stand, refuses a non-local Redis or ClickHouse, and prints the dashboard step that follows it. Prerequisites fail with the command to run. SCENARIOS.md walks every scenario end to end — the command, the clicks, the wording that arrives and the tick cadence — including the two run tasks, whose source lives in the references repo and is carried here as a snippet.
…es and the recent-wakes toast feed Also drops the tracked review-diff artifact from the repo root.
Same boundary rule as the membership read above it: replica lag must not extend access a background check should have revoked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An AI assistant in a side panel on every dashboard page, behind the dashboard-agent feature flag. It reads runs, errors, queues, deploys and health through the public API (read-only, delegated user token), answers with rich cards, and can keep watching things after the conversation ends.
What's inside
@internal/dashboard-agent-contracts(trigger:// URI grammar, intents, watch specs, block envelope), investigations + watches tables, head-start reliability fix, eval sample-rate gate.get_reportrenders the deterministic health report as a card (metric grid, sparklines, Next steps button row); stale telemetry is flagged and never trusted for advice.db:seed:agent-examples, with--heartbeat/--degrade/--recoverfor demos).How to review
GUIDEBOOK.md — 10-minute local setup and a hands-on walkthrough of every case. Component gallery at
/storybook/agent-ui.Notes
canAccessDashboardAgent; no behavior change with the flag off.--heartbeatmode is a review-stand crutch and will be removed before merge.