change capture

This commit is contained in:
2026-02-26 14:34:02 -06:00
parent 7ee3604eb1
commit b43495b26d
47 changed files with 5785 additions and 1525 deletions

View File

@@ -1,7 +1,7 @@
# RabbitMQ Queue Bindings - Quick Reference
**Last Updated:** 2026-02-03
**Related Fix:** Queue Separation for InquiryHandler, CompletionListener, and ExecutionManager
**Last Updated:** 2026-02-26
**Related Fix:** Executor events queue separation (event.created only)
## Overview
@@ -21,7 +21,14 @@ Attune uses three main exchanges:
| Queue | Routing Key | Message Type | Consumer |
|-------|-------------|--------------|----------|
| `attune.events.queue` | `#` (all) | `EventCreatedPayload` | EventProcessor (executor) |
| `attune.events.queue` | `#` (all) | All event types | Sensor service (rule lifecycle) |
| `attune.executor.events.queue` | `event.created` | `EventCreatedPayload` | EventProcessor (executor) |
| `attune.rules.lifecycle.queue` | `rule.created`, `rule.enabled`, `rule.disabled` | `RuleCreated/Enabled/DisabledPayload` | RuleLifecycleListener (sensor) |
| `worker.{id}.packs` | `pack.registered` | `PackRegisteredPayload` | Worker (per-instance) |
> **Note:** The sensor's `attune.events.queue` is bound with `#` (all routing keys) for catch-all
> event monitoring. The executor uses a dedicated `attune.executor.events.queue` bound only to
> `event.created` to avoid deserializing unrelated message types (rule lifecycle, pack registration).
### Executions Exchange (`attune.executions`)

View File

@@ -46,19 +46,30 @@ Each service declares only the queues it consumes:
**Role:** Orchestrates execution lifecycle, enforces rules, manages inquiries
**Queues Owned:**
- `attune.executor.events.queue`
- Exchange: `attune.events`
- Routing: `event.created`
- Purpose: Sensor-generated events for rule evaluation
- Note: Dedicated queue so the executor only receives `EventCreatedPayload` messages,
not rule lifecycle or pack registration messages that also flow through `attune.events`
- `attune.enforcements.queue`
- Exchange: `attune.executions`
- Routing: `enforcement.#`
- Purpose: Rule enforcement requests
- `attune.execution.requests.queue`
- Exchange: `attune.executions`
- Routing: `execution.requested`
- Purpose: New execution requests
- `attune.execution.status.queue`
- Exchange: `attune.executions`
- Routing: `execution.status.changed`
- Purpose: Execution status updates from workers
- `attune.execution.completed.queue`
- Exchange: `attune.executions`
- Routing: `execution.completed`
- Purpose: Completed execution results
- `attune.inquiry.responses.queue`
- Exchange: `attune.executions`
- Routing: `inquiry.responded`
- Purpose: Human-in-the-loop responses
@@ -92,8 +103,16 @@ Each service declares only the queues it consumes:
**Queues Owned:**
- `attune.events.queue`
- Exchange: `attune.events`
- Routing: `#` (all events)
- Purpose: Events generated by sensors and triggers
- Purpose: Catch-all queue for sensor event monitoring
- Note: Bound with `#` to receive all message types on the events exchange.
The sensor service itself uses `attune.rules.lifecycle.queue` for rule changes
(see RuleLifecycleListener). This queue exists for general event monitoring.
- `attune.rules.lifecycle.queue`
- Exchange: `attune.events`
- Routing: `rule.created`, `rule.enabled`, `rule.disabled`
- Purpose: Rule lifecycle events for starting/stopping sensors
**Setup Method:** `Connection::setup_sensor_infrastructure()`
@@ -147,11 +166,11 @@ Exception:
### Rule Enforcement Flow
```
Event Created
→ `attune.events` exchange
→ `attune.events.queue` (consumed by Executor)
→ `attune.events` exchange (routing: event.created)
→ `attune.executor.events.queue` (consumed by Executor EventProcessor)
→ Rule evaluation
→ `enforcement.created` published to `attune.executions`
→ `attune.enforcements.queue` (consumed by Executor)
→ `attune.enforcements.queue` (consumed by Executor EnforcementProcessor)
```
### Execution Flow
@@ -241,7 +260,8 @@ Access at `http://localhost:15672` (credentials: `guest`/`guest`)
**Expected Queues:**
- `attune.dlx.queue` - Dead letter queue
- `attune.events.queue` - Events (Sensor)
- `attune.events.queue` - Events catch-all (Sensor)
- `attune.executor.events.queue` - Event created only (Executor)
- `attune.enforcements.queue` - Enforcements (Executor)
- `attune.execution.requests.queue` - Execution requests (Executor)
- `attune.execution.status.queue` - Status updates (Executor)

View File

@@ -209,24 +209,42 @@ No special SQLx support is needed. History tables are standard PostgreSQL tables
## Implementation Scope
### Phase 1 (this migration)
- [ ] `CREATE EXTENSION IF NOT EXISTS timescaledb`
- [ ] Create four `<entity>_history` tables
- [ ] Convert to hypertables with `create_hypertable()`
- [ ] Create indexes (entity lookup, status change filter, GIN on changed_fields, ref lookup)
- [ ] Create trigger functions for `execution`, `worker`, `enforcement`, `event`
- [ ] Attach triggers to operational tables
- [ ] Configure compression policies
- [ ] Configure retention policies
### Phase 1 (migration)
- [x] `CREATE EXTENSION IF NOT EXISTS timescaledb`
- [x] Create four `<entity>_history` tables
- [x] Convert to hypertables with `create_hypertable()`
- [x] Create indexes (entity lookup, status change filter, GIN on changed_fields, ref lookup)
- [x] Create trigger functions for `execution`, `worker`, `enforcement`, `event`
- [x] Attach triggers to operational tables
- [x] Configure compression policies
- [x] Configure retention policies
### Phase 2 (future — API & UI)
- [ ] History repository in `crates/common/src/repositories/`
- [ ] API endpoints (e.g., `GET /api/v1/executions/:id/history`)
- [ ] Web UI history panel on entity detail pages
- [ ] Continuous aggregates for dashboards
### Phase 2 (API & UI)
- [x] History model in `crates/common/src/models.rs` (`EntityHistoryRecord`, `HistoryEntityType`)
- [x] History repository in `crates/common/src/repositories/entity_history.rs` (`query`, `count`, `find_by_entity_id`, `find_status_changes`, `find_latest`)
- [x] History DTOs in `crates/api/src/dto/history.rs` (`HistoryRecordResponse`, `HistoryQueryParams`)
- [x] API endpoints in `crates/api/src/routes/history.rs`:
- `GET /api/v1/history/{entity_type}` — generic history query with filters & pagination
- `GET /api/v1/executions/{id}/history` — execution-specific history
- `GET /api/v1/workers/{id}/history` — worker-specific history
- `GET /api/v1/enforcements/{id}/history` — enforcement-specific history
- `GET /api/v1/events/{id}/history` — event-specific history
- [x] Web UI history panel on entity detail pages
- `web/src/hooks/useHistory.ts` — React Query hooks (`useEntityHistory`, `useExecutionHistory`, `useWorkerHistory`, `useEnforcementHistory`, `useEventHistory`)
- `web/src/components/common/EntityHistoryPanel.tsx` — Reusable collapsible panel with timeline, field-level diffs, filters (operation, changed_field), and pagination
- Integrated into `ExecutionDetailPage`, `EnforcementDetailPage`, `EventDetailPage` (worker detail page does not exist yet)
- [x] Continuous aggregates for dashboards
- Migration `20260226200000_continuous_aggregates.sql` creates 5 continuous aggregates: `execution_status_hourly`, `execution_throughput_hourly`, `event_volume_hourly`, `worker_status_hourly`, `enforcement_volume_hourly`
- Auto-refresh policies (30 min for most, 1 hour for worker) with 7-day lookback
### Phase 3 (future — analytics)
- [ ] Dashboard widgets showing execution throughput, failure rates, worker health trends
### Phase 3 (analytics)
- [x] Dashboard widgets showing execution throughput, failure rates, worker health trends
- `crates/common/src/repositories/analytics.rs` — repository querying continuous aggregates (execution status/throughput, event volume, worker status, enforcement volume, failure rate)
- `crates/api/src/dto/analytics.rs` — DTOs (`DashboardAnalyticsResponse`, `TimeSeriesPoint`, `FailureRateResponse`, `AnalyticsQueryParams`, etc.)
- `crates/api/src/routes/analytics.rs` — 7 API endpoints under `/api/v1/analytics/` (dashboard, executions/status, executions/throughput, executions/failure-rate, events/volume, workers/status, enforcements/volume)
- `web/src/hooks/useAnalytics.ts` — React Query hooks (`useDashboardAnalytics`, `useExecutionStatusAnalytics`, `useFailureRateAnalytics`, etc.)
- `web/src/components/common/AnalyticsWidgets.tsx` — Dashboard visualization components (MiniBarChart, StackedBarChart, FailureRateCard with SVG ring gauge, StatCard, TimeRangeSelector with 6h/12h/24h/2d/7d presets)
- Integrated into `DashboardPage.tsx` below existing metrics and activity sections
- [ ] Configurable retention periods via admin settings
- [ ] Export/archival to external storage before retention expiry