change capture
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
# RabbitMQ Queue Bindings - Quick Reference
|
||||
|
||||
**Last Updated:** 2026-02-03
|
||||
**Related Fix:** Queue Separation for InquiryHandler, CompletionListener, and ExecutionManager
|
||||
**Last Updated:** 2026-02-26
|
||||
**Related Fix:** Executor events queue separation (event.created only)
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -21,7 +21,14 @@ Attune uses three main exchanges:
|
||||
|
||||
| Queue | Routing Key | Message Type | Consumer |
|
||||
|-------|-------------|--------------|----------|
|
||||
| `attune.events.queue` | `#` (all) | `EventCreatedPayload` | EventProcessor (executor) |
|
||||
| `attune.events.queue` | `#` (all) | All event types | Sensor service (rule lifecycle) |
|
||||
| `attune.executor.events.queue` | `event.created` | `EventCreatedPayload` | EventProcessor (executor) |
|
||||
| `attune.rules.lifecycle.queue` | `rule.created`, `rule.enabled`, `rule.disabled` | `RuleCreated/Enabled/DisabledPayload` | RuleLifecycleListener (sensor) |
|
||||
| `worker.{id}.packs` | `pack.registered` | `PackRegisteredPayload` | Worker (per-instance) |
|
||||
|
||||
> **Note:** The sensor's `attune.events.queue` is bound with `#` (all routing keys) for catch-all
|
||||
> event monitoring. The executor uses a dedicated `attune.executor.events.queue` bound only to
|
||||
> `event.created` to avoid deserializing unrelated message types (rule lifecycle, pack registration).
|
||||
|
||||
### Executions Exchange (`attune.executions`)
|
||||
|
||||
|
||||
@@ -46,19 +46,30 @@ Each service declares only the queues it consumes:
|
||||
**Role:** Orchestrates execution lifecycle, enforces rules, manages inquiries
|
||||
|
||||
**Queues Owned:**
|
||||
- `attune.executor.events.queue`
|
||||
- Exchange: `attune.events`
|
||||
- Routing: `event.created`
|
||||
- Purpose: Sensor-generated events for rule evaluation
|
||||
- Note: Dedicated queue so the executor only receives `EventCreatedPayload` messages,
|
||||
not rule lifecycle or pack registration messages that also flow through `attune.events`
|
||||
- `attune.enforcements.queue`
|
||||
- Exchange: `attune.executions`
|
||||
- Routing: `enforcement.#`
|
||||
- Purpose: Rule enforcement requests
|
||||
- `attune.execution.requests.queue`
|
||||
- Exchange: `attune.executions`
|
||||
- Routing: `execution.requested`
|
||||
- Purpose: New execution requests
|
||||
- `attune.execution.status.queue`
|
||||
- Exchange: `attune.executions`
|
||||
- Routing: `execution.status.changed`
|
||||
- Purpose: Execution status updates from workers
|
||||
- `attune.execution.completed.queue`
|
||||
- Exchange: `attune.executions`
|
||||
- Routing: `execution.completed`
|
||||
- Purpose: Completed execution results
|
||||
- `attune.inquiry.responses.queue`
|
||||
- Exchange: `attune.executions`
|
||||
- Routing: `inquiry.responded`
|
||||
- Purpose: Human-in-the-loop responses
|
||||
|
||||
@@ -92,8 +103,16 @@ Each service declares only the queues it consumes:
|
||||
|
||||
**Queues Owned:**
|
||||
- `attune.events.queue`
|
||||
- Exchange: `attune.events`
|
||||
- Routing: `#` (all events)
|
||||
- Purpose: Events generated by sensors and triggers
|
||||
- Purpose: Catch-all queue for sensor event monitoring
|
||||
- Note: Bound with `#` to receive all message types on the events exchange.
|
||||
The sensor service itself uses `attune.rules.lifecycle.queue` for rule changes
|
||||
(see RuleLifecycleListener). This queue exists for general event monitoring.
|
||||
- `attune.rules.lifecycle.queue`
|
||||
- Exchange: `attune.events`
|
||||
- Routing: `rule.created`, `rule.enabled`, `rule.disabled`
|
||||
- Purpose: Rule lifecycle events for starting/stopping sensors
|
||||
|
||||
**Setup Method:** `Connection::setup_sensor_infrastructure()`
|
||||
|
||||
@@ -147,11 +166,11 @@ Exception:
|
||||
### Rule Enforcement Flow
|
||||
```
|
||||
Event Created
|
||||
→ `attune.events` exchange
|
||||
→ `attune.events.queue` (consumed by Executor)
|
||||
→ `attune.events` exchange (routing: event.created)
|
||||
→ `attune.executor.events.queue` (consumed by Executor EventProcessor)
|
||||
→ Rule evaluation
|
||||
→ `enforcement.created` published to `attune.executions`
|
||||
→ `attune.enforcements.queue` (consumed by Executor)
|
||||
→ `attune.enforcements.queue` (consumed by Executor EnforcementProcessor)
|
||||
```
|
||||
|
||||
### Execution Flow
|
||||
@@ -241,7 +260,8 @@ Access at `http://localhost:15672` (credentials: `guest`/`guest`)
|
||||
|
||||
**Expected Queues:**
|
||||
- `attune.dlx.queue` - Dead letter queue
|
||||
- `attune.events.queue` - Events (Sensor)
|
||||
- `attune.events.queue` - Events catch-all (Sensor)
|
||||
- `attune.executor.events.queue` - Event created only (Executor)
|
||||
- `attune.enforcements.queue` - Enforcements (Executor)
|
||||
- `attune.execution.requests.queue` - Execution requests (Executor)
|
||||
- `attune.execution.status.queue` - Status updates (Executor)
|
||||
|
||||
@@ -209,24 +209,42 @@ No special SQLx support is needed. History tables are standard PostgreSQL tables
|
||||
|
||||
## Implementation Scope
|
||||
|
||||
### Phase 1 (this migration)
|
||||
- [ ] `CREATE EXTENSION IF NOT EXISTS timescaledb`
|
||||
- [ ] Create four `<entity>_history` tables
|
||||
- [ ] Convert to hypertables with `create_hypertable()`
|
||||
- [ ] Create indexes (entity lookup, status change filter, GIN on changed_fields, ref lookup)
|
||||
- [ ] Create trigger functions for `execution`, `worker`, `enforcement`, `event`
|
||||
- [ ] Attach triggers to operational tables
|
||||
- [ ] Configure compression policies
|
||||
- [ ] Configure retention policies
|
||||
### Phase 1 (migration) ✅
|
||||
- [x] `CREATE EXTENSION IF NOT EXISTS timescaledb`
|
||||
- [x] Create four `<entity>_history` tables
|
||||
- [x] Convert to hypertables with `create_hypertable()`
|
||||
- [x] Create indexes (entity lookup, status change filter, GIN on changed_fields, ref lookup)
|
||||
- [x] Create trigger functions for `execution`, `worker`, `enforcement`, `event`
|
||||
- [x] Attach triggers to operational tables
|
||||
- [x] Configure compression policies
|
||||
- [x] Configure retention policies
|
||||
|
||||
### Phase 2 (future — API & UI)
|
||||
- [ ] History repository in `crates/common/src/repositories/`
|
||||
- [ ] API endpoints (e.g., `GET /api/v1/executions/:id/history`)
|
||||
- [ ] Web UI history panel on entity detail pages
|
||||
- [ ] Continuous aggregates for dashboards
|
||||
### Phase 2 (API & UI) ✅
|
||||
- [x] History model in `crates/common/src/models.rs` (`EntityHistoryRecord`, `HistoryEntityType`)
|
||||
- [x] History repository in `crates/common/src/repositories/entity_history.rs` (`query`, `count`, `find_by_entity_id`, `find_status_changes`, `find_latest`)
|
||||
- [x] History DTOs in `crates/api/src/dto/history.rs` (`HistoryRecordResponse`, `HistoryQueryParams`)
|
||||
- [x] API endpoints in `crates/api/src/routes/history.rs`:
|
||||
- `GET /api/v1/history/{entity_type}` — generic history query with filters & pagination
|
||||
- `GET /api/v1/executions/{id}/history` — execution-specific history
|
||||
- `GET /api/v1/workers/{id}/history` — worker-specific history
|
||||
- `GET /api/v1/enforcements/{id}/history` — enforcement-specific history
|
||||
- `GET /api/v1/events/{id}/history` — event-specific history
|
||||
- [x] Web UI history panel on entity detail pages
|
||||
- `web/src/hooks/useHistory.ts` — React Query hooks (`useEntityHistory`, `useExecutionHistory`, `useWorkerHistory`, `useEnforcementHistory`, `useEventHistory`)
|
||||
- `web/src/components/common/EntityHistoryPanel.tsx` — Reusable collapsible panel with timeline, field-level diffs, filters (operation, changed_field), and pagination
|
||||
- Integrated into `ExecutionDetailPage`, `EnforcementDetailPage`, `EventDetailPage` (worker detail page does not exist yet)
|
||||
- [x] Continuous aggregates for dashboards
|
||||
- Migration `20260226200000_continuous_aggregates.sql` creates 5 continuous aggregates: `execution_status_hourly`, `execution_throughput_hourly`, `event_volume_hourly`, `worker_status_hourly`, `enforcement_volume_hourly`
|
||||
- Auto-refresh policies (30 min for most, 1 hour for worker) with 7-day lookback
|
||||
|
||||
### Phase 3 (future — analytics)
|
||||
- [ ] Dashboard widgets showing execution throughput, failure rates, worker health trends
|
||||
### Phase 3 (analytics) ✅
|
||||
- [x] Dashboard widgets showing execution throughput, failure rates, worker health trends
|
||||
- `crates/common/src/repositories/analytics.rs` — repository querying continuous aggregates (execution status/throughput, event volume, worker status, enforcement volume, failure rate)
|
||||
- `crates/api/src/dto/analytics.rs` — DTOs (`DashboardAnalyticsResponse`, `TimeSeriesPoint`, `FailureRateResponse`, `AnalyticsQueryParams`, etc.)
|
||||
- `crates/api/src/routes/analytics.rs` — 7 API endpoints under `/api/v1/analytics/` (dashboard, executions/status, executions/throughput, executions/failure-rate, events/volume, workers/status, enforcements/volume)
|
||||
- `web/src/hooks/useAnalytics.ts` — React Query hooks (`useDashboardAnalytics`, `useExecutionStatusAnalytics`, `useFailureRateAnalytics`, etc.)
|
||||
- `web/src/components/common/AnalyticsWidgets.tsx` — Dashboard visualization components (MiniBarChart, StackedBarChart, FailureRateCard with SVG ring gauge, StatCard, TimeRangeSelector with 6h/12h/24h/2d/7d presets)
|
||||
- Integrated into `DashboardPage.tsx` below existing metrics and activity sections
|
||||
- [ ] Configurable retention periods via admin settings
|
||||
- [ ] Export/archival to external storage before retention expiry
|
||||
|
||||
|
||||
Reference in New Issue
Block a user