attune-system/attune
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

re-uploading work

Browse Source
This commit is contained in:
David Culbreth
2026-02-04 17:46:30 -06:00
commit 3b14c65998
1388 changed files with 381262 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

598
docs/api/api-actions.md Normal file
View File

@@ -0,0 +1,598 @@
# Action Management API
This document describes the Action Management API endpoints for the Attune automation platform.
## Overview
Actions are the executable units in Attune that perform specific tasks. Each action belongs to a pack and can have parameters, output schemas, and runtime requirements.
**Base Path:** `/api/v1/actions`
## Data Model
### Action
```json
{
"id": 1,
"ref": "core.http.get",
"pack": 1,
"pack_ref": "core",
"label": "HTTP GET Request",
"description": "Performs an HTTP GET request to a specified URL",
"entrypoint": "/actions/http_get.py",
"runtime": 1,
"param_schema": {
"type": "object",
"properties": {
"url": { "type": "string" },
"headers": { "type": "object" }
},
"required": ["url"]
},
"out_schema": {
"type": "object",
"properties": {
"status_code": { "type": "integer" },
"body": { "type": "string" }
}
},
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
```
### Action Summary (List View)
```json
{
"id": 1,
"ref": "core.http.get",
"pack_ref": "core",
"label": "HTTP GET Request",
"description": "Performs an HTTP GET request to a specified URL",
"entrypoint": "/actions/http_get.py",
"runtime": 1,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
```
## Endpoints
### List All Actions
Retrieve a paginated list of all actions.
**Endpoint:** `GET /api/v1/actions`
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
```json
{
"data": [
{
"id": 1,
"ref": "core.http.get",
"pack_ref": "core",
"label": "HTTP GET Request",
"description": "Performs an HTTP GET request",
"entrypoint": "/actions/http_get.py",
"runtime": 1,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
],
"pagination": {
"page": 1,
"per_page": 20,
"total": 1,
"total_pages": 1
}
}
```
---
### List Actions by Pack
Retrieve all actions belonging to a specific pack.
**Endpoint:** `GET /api/v1/packs/:pack_ref/actions`
**Path Parameters:**
- `pack_ref` (string): Pack reference identifier
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
```json
{
"data": [
{
"id": 1,
"ref": "core.http.get",
"pack_ref": "core",
"label": "HTTP GET Request",
"description": "Performs an HTTP GET request",
"entrypoint": "/actions/http_get.py",
"runtime": 1,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
],
"pagination": {
"page": 1,
"per_page": 20,
"total": 1,
"total_pages": 1
}
}
```
**Errors:**
- `404 Not Found`: Pack with the specified ref does not exist
---
### Get Action by Reference
Retrieve a single action by its reference identifier.
**Endpoint:** `GET /api/v1/actions/:ref`
**Path Parameters:**
- `ref` (string): Action reference identifier (e.g., "core.http.get")
**Response:** `200 OK`
```json
{
"data": {
"id": 1,
"ref": "core.http.get",
"pack": 1,
"pack_ref": "core",
"label": "HTTP GET Request",
"description": "Performs an HTTP GET request to a specified URL",
"entrypoint": "/actions/http_get.py",
"runtime": 1,
"param_schema": { ... },
"out_schema": { ... },
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
}
```
**Errors:**
- `404 Not Found`: Action with the specified ref does not exist
---
### Create Action
Create a new action in the system.
**Endpoint:** `POST /api/v1/actions`
**Request Body:**
```json
{
"ref": "core.http.get",
"pack_ref": "core",
"label": "HTTP GET Request",
"description": "Performs an HTTP GET request to a specified URL",
"entrypoint": "/actions/http_get.py",
"runtime": 1,
"param_schema": {
"type": "object",
"properties": {
"url": { "type": "string" },
"headers": { "type": "object" }
},
"required": ["url"]
},
"out_schema": {
"type": "object",
"properties": {
"status_code": { "type": "integer" },
"body": { "type": "string" }
}
}
}
```
**Required Fields:**
- `ref`: Unique reference identifier (alphanumeric, dots, underscores, hyphens)
- `pack_ref`: Reference to the parent pack
- `label`: Human-readable name (1-255 characters)
- `description`: Action description (min 1 character)
- `entrypoint`: Execution entry point (1-1024 characters)
**Optional Fields:**
- `runtime`: Runtime ID for execution environment
- `param_schema`: JSON Schema defining input parameters
- `out_schema`: JSON Schema defining expected outputs
**Response:** `201 Created`
```json
{
"data": {
"id": 1,
"ref": "core.http.get",
"pack": 1,
"pack_ref": "core",
"label": "HTTP GET Request",
"description": "Performs an HTTP GET request to a specified URL",
"entrypoint": "/actions/http_get.py",
"runtime": 1,
"param_schema": { ... },
"out_schema": { ... },
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
},
"message": "Action created successfully"
}
```
**Errors:**
- `400 Bad Request`: Invalid request data or validation failure
- `404 Not Found`: Referenced pack does not exist
- `409 Conflict`: Action with the same ref already exists
---
### Update Action
Update an existing action's properties.
**Endpoint:** `PUT /api/v1/actions/:ref`
**Path Parameters:**
- `ref` (string): Action reference identifier
**Request Body:**
All fields are optional. Only provided fields will be updated.
```json
{
"label": "Updated HTTP GET Request",
"description": "Updated description",
"entrypoint": "/actions/http_get_v2.py",
"runtime": 2,
"param_schema": { ... },
"out_schema": { ... }
}
```
**Response:** `200 OK`
```json
{
"data": {
"id": 1,
"ref": "core.http.get",
"pack": 1,
"pack_ref": "core",
"label": "Updated HTTP GET Request",
"description": "Updated description",
"entrypoint": "/actions/http_get_v2.py",
"runtime": 2,
"param_schema": { ... },
"out_schema": { ... },
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T12:00:00Z"
},
"message": "Action updated successfully"
}
```
**Errors:**
- `400 Bad Request`: Invalid request data or validation failure
- `404 Not Found`: Action with the specified ref does not exist
---
### Delete Action
Delete an action from the system.
**Endpoint:** `DELETE /api/v1/actions/:ref`
**Path Parameters:**
- `ref` (string): Action reference identifier
**Response:** `200 OK`
```json
{
"success": true,
"message": "Action 'core.http.get' deleted successfully"
}
```
**Errors:**
- `404 Not Found`: Action with the specified ref does not exist
---
### Get Queue Statistics
Retrieve real-time queue statistics for an action's execution queue.
**Endpoint:** `GET /api/v1/actions/:ref/queue-stats`
**Path Parameters:**
- `ref` (string): Action reference identifier
**Response:** `200 OK`
```json
{
"data": {
"action_id": 1,
"action_ref": "core.http.get",
"queue_length": 5,
"active_count": 2,
"max_concurrent": 3,
"oldest_enqueued_at": "2025-01-27T10:30:00Z",
"total_enqueued": 1250,
"total_completed": 1245,
"last_updated": "2025-01-27T12:45:30Z"
}
}
```
**Response Fields:**
- `action_id`: Numeric action ID
- `action_ref`: Action reference identifier
- `queue_length`: Number of executions waiting in queue
- `active_count`: Number of currently running executions
- `max_concurrent`: Maximum concurrent executions allowed (from policy)
- `oldest_enqueued_at`: Timestamp of oldest queued execution (null if queue empty)
- `total_enqueued`: Lifetime count of executions enqueued
- `total_completed`: Lifetime count of executions completed
- `last_updated`: Last time statistics were updated
**Response When No Queue Stats Available:** `200 OK`
```json
{
"data": {
"action_id": 1,
"action_ref": "core.http.get",
"queue_length": 0,
"active_count": 0,
"max_concurrent": null,
"oldest_enqueued_at": null,
"total_enqueued": 0,
"total_completed": 0,
"last_updated": null
}
}
```
**Errors:**
- `404 Not Found`: Action with the specified ref does not exist
**Use Cases:**
- Monitor action execution queue depth
- Detect stuck or growing queues
- Track execution throughput
- Validate policy enforcement
- Operational dashboards
**Related Documentation:**
- [Queue Architecture](./queue-architecture.md)
- [Policy Enforcement](./executor-service.md#policy-enforcement)
---
## Examples
### Creating a Simple Action
```bash
curl -X POST http://localhost:3000/api/v1/actions \
-H "Content-Type: application/json" \
-d '{
"ref": "mypack.hello_world",
"pack_ref": "mypack",
"label": "Hello World",
"description": "Prints hello world",
"entrypoint": "/actions/hello.py"
}'
```
### Creating an Action with Parameter Schema
```bash
curl -X POST http://localhost:3000/api/v1/actions \
-H "Content-Type: application/json" \
-d '{
"ref": "mypack.send_email",
"pack_ref": "mypack",
"label": "Send Email",
"description": "Sends an email message",
"entrypoint": "/actions/send_email.py",
"param_schema": {
"type": "object",
"properties": {
"to": { "type": "string", "format": "email" },
"subject": { "type": "string" },
"body": { "type": "string" }
},
"required": ["to", "subject", "body"]
}
}'
```
### Listing Actions for a Pack
```bash
curl http://localhost:3000/api/v1/packs/core/actions
```
### Updating an Action
```bash
curl -X PUT http://localhost:3000/api/v1/actions/mypack.hello_world \
-H "Content-Type: application/json" \
-d '{
"label": "Hello World v2",
"description": "Updated hello world action"
}'
```
### Deleting an Action
```bash
curl -X DELETE http://localhost:3000/api/v1/actions/mypack.hello_world
```
### Getting Queue Statistics
```bash
curl http://localhost:3000/api/v1/actions/core.http.get/queue-stats
```
**Example Response:**
```json
{
"data": {
"action_id": 1,
"action_ref": "core.http.get",
"queue_length": 12,
"active_count": 5,
"max_concurrent": 5,
"oldest_enqueued_at": "2025-01-27T12:40:00Z",
"total_enqueued": 523,
"total_completed": 511,
"last_updated": "2025-01-27T12:45:30Z"
}
}
```
---
## Validation Rules
### Action Reference (`ref`)
- Must be unique across all actions
- Can contain alphanumeric characters, dots (.), underscores (_), and hyphens (-)
- Typically follows the pattern: `pack_name.action_name`
- Example: `core.http.get`, `aws.ec2.start_instance`
### Pack Reference (`pack_ref`)
- Must reference an existing pack
- The pack must exist before creating actions for it
### Entry Point (`entrypoint`)
- Path or identifier for the executable code
- Can be a file path, module name, function name, etc.
- Format depends on the runtime environment
### Schemas (`param_schema`, `out_schema`)
- Must be valid JSON Schema documents
- Used for validation during action execution
- Helps with auto-generating documentation and UI
---
## Best Practices
1. **Naming Conventions**
- Use descriptive, hierarchical names: `pack.category.action`
- Keep names concise but meaningful
- Use lowercase with dots as separators
2. **Schema Definitions**
- Always provide `param_schema` for clarity
- Define `required` fields in schemas
- Use appropriate JSON Schema types and formats
- Document schema fields with descriptions
3. **Entry Points**
- Use consistent paths relative to the pack root
- Keep entry points simple and maintainable
- Consider versioning: `/actions/v1/http_get.py`
4. **Runtime Association**
- Specify runtime when actions have specific dependencies
- Null runtime means use default/generic runtime
- Ensure runtime exists before creating action
5. **Error Handling**
- Design actions to handle errors gracefully
- Use output schemas to define error structures
- Log execution details for debugging
6. **Queue Monitoring**
- Use `/queue-stats` endpoint to monitor execution queues
- Alert on high `queue_length` (> 100)
- Investigate when `oldest_enqueued_at` is old (> 30 minutes)
- Track completion rate: `total_completed / total_enqueued`
---
## Queue Statistics
The `/queue-stats` endpoint provides real-time visibility into action execution queues.
### Understanding Queue Metrics
- **queue_length**: Executions waiting to run (0 = healthy)
- **active_count**: Executions currently running
- **max_concurrent**: Policy-enforced concurrency limit
- **oldest_enqueued_at**: How long the oldest execution has been waiting
- **total_enqueued/completed**: Lifetime throughput metrics
### Healthy vs Unhealthy Queues
**Healthy:**
- ✅ `queue_length` is 0 or low (< 10)
- ✅ `active_count` ≈ `max_concurrent` during load
- ✅ `oldest_enqueued_at` is recent (< 5 minutes)
- ✅ `total_completed` increases steadily
**Unhealthy:**
- ⚠️ `queue_length` consistently high (> 50)
- ⚠️ `oldest_enqueued_at` is old (> 30 minutes)
- 🚨 Queue not progressing (stats not updating)
- 🚨 `active_count` < `max_concurrent` (workers stuck)
### Monitoring Recommendations
1. **Set up alerts** for high queue depths
2. **Track trends** in `total_enqueued` vs `total_completed`
3. **Investigate spikes** in `queue_length`
4. **Scale workers** when queues consistently fill
5. **Adjust policies** if concurrency limits are too restrictive
For detailed queue architecture and troubleshooting, see [Queue Architecture Documentation](./queue-architecture.md).
---
## Related Documentation
- [Pack Management API](./api-packs.md)
- [Runtime Management API](./api-runtimes.md)
- [Rule Management API](./api-rules.md)
- [Execution API](./api-executions.md)
- [Queue Architecture](./queue-architecture.md)
- [Executor Service](./executor-service.md)
---
**Last Updated:** January 27, 2025

527
docs/api/api-completion-plan.md Normal file
View File

@@ -0,0 +1,527 @@
# API Completion Plan: Implementing Suppressed Features
**Date:** 2026-01-27
**Status:** Phase 1 & 3 Complete, Phase 2 In Progress
**Context:** During zero-warnings cleanup, several API methods and features were marked with `#[allow(dead_code)]` because they represent planned but unimplemented functionality. This document outlines a plan to implement these features.
## Overview
We identified 4 main categories of suppressed features:
1. **CLI Client REST operations** - Missing HTTP methods for full CRUD
2. **CLI Configuration management** - Incomplete config commands and profile support
3. **Token refresh mechanism** - Session management for long-running CLI usage
4. **Executor monitoring APIs** - Runtime inspection of policies and queues
---
## Priority 1: Token Refresh Mechanism (High Impact)
### Problem
- Access tokens expire after 1 hour
- Long-running CLI sessions require manual re-login
- No automatic token refresh flow
### Suppressed Methods
- `ApiClient::set_auth_token()` - Update token after refresh
- `ApiClient::clear_auth_token()` - Clear on logout/refresh failure
- `CliConfig::refresh_token()` - Get refresh token from config
### Implementation Plan
#### 1.1: Add Token Refresh API Endpoint
**File:** `attune/crates/api/src/routes/auth.rs`
```rust
#[utoipa::path(
post,
path = "/auth/refresh",
request_body = RefreshTokenRequest,
responses(
(status = 200, description = "Token refreshed", body = LoginResponse),
(status = 401, description = "Invalid refresh token")
),
tag = "auth"
)]
async fn refresh_token(
State(state): State<AppState>,
Json(req): Json<RefreshTokenRequest>,
) -> Result<Json<ApiResponse<LoginResponse>>, ApiError> {
// Validate refresh token
// Generate new access token (and optionally new refresh token)
// Return new tokens
}
```
**Add DTO:** `attune/crates/api/src/dto/auth.rs`
```rust
#[derive(Deserialize)]
pub struct RefreshTokenRequest {
pub refresh_token: String,
}
```
#### 1.2: Implement Automatic Refresh in CLI Client
**File:** `attune/crates/cli/src/client.rs`
Add method:
```rust
pub async fn refresh_auth_token(&mut self) -> Result<()> {
// Get refresh token from config
// Call /auth/refresh endpoint
// Update auth_token with set_auth_token()
// Update config file with new tokens
}
```
Enhance `execute()` method:
```rust
async fn execute<T: DeserializeOwned>(&self, req: RequestBuilder) -> Result<T> {
let response = req.send().await?;
// If 401 and we have refresh token, try refresh once
if response.status() == StatusCode::UNAUTHORIZED {
if let Ok(Some(_)) = config.refresh_token() {
self.refresh_auth_token().await?;
// Retry original request with new token
}
}
self.handle_response(response).await
}
```
#### 1.3: Add Refresh Command (Optional)
**File:** `attune/crates/cli/src/commands/auth.rs`
```rust
AuthCommands::Refresh => {
// Manually refresh token
// Useful for testing or explicit refresh
}
```
**Effort:** 4-6 hours
**Dependencies:** None
**Value:** High - significantly improves CLI UX
---
## Priority 2: Complete CRUD Operations (High Impact)
### Problem
- CLI can only Create (POST) and Read (GET) resources
- No Update (PUT/PATCH) or Delete (DELETE) commands
- REST client API incomplete
### Suppressed Methods
- `ApiClient::put()` - For update operations
- `ApiClient::delete()` - For delete operations
- `ApiClient::get_with_query()` - For filtering/search
### Implementation Plan
#### 2.1: Resource Update Commands
Add update subcommands to existing command modules:
**Pack Updates** (`attune/crates/cli/src/commands/pack.rs`):
```rust
PackCommands::Update {
ref_name: String,
#[arg(long)] name: Option<String>,
#[arg(long)] description: Option<String>,
#[arg(long)] version: Option<String>,
#[arg(long)] enabled: Option<bool>,
} => {
let update = UpdatePackRequest { name, description, version, enabled };
let pack = client.put(&format!("/packs/{}", ref_name), &update).await?;
// Display updated pack
}
```
**Action Updates** (`attune/crates/cli/src/commands/action.rs`):
```rust
ActionCommands::Update {
ref_name: String,
#[arg(long)] name: Option<String>,
#[arg(long)] description: Option<String>,
#[arg(long)] enabled: Option<bool>,
} => {
// Similar pattern
}
```
Repeat for: Rules, Triggers, Sensors, Workflows
#### 2.2: Resource Delete Commands
Add delete subcommands:
```rust
PackCommands::Delete {
ref_name: String,
#[arg(long)] force: bool, // Skip confirmation
} => {
if !force {
let confirm = dialoguer::Confirm::new()
.with_prompt(&format!("Delete pack '{}'?", ref_name))
.interact()?;
if !confirm { return Ok(()); }
}
client.delete::<()>(&format!("/packs/{}", ref_name)).await?;
print_success(&format!("Pack '{}' deleted", ref_name));
}
```
#### 2.3: Search and Filtering
Enhance list commands with query parameters:
```rust
PackCommands::List {
#[arg(long)] enabled: Option<bool>,
#[arg(long)] search: Option<String>,
#[arg(long)] limit: Option<u32>,
#[arg(long)] offset: Option<u32>,
} => {
let mut query = vec![];
if let Some(enabled) = enabled {
query.push(format!("enabled={}", enabled));
}
if let Some(search) = search {
query.push(format!("search={}", search));
}
// Build query string and use get_with_query()
}
```
**Effort:** 8-12 hours
**Dependencies:** API endpoints must support PUT/DELETE (most already do)
**Value:** High - completes CLI feature parity with API
---
## Priority 3: Profile and Configuration Management (Medium Impact)
### Problem
- `--profile` flag declared in main.rs but not used
- `attune config set api-url` command defined but not implemented
- Profile switching requires manual `attune config use` command
### Suppressed Methods
- `CliConfig::set_api_url()` - Direct API URL update
- `CliConfig::load_with_profile()` - Load with profile override
- `CliConfig::api_url()` - Get current API URL
### Implementation Plan
#### 3.1: Implement `--profile` Flag Support
**File:** `attune/crates/cli/src/main.rs`
Currently the flag exists but is unused. Update `main()`:
```rust
#[tokio::main]
async fn main() -> Result<()> {
let cli = Cli::parse();
// Load config with profile override
let config = if let Some(profile) = &cli.profile {
CliConfig::load_with_profile(Some(profile))?
} else {
CliConfig::load()?
};
// Pass config to command handlers instead of loading fresh each time
}
```
Update all command handlers to accept `&CliConfig` parameter.
#### 3.2: Complete Config Set Command
**File:** `attune/crates/cli/src/commands/config.rs`
The `handle_set()` function exists but uses `set_value()` which only handles specific keys. Add:
```rust
async fn handle_set(key: String, value: String, output_format: OutputFormat) -> Result<()> {
let mut config = CliConfig::load()?;
match key.as_str() {
"api_url" | "api-url" => {
config.set_api_url(value.clone())?;
}
_ => {
config.set_value(&key, value.clone())?;
}
}
// Output handling...
}
```
#### 3.3: Add Config Get/Show Commands
```rust
ConfigCommands::Show => {
let config = CliConfig::load()?;
let profile = config.current_profile()?;
// Display full config including:
// - Current profile
// - API URL (from config.api_url())
// - Auth status
// - All settings
}
```
**Effort:** 3-4 hours
**Dependencies:** None
**Value:** Medium - improves multi-environment workflow
---
## Priority 4: Executor Monitoring APIs (Low-Medium Impact)
### Problem
- No visibility into executor runtime state
- Cannot inspect queue depths or policy enforcement
- No way to adjust policies at runtime
### Suppressed Methods in `attune/crates/executor/src/`:
- `PolicyEnforcer::new()`, `with_global_policy()`, `set_*_policy()`
- `PolicyEnforcer::check_policies()`, `wait_for_policy_compliance()`
- `QueueManager::get_all_queue_stats()`, `cancel_execution()`, `clear_all_queues()`
- `ExecutorService::pool()`, `config()`, `publisher()`
### Implementation Plan
#### 4.1: Add Executor Admin API Endpoints
**New File:** `attune/crates/api/src/routes/executor_admin.rs`
```rust
// Queue monitoring
GET /api/v1/admin/executor/queues
-> List all action queues with stats (depth, active, capacity)
GET /api/v1/admin/executor/queues/{action_id}
-> Detailed queue stats for specific action
// Policy inspection
GET /api/v1/admin/executor/policies
-> List all configured policies (global, pack, action)
// Policy management (future)
POST /api/v1/admin/executor/policies/global
-> Set global execution policy
POST /api/v1/admin/executor/policies/pack/{pack_id}
-> Set pack-specific policy
POST /api/v1/admin/executor/policies/action/{action_id}
-> Set action-specific policy
// Queue operations
POST /api/v1/admin/executor/queues/{action_id}/clear
-> Clear queue for action (emergency)
DELETE /api/v1/admin/executor/queues/{action_id}/executions/{execution_id}
-> Cancel queued execution
```
#### 4.2: Expose Executor State via Message Queue
**Alternative approach:** Instead of HTTP API, use pub/sub:
```rust
// Executor publishes stats periodically
topic: "executor.stats"
payload: {
"queues": [...],
"policies": [...],
"active_executions": 42
}
// API service subscribes and caches latest stats
// Serves from cache on GET /admin/executor/stats
```
#### 4.3: Add CLI Admin Commands
**New File:** `attune/crates/cli/src/commands/admin.rs`
```rust
#[derive(Subcommand)]
pub enum AdminCommands {
/// Executor administration
Executor {
#[command(subcommand)]
command: ExecutorAdminCommands,
},
}
#[derive(Subcommand)]
pub enum ExecutorAdminCommands {
/// Show queue statistics
Queues {
#[arg(long)] action: Option<String>,
},
/// Show policy configuration
Policies,
/// Clear a queue (dangerous)
ClearQueue {
action: String,
#[arg(long)] confirm: bool,
},
}
```
**Effort:** 6-10 hours
**Dependencies:** Requires deciding on HTTP API vs. message queue approach
**Value:** Medium - mainly for ops/debugging, not end-user facing
---
## Non-Priority Items (Keep Suppressed)
These items should remain with `#[allow(dead_code)]` for now:
### Test Infrastructure
- `attune/crates/api/tests/helpers.rs` - Test helper methods
- `attune/crates/cli/tests/common/mod.rs` - Mock functions
- `attune/crates/executor/tests/*` - Test runtime helpers
**Reason:** These are test utilities that may be used in future tests. They provide a complete test API surface even if not all methods are currently used.
### Service Internal Fields
- `WorkerService.config` - Kept for potential future use
- `ExecutorService.queue_name` - Backward compatibility
- `Subscriber.client_id`, `user_id` - Structural completeness
- `WorkflowExecutionHandle.workflow_def_id` - May be needed for advanced workflow features
**Reason:** These maintain API completeness and may be needed for future features. Removing them would require breaking changes.
### Methods Only Used in Tests
- `PolicyEnforcer::new()` (vs. `with_queue_manager()`)
- `QueueManager::new()`, `with_defaults()` (vs. `with_db_pool()`)
**Reason:** Simpler constructors useful for unit testing. Production code uses more complete constructors.
---
## Implementation Roadmap
### Phase 1: Foundation ✅ COMPLETE
1. ✅ **Token Refresh Mechanism** (Priority 1) - COMPLETE 2026-01-27
- ✅ API endpoint (already existed)
- ✅ CLI client auto-refresh implemented
- ✅ Manual refresh command (`attune auth refresh`)
- ✅ Config file persistence of refreshed tokens
- ✅ Zero warnings, fully functional
- **Testing:** Auto-refresh on 401, manual refresh command tested
### Phase 2: CLI Feature Completion ✅ COMPLETE
2. ✅ **CRUD Operations** (Priority 2) - COMPLETE 2026-01-27
- ✅ Update commands implemented for actions, rules, triggers, packs
- ✅ Delete commands with confirmation for actions, triggers (rules/packs already had it)
- ✅ All ApiClient HTTP methods now active (PUT, DELETE)
- ✅ Zero dead_code warnings for implemented features
- **Testing:** All CLI tests pass (2 pre-existing auth failures unrelated)
### Phase 3: Configuration Enhancement ✅ COMPLETE
3. ✅ **Profile Management** (Priority 3) - COMPLETE 2026-01-27
- ✅ `--profile` flag fully implemented across all commands
- ✅ `CliConfig::load_with_profile()` method
- ✅ All command handlers updated (auth, action, rule, execution, key, sensor, trigger, pack, config)
- ✅ Zero warnings, fully functional
- **Testing:** All CLI tests pass with profile support
### Phase 4: Operational Visibility (Week 6-8, Optional)
4. ⏳ **Executor Monitoring** (Priority 4)
- Design decision: HTTP API vs. pub/sub
- Implement chosen approach
- CLI admin commands
- **Blockers:** Architecture decision needed
- **Testing:** Requires running executor instances
---
## Success Criteria
### Phase 1 Complete ✅
- [x] Users can work with CLI for >1 hour without re-login
- [x] Expired tokens automatically refresh transparently
- [x] Manual `attune auth refresh` command works
### Phase 2 Complete ✅
- [x] All resources support full CRUD via CLI (Create, Read, Update, Delete)
- [x] Update commands accept optional fields (label, description, etc.)
- [x] Delete operations require confirmation (unless --yes flag)
- [x] Users can manage entire platform lifecycle from CLI
### Phase 3 Complete ✅
- [x] `--profile dev/staging/prod` flag works across all commands
- [x] `attune config set api-url <url>` updates current profile
- [x] Multi-environment workflows seamless
### Phase 4 Complete
- [ ] Operators can view executor queue depths
- [ ] Policy configuration visible via API/CLI
- [ ] Emergency queue operations available
---
## Risk Assessment
### Low Risk
- **Token refresh:** Well-defined pattern, existing JWT infrastructure
- **CRUD completion:** API endpoints mostly exist, just need CLI wiring
- **Profile flag:** Simple config plumbing
### Medium Risk
- **Executor monitoring:** Architecture decision required (HTTP vs. message queue)
- HTTP: Simpler but requires executor to expose endpoints
- Message Queue: More scalable but adds complexity
- **Recommendation:** Start with HTTP for simplicity
### Dependency Risks
- No major dependencies between phases
- Each phase can be implemented independently
- Phase 4 can be deferred if priorities change
---
## Documentation Updates Required
After implementation:
1. Update `docs/cli-reference.md` with new commands
2. Update `docs/api-authentication.md` with refresh flow
3. Add `docs/cli-configuration.md` with profile examples
4. Add `docs/executor-monitoring.md` (Phase 4)
5. Update this document's status to "Complete"
---
## Alternatives Considered
### Alternative 1: Remove Suppressed Methods
**Rejected:** These methods represent planned functionality with clear use cases. Removing them would require re-adding later with breaking changes.
### Alternative 2: Implement All at Once
**Rejected:** Phased approach allows for incremental value delivery and testing. Phase 4 can be deferred if needed.
### Alternative 3: Auto-generate CLI from OpenAPI
**Deferred:** Would eliminate need for manual CLI CRUD implementation, but requires significant tooling investment. Consider for future major version.
---
## Next Steps
1. **Review this plan** with team/stakeholders
2. **Decide on Phase 4 architecture** (HTTP vs. message queue)
3. **Create GitHub issues** for each phase
4. **Implement Phase 1** (highest impact, no blockers)
5. **Update this document** as implementation progresses
---
## Related Documentation
- `attune/docs/dead-code-cleanup.md` - Warning cleanup context
- `attune/docs/cli-reference.md` - Current CLI commands
- `attune/docs/api-authentication.md` - Auth flow documentation
- `attune/CHANGELOG.md` - Historical context on warning fixes

581
docs/api/api-events-enforcements.md Normal file
View File

@@ -0,0 +1,581 @@
# Event & Enforcement Query API
The Event & Enforcement Query API provides read-only endpoints for querying events and enforcements in the Attune automation platform. These endpoints enable monitoring of trigger firings (events) and rule activations (enforcements), which are fundamental to understanding automation workflow execution.
## Table of Contents
- [Overview](#overview)
- [Event Model](#event-model)
- [Enforcement Model](#enforcement-model)
- [Authentication](#authentication)
- [Endpoints](#endpoints)
- [List Events](#list-events)
- [Get Event by ID](#get-event-by-id)
- [List Enforcements](#list-enforcements)
- [Get Enforcement by ID](#get-enforcement-by-id)
- [Use Cases](#use-cases)
- [Related Resources](#related-resources)
---
## Overview
### Events
**Events** represent trigger firings - instances when a trigger condition is met and an event is generated. Events are the starting point of automation workflows, carrying the data payload that flows through the system.
**Key Characteristics:**
- Immutable records of trigger activations
- Contain payload data from the trigger source
- Link to the trigger that generated them
- Can be queried by trigger, source, or time range
### Enforcements
**Enforcements** represent rule activations - instances when a rule matches an event and schedules actions for execution. Enforcements are the bridge between events and executions.
**Key Characteristics:**
- Created when a rule's conditions match an event
- Track the status of rule execution (pending, scheduled, running, completed, failed)
- Link events to the resulting executions
- Store condition evaluation results
### Event Flow
```
Sensor/Trigger → Event → Rule Evaluation → Enforcement → Execution(s)
```
---
## Event Model
### Event Object
```json
{
"id": 123,
"trigger": 456,
"trigger_ref": "core.webhook_received",
"config": {
"endpoint": "/webhooks/github",
"method": "POST"
},
"payload": {
"repository": "attune/platform",
"action": "push",
"commit": "abc123"
},
"source": 789,
"source_ref": "github_webhook_sensor",
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
}
```
### Fields
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Unique event identifier |
| `trigger` | integer | Optional trigger ID that generated this event |
| `trigger_ref` | string | Reference to the trigger (e.g., "core.webhook_received") |
| `config` | object | Optional configuration data for the event |
| `payload` | object | Event payload data from the trigger source |
| `source` | integer | Optional ID of the sensor that created the event |
| `source_ref` | string | Optional reference to the sensor |
| `created` | datetime | Timestamp when event was created |
| `updated` | datetime | Timestamp of last update |
---
## Enforcement Model
### Enforcement Object
```json
{
"id": 234,
"rule": 567,
"rule_ref": "deploy_on_push",
"trigger_ref": "core.webhook_received",
"config": {
"branch": "main",
"environment": "production"
},
"event": 123,
"status": "completed",
"payload": {
"repository": "attune/platform",
"commit": "abc123"
},
"condition": "passed",
"conditions": {
"branch_check": true,
"approval_check": true
},
"created": "2024-01-15T10:00:01Z",
"updated": "2024-01-15T10:05:00Z"
}
```
### Fields
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Unique enforcement identifier |
| `rule` | integer | Optional rule ID that created this enforcement |
| `rule_ref` | string | Reference to the rule |
| `trigger_ref` | string | Reference to the trigger that fired |
| `config` | object | Optional configuration data |
| `event` | integer | Optional ID of the event that triggered this enforcement |
| `status` | string | Enforcement status (see below) |
| `payload` | object | Data payload for the enforcement |
| `condition` | string | Overall condition result: `passed`, `failed`, `skipped` |
| `conditions` | object | Detailed condition evaluation results |
| `created` | datetime | Timestamp when enforcement was created |
| `updated` | datetime | Timestamp of last update |
### Enforcement Status
| Status | Description |
|--------|-------------|
| `pending` | Enforcement created, waiting to be processed |
| `scheduled` | Actions scheduled for execution |
| `running` | Enforcement is actively running |
| `completed` | All actions completed successfully |
| `failed` | Enforcement failed due to error |
| `cancelled` | Enforcement was cancelled |
### Enforcement Condition
| Condition | Description |
|-----------|-------------|
| `passed` | All rule conditions matched |
| `failed` | One or more conditions failed |
| `skipped` | Enforcement was skipped (e.g., rule disabled) |
---
## Authentication
All event and enforcement endpoints require authentication. Include a valid JWT access token in the `Authorization` header:
```
Authorization: Bearer <access_token>
```
See the [Authentication Guide](./authentication.md) for details on obtaining tokens.
---
## Endpoints
### List Events
Retrieve a paginated list of events with optional filtering.
**Endpoint:** `GET /api/v1/events`
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `trigger` | integer | - | Filter by trigger ID |
| `trigger_ref` | string | - | Filter by trigger reference |
| `source` | integer | - | Filter by source ID |
| `page` | integer | 1 | Page number (1-indexed) |
| `per_page` | integer | 50 | Items per page (max 100) |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/events?trigger_ref=core.webhook_received&page=1&per_page=20" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": [
{
"id": 123,
"trigger": 456,
"trigger_ref": "core.webhook_received",
"source": 789,
"source_ref": "github_webhook_sensor",
"has_payload": true,
"created": "2024-01-15T10:00:00Z"
}
],
"pagination": {
"page": 1,
"page_size": 20,
"total_items": 1,
"total_pages": 1
}
}
```
---
### Get Event by ID
Retrieve a single event by its ID.
**Endpoint:** `GET /api/v1/events/:id`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `id` | integer | Event ID |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/events/123" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": {
"id": 123,
"trigger": 456,
"trigger_ref": "core.webhook_received",
"config": {
"endpoint": "/webhooks/github",
"method": "POST"
},
"payload": {
"repository": "attune/platform",
"action": "push",
"commit": "abc123",
"branch": "main"
},
"source": 789,
"source_ref": "github_webhook_sensor",
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
}
}
```
**Error Responses:**
- `404 Not Found`: Event not found
---
### List Enforcements
Retrieve a paginated list of enforcements with optional filtering.
**Endpoint:** `GET /api/v1/enforcements`
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `rule` | integer | - | Filter by rule ID |
| `event` | integer | - | Filter by event ID |
| `status` | string | - | Filter by status (pending, scheduled, running, completed, failed, cancelled) |
| `trigger_ref` | string | - | Filter by trigger reference |
| `page` | integer | 1 | Page number (1-indexed) |
| `per_page` | integer | 50 | Items per page (max 100) |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/enforcements?status=completed&page=1" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": [
{
"id": 234,
"rule": 567,
"rule_ref": "deploy_on_push",
"trigger_ref": "core.webhook_received",
"event": 123,
"status": "completed",
"condition": "passed",
"created": "2024-01-15T10:00:01Z"
}
],
"pagination": {
"page": 1,
"page_size": 50,
"total_items": 1,
"total_pages": 1
}
}
```
---
### Get Enforcement by ID
Retrieve a single enforcement by its ID.
**Endpoint:** `GET /api/v1/enforcements/:id`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `id` | integer | Enforcement ID |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/enforcements/234" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": {
"id": 234,
"rule": 567,
"rule_ref": "deploy_on_push",
"trigger_ref": "core.webhook_received",
"config": {
"branch": "main",
"environment": "production"
},
"event": 123,
"status": "completed",
"payload": {
"repository": "attune/platform",
"commit": "abc123"
},
"condition": "passed",
"conditions": {
"branch_check": true,
"approval_check": true,
"deployment_ready": true
},
"created": "2024-01-15T10:00:01Z",
"updated": "2024-01-15T10:05:00Z"
}
}
```
**Error Responses:**
- `404 Not Found`: Enforcement not found
---
## Use Cases
### Monitoring Event Flow
Track events as they flow through the system:
```bash
# Get recent events for a specific trigger
curl -X GET "http://localhost:8080/api/v1/events?trigger_ref=core.webhook_received&per_page=10" \
-H "Authorization: Bearer <token>"
# Get a specific event's details
curl -X GET "http://localhost:8080/api/v1/events/123" \
-H "Authorization: Bearer <token>"
```
### Tracking Rule Activations
Monitor which rules are being triggered:
```bash
# Get all enforcements for a specific rule
curl -X GET "http://localhost:8080/api/v1/enforcements?rule=567" \
-H "Authorization: Bearer <token>"
# Get completed enforcements
curl -X GET "http://localhost:8080/api/v1/enforcements?status=completed" \
-H "Authorization: Bearer <token>"
```
### Debugging Workflow Issues
Investigate why a workflow didn't execute as expected:
```bash
# 1. Find the event
curl -X GET "http://localhost:8080/api/v1/events?trigger_ref=core.webhook_received" \
-H "Authorization: Bearer <token>"
# 2. Check if enforcement was created for the event
curl -X GET "http://localhost:8080/api/v1/enforcements?event=123" \
-H "Authorization: Bearer <token>"
# 3. Examine enforcement details and condition evaluation
curl -X GET "http://localhost:8080/api/v1/enforcements/234" \
-H "Authorization: Bearer <token>"
```
### Auditing System Activity
Audit automation activity over time:
```bash
# Get all enforcements for a specific trigger type
curl -X GET "http://localhost:8080/api/v1/enforcements?trigger_ref=core.webhook_received" \
-H "Authorization: Bearer <token>"
# Check failed enforcements
curl -X GET "http://localhost:8080/api/v1/enforcements?status=failed" \
-H "Authorization: Bearer <token>"
```
---
## Event-to-Execution Tracing
To trace the full flow from event to execution:
1. **Find the Event**: Query events by trigger or time range
2. **Find Enforcements**: Query enforcements by event ID
3. **Find Executions**: Query executions by enforcement ID (see [Execution API](./api-executions.md))
**Example Flow:**
```bash
# 1. Get event
EVENT_ID=123
# 2. Get enforcements for this event
curl -X GET "http://localhost:8080/api/v1/enforcements?event=${EVENT_ID}" \
-H "Authorization: Bearer <token>"
# 3. Get executions for the enforcement (from Execution API)
ENFORCEMENT_ID=234
curl -X GET "http://localhost:8080/api/v1/executions?enforcement=${ENFORCEMENT_ID}" \
-H "Authorization: Bearer <token>"
```
---
## Best Practices
### 1. Use Filters to Reduce Data Volume
Always filter by specific criteria when possible:
```bash
# Good: Filter by trigger
curl -X GET "http://localhost:8080/api/v1/events?trigger_ref=core.webhook_received"
# Better: Add pagination
curl -X GET "http://localhost:8080/api/v1/events?trigger_ref=core.webhook_received&per_page=20"
```
### 2. Monitor Enforcement Status
Regularly check for failed or stuck enforcements:
```bash
# Check for failed enforcements
curl -X GET "http://localhost:8080/api/v1/enforcements?status=failed"
# Check for long-running enforcements
curl -X GET "http://localhost:8080/api/v1/enforcements?status=running"
```
### 3. Correlate Events and Enforcements
Always correlate events with their enforcements to understand rule behavior:
```bash
# Get event details
curl -X GET "http://localhost:8080/api/v1/events/123"
# Get related enforcements
curl -X GET "http://localhost:8080/api/v1/enforcements?event=123"
```
### 4. Use Pagination for Large Result Sets
Always use pagination when querying large datasets:
```bash
curl -X GET "http://localhost:8080/api/v1/events?page=1&per_page=50"
```
---
## Error Handling
### Common Error Codes
| Status Code | Description |
|-------------|-------------|
| `400 Bad Request` | Invalid query parameters |
| `401 Unauthorized` | Missing or invalid authentication token |
| `404 Not Found` | Event or enforcement not found |
| `500 Internal Server Error` | Server error |
### Example Error Response
```json
{
"error": "Event with ID 999 not found",
"status": 404
}
```
---
## Performance Considerations
### Query Optimization
- **Filter at the database level**: Use query parameters like `trigger`, `rule`, `event`, and `status`
- **Limit result sets**: Use `per_page` to control result size
- **Index-aware queries**: Queries by ID, trigger, rule, and status are optimized with database indexes
### Pagination Best Practices
- Default page size: 50 items
- Maximum page size: 100 items
- Use `page` and `per_page` parameters consistently
---
## Related Resources
- [Trigger & Sensor Management API](./api-triggers-sensors.md) - Manage triggers that create events
- [Rule Management API](./api-rules.md) - Manage rules that create enforcements
- [Execution Management API](./api-executions.md) - View executions created by enforcements
- [Authentication Guide](./authentication.md) - API authentication details
---
## Future Enhancements
### Planned Features
1. **Time Range Filtering**: Filter events and enforcements by creation time range
2. **Advanced Search**: Full-text search in event payloads
3. **Aggregation Queries**: Count events/enforcements by trigger/rule/status
4. **Real-time Streaming**: WebSocket support for live event/enforcement updates
5. **Export Capabilities**: Export events and enforcements to CSV/JSON
6. **Event Replay**: Replay events for testing and debugging
7. **Retention Policies**: Automatic archival of old events and enforcements
---
**Last Updated:** 2024-01-13
**API Version:** v1

681
docs/api/api-executions.md Normal file
View File

@@ -0,0 +1,681 @@
# Execution Management API
This document describes the Execution Management API endpoints for the Attune automation platform.
## Overview
Executions represent the runtime instances of actions being performed. The Execution Management API provides observability into the automation system, allowing you to track action executions, monitor their status, analyze results, and understand system activity.
**Base Path:** `/api/v1/executions`
## Data Model
### Execution
```json
{
"id": 1,
"action": 5,
"action_ref": "slack.send_message",
"config": {
"channel": "#alerts",
"message": "Error detected in production"
},
"parent": null,
"enforcement": 3,
"executor": 1,
"status": "completed",
"result": {
"message_id": "1234567890.123456",
"success": true
},
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:05Z"
}
```
### Execution Summary (List View)
```json
{
"id": 1,
"action_ref": "slack.send_message",
"status": "completed",
"parent": null,
"enforcement": 3,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:05Z"
}
```
### Execution Status Values
- **`requested`** - Execution has been requested but not yet scheduled
- **`scheduling`** - Execution is being scheduled to a worker
- **`scheduled`** - Execution has been scheduled and queued
- **`running`** - Execution is currently in progress
- **`completed`** - Execution finished successfully
- **`failed`** - Execution failed with an error
- **`canceling`** - Execution is being cancelled
- **`cancelled`** - Execution was cancelled
- **`timeout`** - Execution exceeded time limit
- **`abandoned`** - Execution was abandoned (worker died, etc.)
## Endpoints
### List All Executions
Retrieve a paginated list of executions with optional filters.
**Endpoint:** `GET /api/v1/executions`
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
- `status` (string, optional): Filter by execution status
- `action_ref` (string, optional): Filter by action reference
- `pack_name` (string, optional): Filter by pack name
- `result_contains` (string, optional): Search in result JSON (case-insensitive substring match)
- `enforcement` (integer, optional): Filter by enforcement ID
- `parent` (integer, optional): Filter by parent execution ID
**Response:** `200 OK`
```json
{
"data": [
{
"id": 1,
"action_ref": "slack.send_message",
"status": "completed",
"parent": null,
"enforcement": 3,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:05Z"
}
],
"pagination": {
"page": 1,
"page_size": 20,
"total_items": 1,
"total_pages": 1
}
}
```
**Examples:**
```bash
# List all executions
curl http://localhost:3000/api/v1/executions
# Filter by status
curl http://localhost:3000/api/v1/executions?status=completed
# Filter by action
curl http://localhost:3000/api/v1/executions?action_ref=slack.send_message
# Filter by pack
curl http://localhost:3000/api/v1/executions?pack_name=core
# Search in execution results
curl http://localhost:3000/api/v1/executions?result_contains=error
# Multiple filters with pagination
curl http://localhost:3000/api/v1/executions?pack_name=monitoring&status=failed&result_contains=timeout&page=2&per_page=50
```
---
### Get Execution by ID
Retrieve detailed information about a specific execution.
**Endpoint:** `GET /api/v1/executions/:id`
**Path Parameters:**
- `id` (integer): Execution ID
**Response:** `200 OK`
```json
{
"data": {
"id": 1,
"action": 5,
"action_ref": "slack.send_message",
"config": {
"channel": "#alerts",
"message": "Error detected in production"
},
"parent": null,
"enforcement": 3,
"executor": 1,
"status": "completed",
"result": {
"message_id": "1234567890.123456",
"success": true
},
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:05Z"
}
}
```
**Errors:**
- `404 Not Found`: Execution with the specified ID does not exist
---
### List Executions by Status
Retrieve all executions with a specific status.
**Endpoint:** `GET /api/v1/executions/status/:status`
**Path Parameters:**
- `status` (string): Execution status (lowercase)
- Valid values: `requested`, `scheduling`, `scheduled`, `running`, `completed`, `failed`, `canceling`, `cancelled`, `timeout`, `abandoned`
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
**Examples:**
```bash
# Get all running executions
curl http://localhost:3000/api/v1/executions/status/running
# Get all failed executions
curl http://localhost:3000/api/v1/executions/status/failed
# Get completed executions with pagination
curl http://localhost:3000/api/v1/executions/status/completed?page=1&per_page=50
```
**Errors:**
- `400 Bad Request`: Invalid status value
---
### List Executions by Enforcement
Retrieve all executions that were triggered by a specific rule enforcement.
**Endpoint:** `GET /api/v1/executions/enforcement/:enforcement_id`
**Path Parameters:**
- `enforcement_id` (integer): Enforcement ID
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
**Example:**
```bash
curl http://localhost:3000/api/v1/executions/enforcement/42
```
---
### Get Execution Statistics
Retrieve aggregate statistics about executions.
**Endpoint:** `GET /api/v1/executions/stats`
**Response:** `200 OK`
```json
{
"data": {
"total": 1523,
"completed": 1420,
"failed": 45,
"running": 12,
"pending": 28,
"cancelled": 15,
"timeout": 2,
"abandoned": 1
}
}
```
**Description:**
- `total`: Total number of executions (limited to most recent 1000)
- `completed`: Executions that finished successfully
- `failed`: Executions that failed with errors
- `running`: Currently executing
- `pending`: Requested, scheduling, or scheduled
- `cancelled`: Cancelled by user or system
- `timeout`: Exceeded time limit
- `abandoned`: Worker died or lost connection
**Example:**
```bash
curl http://localhost:3000/api/v1/executions/stats
```
---
## Use Cases
### Monitoring Active Executions
Track what's currently running in your automation system:
```bash
# Get all running executions
curl http://localhost:3000/api/v1/executions/status/running
# Get scheduled executions waiting to run
curl http://localhost:3000/api/v1/executions/status/scheduled
```
### Debugging Failed Executions
Investigate failures to understand and fix issues:
```bash
# List all failed executions
curl http://localhost:3000/api/v1/executions/status/failed
# Get details of a specific failed execution
curl http://localhost:3000/api/v1/executions/123
# Filter failures for a specific action
curl http://localhost:3000/api/v1/executions?status=failed&action_ref=aws.ec2.start_instance
```
### Tracking Rule Executions
See what actions were triggered by a specific rule enforcement:
```bash
# Get all executions for an enforcement
curl http://localhost:3000/api/v1/executions/enforcement/42
```
### System Health Monitoring
Monitor overall system health and performance:
```bash
# Get aggregate statistics
curl http://localhost:3000/api/v1/executions/stats
# Check for abandoned executions (potential worker issues)
curl http://localhost:3000/api/v1/executions/status/abandoned
# Monitor timeout rate
curl http://localhost:3000/api/v1/executions/status/timeout
```
### Workflow Tracing
Follow execution chains for complex workflows:
```bash
# Get parent execution
curl http://localhost:3000/api/v1/executions/100
# Find child executions
curl http://localhost:3000/api/v1/executions?parent=100
```
---
## Execution Lifecycle
Understanding the execution lifecycle helps with monitoring and debugging:
```
1. requested → Action execution requested
2. scheduling → Finding available worker
3. scheduled → Assigned to worker, queued
4. running → Currently executing
5. completed → Finished successfully
OR
failed → Error occurred
OR
timeout → Exceeded time limit
OR
cancelled → User/system cancelled
OR
abandoned → Worker lost
```
### State Transitions
**Normal Flow:**
```
requested → scheduling → scheduled → running → completed
```
**Failure Flow:**
```
requested → scheduling → scheduled → running → failed
```
**Cancellation:**
```
(any state) → canceling → cancelled
```
**Timeout:**
```
scheduled/running → timeout
```
**Abandonment:**
```
scheduled/running → abandoned
```
---
## Data Fields
### Execution Fields
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Unique execution identifier |
| `action` | integer | Action ID (null for ad-hoc executions) |
| `action_ref` | string | Action reference identifier |
| `config` | object | Execution configuration/parameters |
| `parent` | integer | Parent execution ID (for nested executions) |
| `enforcement` | integer | Rule enforcement that triggered this execution |
| `executor` | integer | Worker/executor that ran this execution |
| `status` | string | Current execution status |
| `result` | object | Execution result/output |
| `created` | datetime | When execution was created |
| `updated` | datetime | Last update timestamp |
### Config Field
The `config` field contains the parameters passed to the action:
```json
{
"config": {
"url": "https://api.example.com",
"method": "POST",
"headers": {
"Authorization": "Bearer token123"
},
"body": {
"message": "Alert!"
}
}
}
```
### Result Field
The `result` field contains the output from the action execution:
```json
{
"result": {
"status_code": 200,
"response_body": {
"success": true,
"id": "msg_12345"
},
"duration_ms": 234
}
}
```
For failed executions, result typically includes error information:
```json
{
"result": {
"error": "Connection timeout",
"error_code": "ETIMEDOUT",
"stack_trace": "..."
}
}
```
---
## Query Patterns
### Time-Based Queries
While not directly supported by the API, you can filter results client-side:
```bash
# Get recent executions (server returns newest first)
curl http://localhost:3000/api/v1/executions?per_page=100
# Then filter by timestamp in your application
```
### Action Performance Analysis
```bash
# Get all executions for a specific action
curl http://localhost:3000/api/v1/executions?action_ref=slack.send_message
# Check success rate
curl http://localhost:3000/api/v1/executions?action_ref=slack.send_message&status=completed
curl http://localhost:3000/api/v1/executions?action_ref=slack.send_message&status=failed
```
### Enforcement Tracing
```bash
# Get the enforcement details first
curl http://localhost:3000/api/v1/enforcements/42
# Then get all executions triggered by it
curl http://localhost:3000/api/v1/executions/enforcement/42
```
---
## Best Practices
### 1. Polling for Status Updates
When waiting for execution completion:
```bash
# Poll every few seconds
while true; do
status=$(curl -s http://localhost:3000/api/v1/executions/123 | jq -r '.data.status')
echo "Status: $status"
if [[ "$status" == "completed" || "$status" == "failed" ]]; then
break
fi
sleep 2
done
```
**Better approach:** Use WebSocket notifications (via Notifier service) instead of polling.
### 2. Monitoring Dashboard
Build a real-time dashboard:
```bash
# Get current statistics
curl http://localhost:3000/api/v1/executions/stats
# Get active executions
curl http://localhost:3000/api/v1/executions/status/running
# Get recent failures
curl http://localhost:3000/api/v1/executions/status/failed?per_page=10
```
### 3. Debugging Workflow
When investigating issues:
1. Check statistics for anomalies
2. Filter by failed status
3. Get execution details
4. Check result field for error messages
5. Trace back to enforcement and rule
6. Check action configuration
### 4. Performance Monitoring
Track execution patterns:
- Monitor average execution duration via `created` and `updated` timestamps
- Track failure rates by status counts
- Identify slow actions by filtering and analyzing durations
- Monitor timeout frequency to adjust limits
### 5. Cleanup and Archival
Executions accumulate over time. Plan for:
- Regular archival of old executions
- Retention policies based on status
- Separate storage for failed executions (debugging)
- Aggregated metrics for long-term analysis
---
## Limitations
### Current Limitations
1. **Read-Only API**: Currently, executions cannot be created or modified via API. They are created by the executor service when rules trigger.
2. **No Direct Cancellation**: Cancellation endpoint not yet implemented. Will be added in future release.
3. **Limited History**: List endpoint returns most recent 1000 executions from repository.
4. **Client-Side Filtering**: Some filters (action_ref, parent) are applied client-side rather than in database.
5. **No Time Range Queries**: Cannot directly query by time range. Results are ordered by creation time (newest first).
### Future Enhancements
Planned improvements:
- **Cancellation**: `POST /api/v1/executions/:id/cancel` - Cancel a running execution
- **Retry**: `POST /api/v1/executions/:id/retry` - Retry a failed execution
- **Database-Level Filtering**: Move all filters to SQL queries for better performance
- **Time Range Queries**: Add `created_after` and `created_before` parameters
- **Aggregations**: More detailed statistics and analytics
- **Bulk Operations**: Cancel multiple executions at once
- **Execution Logs**: Stream execution logs via WebSocket
- **Export**: Export execution history to CSV/JSON
---
## Error Responses
### 404 Not Found
Execution does not exist:
```json
{
"error": "Execution with ID 999 not found",
"status": 404
}
```
### 400 Bad Request
Invalid status value:
```json
{
"error": "Invalid execution status: invalid_status",
"status": 400
}
```
---
## Integration Examples
### JavaScript/Node.js
```javascript
// Get execution statistics
async function getExecutionStats() {
const response = await fetch('http://localhost:3000/api/v1/executions/stats');
const data = await response.json();
console.log(`Total: ${data.data.total}, Failed: ${data.data.failed}`);
}
// Monitor execution completion
async function waitForExecution(executionId) {
while (true) {
const response = await fetch(`http://localhost:3000/api/v1/executions/${executionId}`);
const data = await response.json();
if (data.data.status === 'completed') {
console.log('Execution completed!', data.data.result);
return data.data.result;
} else if (data.data.status === 'failed') {
throw new Error(`Execution failed: ${JSON.stringify(data.data.result)}`);
}
await new Promise(resolve => setTimeout(resolve, 2000));
}
}
```
### Python
```python
import requests
import time
# Get failed executions
def get_failed_executions():
response = requests.get('http://localhost:3000/api/v1/executions/status/failed')
data = response.json()
return data['data']
# Wait for execution
def wait_for_execution(execution_id, timeout=300):
start_time = time.time()
while time.time() - start_time < timeout:
response = requests.get(f'http://localhost:3000/api/v1/executions/{execution_id}')
data = response.json()
status = data['data']['status']
if status == 'completed':
return data['data']['result']
elif status == 'failed':
raise Exception(f"Execution failed: {data['data']['result']}")
time.sleep(2)
raise TimeoutError(f"Execution {execution_id} did not complete within {timeout}s")
```
---
## Related Documentation
- [Action Management API](./api-actions.md)
- [Rule Management API](./api-rules.md)
- [Enforcement API](./api-enforcements.md)
- [Worker Service](./worker-service.md)
- [Executor Service](./executor-service.md)
---
**Last Updated:** January 13, 2026

790
docs/api/api-inquiries.md Normal file
View File

@@ -0,0 +1,790 @@
# Inquiry Management API
The Inquiry Management API provides endpoints for managing human-in-the-loop interactions within Attune workflows. Inquiries allow executions to pause and request input from users before continuing, enabling approval workflows, data collection, and interactive automation.
## Table of Contents
- [Overview](#overview)
- [Inquiry Model](#inquiry-model)
- [Authentication](#authentication)
- [Endpoints](#endpoints)
- [List Inquiries](#list-inquiries)
- [Get Inquiry by ID](#get-inquiry-by-id)
- [List Inquiries by Status](#list-inquiries-by-status)
- [List Inquiries by Execution](#list-inquiries-by-execution)
- [Create Inquiry](#create-inquiry)
- [Update Inquiry](#update-inquiry)
- [Respond to Inquiry](#respond-to-inquiry)
- [Delete Inquiry](#delete-inquiry)
- [Use Cases](#use-cases)
- [Related Resources](#related-resources)
---
## Overview
Inquiries represent questions or prompts that require human input during workflow execution. They support:
- **Approval Workflows**: Request approval before proceeding with critical actions
- **Data Collection**: Gather additional information from users during execution
- **Interactive Automation**: Enable dynamic workflows that adapt based on user input
- **Assignment**: Direct inquiries to specific users or teams
- **Timeouts**: Automatically expire inquiries after a specified time
- **Schema Validation**: Define expected response formats using JSON Schema
### Key Features
- **Status Tracking**: Monitor inquiry lifecycle (pending, responded, timeout, canceled)
- **Response Validation**: Optionally validate responses against JSON schemas
- **User Assignment**: Assign inquiries to specific users for accountability
- **Timeout Handling**: Automatically handle expired inquiries
- **Execution Integration**: Link inquiries to specific workflow executions
---
## Inquiry Model
### Inquiry Object
```json
{
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"},
"comment": {"type": "string"}
},
"required": ["approved"]
},
"assigned_to": 789,
"status": "pending",
"response": null,
"timeout_at": "2024-01-15T12:00:00Z",
"responded_at": null,
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
}
```
### Fields
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Unique inquiry identifier |
| `execution` | integer | ID of the execution this inquiry belongs to |
| `prompt` | string | Question or prompt text displayed to the user |
| `response_schema` | object | Optional JSON Schema defining expected response format |
| `assigned_to` | integer | Optional user ID this inquiry is assigned to |
| `status` | string | Current status: `pending`, `responded`, `timeout`, `canceled` |
| `response` | object | User's response data (null until responded) |
| `timeout_at` | datetime | Optional timestamp when inquiry expires |
| `responded_at` | datetime | Timestamp when user responded (null until responded) |
| `created` | datetime | Timestamp when inquiry was created |
| `updated` | datetime | Timestamp of last update |
### Inquiry Status
| Status | Description |
|--------|-------------|
| `pending` | Inquiry is waiting for a response |
| `responded` | User has provided a response |
| `timeout` | Inquiry expired without receiving a response |
| `canceled` | Inquiry was canceled before completion |
---
## Authentication
All inquiry endpoints require authentication. Include a valid JWT access token in the `Authorization` header:
```
Authorization: Bearer <access_token>
```
See the [Authentication Guide](./authentication.md) for details on obtaining tokens.
---
## Endpoints
### List Inquiries
Retrieve a paginated list of inquiries with optional filtering.
**Endpoint:** `GET /api/v1/inquiries`
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `status` | string | - | Filter by status (`pending`, `responded`, `timeout`, `canceled`) |
| `execution` | integer | - | Filter by execution ID |
| `assigned_to` | integer | - | Filter by assigned user ID |
| `page` | integer | 1 | Page number (1-indexed) |
| `per_page` | integer | 50 | Items per page (max 100) |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/inquiries?status=pending&page=1&per_page=20" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": [
{
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"assigned_to": 789,
"status": "pending",
"has_response": false,
"timeout_at": "2024-01-15T12:00:00Z",
"created": "2024-01-15T10:00:00Z"
}
],
"meta": {
"page": 1,
"page_size": 20,
"total": 1,
"total_pages": 1
}
}
```
---
### Get Inquiry by ID
Retrieve a single inquiry by its ID.
**Endpoint:** `GET /api/v1/inquiries/:id`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `id` | integer | Inquiry ID |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/inquiries/123" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": {
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"},
"comment": {"type": "string"}
},
"required": ["approved"]
},
"assigned_to": 789,
"status": "pending",
"response": null,
"timeout_at": "2024-01-15T12:00:00Z",
"responded_at": null,
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
}
}
```
**Error Responses:**
- `404 Not Found`: Inquiry not found
---
### List Inquiries by Status
Retrieve inquiries filtered by status.
**Endpoint:** `GET /api/v1/inquiries/status/:status`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `status` | string | Status filter: `pending`, `responded`, `timeout`, `canceled` |
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `page` | integer | 1 | Page number |
| `page_size` | integer | 50 | Items per page |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/inquiries/status/pending" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": [
{
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"assigned_to": 789,
"status": "pending",
"has_response": false,
"timeout_at": "2024-01-15T12:00:00Z",
"created": "2024-01-15T10:00:00Z"
}
],
"meta": {
"page": 1,
"page_size": 50,
"total": 1,
"total_pages": 1
}
}
```
**Error Responses:**
- `400 Bad Request`: Invalid status value
---
### List Inquiries by Execution
Retrieve all inquiries associated with a specific execution.
**Endpoint:** `GET /api/v1/executions/:execution_id/inquiries`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `execution_id` | integer | Execution ID |
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `page` | integer | 1 | Page number |
| `page_size` | integer | 50 | Items per page |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/executions/456/inquiries" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": [
{
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"assigned_to": 789,
"status": "pending",
"has_response": false,
"timeout_at": "2024-01-15T12:00:00Z",
"created": "2024-01-15T10:00:00Z"
}
],
"meta": {
"page": 1,
"page_size": 50,
"total": 1,
"total_pages": 1
}
}
```
**Error Responses:**
- `404 Not Found`: Execution not found
---
### Create Inquiry
Create a new inquiry for an execution.
**Endpoint:** `POST /api/v1/inquiries`
**Request Body:**
```json
{
"execution": 456,
"prompt": "Approve deployment to production?",
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"},
"comment": {"type": "string"}
},
"required": ["approved"]
},
"assigned_to": 789,
"timeout_at": "2024-01-15T12:00:00Z"
}
```
**Field Validation:**
| Field | Required | Constraints |
|-------|----------|-------------|
| `execution` | Yes | Must be a valid execution ID |
| `prompt` | Yes | 1-10,000 characters |
| `response_schema` | No | Valid JSON Schema object |
| `assigned_to` | No | Valid user ID |
| `timeout_at` | No | ISO 8601 datetime in the future |
**Example Request:**
```bash
curl -X POST "http://localhost:8080/api/v1/inquiries" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"execution": 456,
"prompt": "Approve deployment to production?",
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"}
}
},
"timeout_at": "2024-01-15T12:00:00Z"
}'
```
**Response:** `201 Created`
```json
{
"data": {
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"}
}
},
"assigned_to": null,
"status": "pending",
"response": null,
"timeout_at": "2024-01-15T12:00:00Z",
"responded_at": null,
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
},
"message": "Inquiry created successfully"
}
```
**Error Responses:**
- `400 Bad Request`: Validation error
- `404 Not Found`: Execution not found
---
### Update Inquiry
Update an existing inquiry's properties.
**Endpoint:** `PUT /api/v1/inquiries/:id`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `id` | integer | Inquiry ID |
**Request Body:**
```json
{
"status": "canceled",
"assigned_to": 999
}
```
**Updatable Fields:**
| Field | Type | Description |
|-------|------|-------------|
| `status` | string | Update status (any valid status) |
| `response` | object | Manually set response data |
| `assigned_to` | integer | Change assignment |
**Example Request:**
```bash
curl -X PUT "http://localhost:8080/api/v1/inquiries/123" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"status": "canceled"
}'
```
**Response:** `200 OK`
```json
{
"data": {
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"response_schema": null,
"assigned_to": null,
"status": "canceled",
"response": null,
"timeout_at": "2024-01-15T12:00:00Z",
"responded_at": null,
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:05:00Z"
},
"message": "Inquiry updated successfully"
}
```
**Error Responses:**
- `404 Not Found`: Inquiry not found
- `400 Bad Request`: Validation error
---
### Respond to Inquiry
Submit a response to a pending inquiry. This is the primary endpoint for users to answer inquiries.
**Endpoint:** `POST /api/v1/inquiries/:id/respond`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `id` | integer | Inquiry ID |
**Request Body:**
```json
{
"response": {
"approved": true,
"comment": "Deployment approved for production release v2.1.0"
}
}
```
**Behavior:**
- Only `pending` inquiries can be responded to
- If inquiry has `assigned_to`, only that user can respond
- If inquiry has timed out, response is rejected
- Automatically sets `status` to `responded`
- Automatically sets `responded_at` timestamp
- Response should conform to `response_schema` if defined (future validation)
**Example Request:**
```bash
curl -X POST "http://localhost:8080/api/v1/inquiries/123/respond" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"response": {
"approved": true,
"comment": "Looks good, proceeding with deployment"
}
}'
```
**Response:** `200 OK`
```json
{
"data": {
"id": 123,
"execution": 456,
"prompt": "Approve deployment to production?",
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"},
"comment": {"type": "string"}
}
},
"assigned_to": 789,
"status": "responded",
"response": {
"approved": true,
"comment": "Looks good, proceeding with deployment"
},
"timeout_at": "2024-01-15T12:00:00Z",
"responded_at": "2024-01-15T10:30:00Z",
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:30:00Z"
},
"message": "Response submitted successfully"
}
```
**Error Responses:**
- `404 Not Found`: Inquiry not found
- `400 Bad Request`: Inquiry is not in pending status or has timed out
- `403 Forbidden`: User is not authorized to respond (inquiry assigned to someone else)
---
### Delete Inquiry
Delete an inquiry.
**Endpoint:** `DELETE /api/v1/inquiries/:id`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `id` | integer | Inquiry ID |
**Example Request:**
```bash
curl -X DELETE "http://localhost:8080/api/v1/inquiries/123" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"message": "Inquiry deleted successfully",
"success": true
}
```
**Error Responses:**
- `404 Not Found`: Inquiry not found
---
## Use Cases
### Approval Workflows
Create inquiries to request approval before executing critical actions:
```bash
# Create an approval inquiry
curl -X POST "http://localhost:8080/api/v1/inquiries" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"execution": 456,
"prompt": "Approve deletion of production database backup?",
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"},
"reason": {"type": "string"}
},
"required": ["approved", "reason"]
},
"assigned_to": 789,
"timeout_at": "2024-01-15T18:00:00Z"
}'
```
### Data Collection
Gather additional information during workflow execution:
```bash
# Request deployment details
curl -X POST "http://localhost:8080/api/v1/inquiries" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"execution": 456,
"prompt": "Enter deployment configuration",
"response_schema": {
"type": "object",
"properties": {
"environment": {"type": "string", "enum": ["staging", "production"]},
"replicas": {"type": "integer", "minimum": 1, "maximum": 10},
"rollback_enabled": {"type": "boolean"}
},
"required": ["environment", "replicas"]
}
}'
```
### Monitoring Pending Inquiries
List all pending inquiries requiring attention:
```bash
# Get all pending inquiries
curl -X GET "http://localhost:8080/api/v1/inquiries?status=pending" \
-H "Authorization: Bearer <token>"
# Get inquiries assigned to a specific user
curl -X GET "http://localhost:8080/api/v1/inquiries?status=pending&assigned_to=789" \
-H "Authorization: Bearer <token>"
```
### Responding to Inquiries
Users can respond to assigned inquiries:
```bash
curl -X POST "http://localhost:8080/api/v1/inquiries/123/respond" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"response": {
"environment": "production",
"replicas": 5,
"rollback_enabled": true
}
}'
```
---
## Best Practices
### 1. Use Response Schemas
Define clear response schemas to validate user input:
```json
{
"response_schema": {
"type": "object",
"properties": {
"approved": {"type": "boolean"},
"justification": {"type": "string", "minLength": 10}
},
"required": ["approved", "justification"]
}
}
```
### 2. Set Reasonable Timeouts
Always set timeouts to prevent inquiries from blocking workflows indefinitely:
```json
{
"timeout_at": "2024-01-15T23:59:59Z"
}
```
### 3. Assign to Specific Users
For accountability, assign inquiries to responsible users:
```json
{
"assigned_to": 789,
"prompt": "Review and approve security patch deployment"
}
```
### 4. Handle Timeouts Gracefully
Monitor inquiries and handle timeout status appropriately in your workflows.
### 5. Provide Clear Prompts
Write descriptive prompts that clearly explain what information is needed:
```json
{
"prompt": "The deployment to production requires approval. Review the changes at https://github.com/org/repo/pull/123 and approve or reject."
}
```
---
## Error Handling
### Common Error Codes
| Status Code | Description |
|-------------|-------------|
| `400 Bad Request` | Invalid input or inquiry not in correct state |
| `401 Unauthorized` | Missing or invalid authentication token |
| `403 Forbidden` | User not authorized to respond to inquiry |
| `404 Not Found` | Inquiry or execution not found |
| `422 Unprocessable Entity` | Validation errors |
| `500 Internal Server Error` | Server error |
### Example Error Response
```json
{
"error": "Cannot respond to inquiry with status 'responded'. Only pending inquiries can be responded to.",
"status": 400
}
```
---
## Related Resources
- [Execution Management API](./api-executions.md) - Manage workflow executions
- [Action Management API](./api-actions.md) - Define executable actions
- [Rule Management API](./api-rules.md) - Create automation rules
- [Authentication Guide](./authentication.md) - API authentication details
---
## Future Enhancements
### Planned Features
1. **Response Schema Validation**: Automatic validation of responses against JSON Schema
2. **Inquiry Templates**: Reusable inquiry templates for common patterns
3. **Batch Operations**: Respond to multiple inquiries at once
4. **Notification Integration**: Automatic notifications when inquiries are created
5. **Audit Trail**: Detailed logging of inquiry lifecycle events
6. **Custom Actions**: Trigger actions on inquiry state changes
7. **WebSocket Updates**: Real-time inquiry status updates
---
**Last Updated:** 2024-01-15
**API Version:** v1

646
docs/api/api-pack-testing.md Normal file
View File

@@ -0,0 +1,646 @@
# Pack Testing API Endpoints
**API endpoints for executing and retrieving pack test results**
---
## Overview
The Pack Testing API enables programmatic execution of pack tests and retrieval of test history. Tests are executed using the pack's `pack.yaml` configuration and results are stored in the database for audit and monitoring purposes.
**Base Path**: `/api/v1/packs/{ref}/`
---
## Endpoints
### 1. Execute Pack Tests
Execute all tests defined in a pack's `pack.yaml` configuration.
**Endpoint**: `POST /api/v1/packs/{ref}/test`
**Authentication**: Required (Bearer token)
**Path Parameters**:
- `ref` (string, required): Pack reference identifier
**Request Body**: None
**Response**: `200 OK`
```json
{
"data": {
"packRef": "core",
"packVersion": "1.0.0",
"executionTime": "2026-01-22T03:30:00Z",
"totalTests": 2,
"passed": 2,
"failed": 0,
"skipped": 0,
"passRate": 1.0,
"durationMs": 25542,
"testSuites": [
{
"name": "shell",
"runnerType": "script",
"total": 1,
"passed": 1,
"failed": 0,
"skipped": 0,
"durationMs": 12305,
"testCases": [
{
"name": "shell_suite",
"status": "passed",
"durationMs": 12305,
"errorMessage": null,
"stdout": "...",
"stderr": null
}
]
},
{
"name": "python",
"runnerType": "unittest",
"total": 1,
"passed": 1,
"failed": 0,
"skipped": 0,
"durationMs": 13235,
"testCases": [
{
"name": "python_suite",
"status": "passed",
"durationMs": 13235,
"errorMessage": null,
"stdout": null,
"stderr": "..."
}
]
}
]
},
"message": "Pack tests executed successfully"
}
```
**Error Responses**:
- `400 Bad Request`: Testing not enabled or no test configuration found
```json
{
"error": "No testing configuration found in pack.yaml"
}
```
- `404 Not Found`: Pack not found
```json
{
"error": "Pack 'my_pack' not found"
}
```
- `500 Internal Server Error`: Test execution failed
```json
{
"error": "Test execution failed: timeout after 120s"
}
```
**Example**:
```bash
# Execute tests for core pack
curl -X POST http://localhost:8080/api/v1/packs/core/test \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json"
```
**Behavior**:
1. Loads pack from database
2. Reads `pack.yaml` from filesystem
3. Parses test configuration
4. Executes test suites (shell, python, etc.)
5. Stores results in database
6. Returns structured test results
**Notes**:
- Test results are stored with `trigger_reason: "manual"`
- Tests run synchronously (blocking request)
- Large test suites may timeout (consider async execution in future)
---
### 2. Get Pack Test History
Retrieve paginated test execution history for a pack.
**Endpoint**: `GET /api/v1/packs/{ref}/tests`
**Authentication**: Required (Bearer token)
**Path Parameters**:
- `ref` (string, required): Pack reference identifier
**Query Parameters**:
- `page` (integer, optional, default: 1): Page number (1-based)
- `limit` (integer, optional, default: 20, max: 100): Items per page
**Response**: `200 OK`
```json
{
"data": [
{
"id": 123,
"packId": 1,
"packVersion": "1.0.0",
"executionTime": "2026-01-22T03:30:00Z",
"triggerReason": "manual",
"totalTests": 74,
"passed": 74,
"failed": 0,
"skipped": 0,
"passRate": 1.0,
"durationMs": 25542,
"result": { ... },
"created": "2026-01-22T03:30:00Z"
},
{
"id": 122,
"packId": 1,
"packVersion": "1.0.0",
"executionTime": "2026-01-21T10:15:00Z",
"triggerReason": "install",
"totalTests": 74,
"passed": 73,
"failed": 1,
"skipped": 0,
"passRate": 0.986,
"durationMs": 28100,
"result": { ... },
"created": "2026-01-21T10:15:00Z"
}
],
"pagination": {
"page": 1,
"limit": 20,
"total": 45,
"totalPages": 3
}
}
```
**Error Responses**:
- `404 Not Found`: Pack not found
```json
{
"error": "Pack 'my_pack' not found"
}
```
**Example**:
```bash
# Get first page of test history
curl -X GET "http://localhost:8080/api/v1/packs/core/tests?page=1&limit=10" \
-H "Authorization: Bearer $TOKEN"
# Get second page
curl -X GET "http://localhost:8080/api/v1/packs/core/tests?page=2&limit=10" \
-H "Authorization: Bearer $TOKEN"
```
**Notes**:
- Results are ordered by execution time (newest first)
- Full test result JSON is included in `result` field
- Trigger reasons: `manual`, `install`, `update`, `validation`
---
### 3. Get Latest Pack Test Result
Retrieve the most recent test execution for a pack.
**Endpoint**: `GET /api/v1/packs/{ref}/tests/latest`
**Authentication**: Required (Bearer token)
**Path Parameters**:
- `ref` (string, required): Pack reference identifier
**Response**: `200 OK`
```json
{
"data": {
"id": 123,
"packId": 1,
"packVersion": "1.0.0",
"executionTime": "2026-01-22T03:30:00Z",
"triggerReason": "manual",
"totalTests": 74,
"passed": 74,
"failed": 0,
"skipped": 0,
"passRate": 1.0,
"durationMs": 25542,
"result": {
"packRef": "core",
"packVersion": "1.0.0",
"executionTime": "2026-01-22T03:30:00Z",
"totalTests": 2,
"passed": 2,
"failed": 0,
"skipped": 0,
"passRate": 1.0,
"durationMs": 25542,
"testSuites": [ ... ]
},
"created": "2026-01-22T03:30:00Z"
}
}
```
**Error Responses**:
- `404 Not Found`: Pack not found or no tests available
```json
{
"error": "No test results found for pack 'my_pack'"
}
```
**Example**:
```bash
# Get latest test result for core pack
curl -X GET http://localhost:8080/api/v1/packs/core/tests/latest \
-H "Authorization: Bearer $TOKEN"
# Check if tests are passing
curl -s -X GET http://localhost:8080/api/v1/packs/core/tests/latest \
-H "Authorization: Bearer $TOKEN" | jq '.data.passRate'
```
**Use Cases**:
- Health monitoring dashboards
- CI/CD pipeline validation
- Pack quality badges
- Automated alerts on test failures
---
## Data Models
### PackTestResult
Main test execution result structure.
```typescript
{
packRef: string; // Pack reference identifier
packVersion: string; // Pack version tested
executionTime: string; // ISO 8601 timestamp
totalTests: number; // Total number of tests
passed: number; // Number of passed tests
failed: number; // Number of failed tests
skipped: number; // Number of skipped tests
passRate: number; // Pass rate (0.0 to 1.0)
durationMs: number; // Total duration in milliseconds
testSuites: TestSuiteResult[]; // Test suites executed
}
```
### TestSuiteResult
Result for a single test suite (e.g., shell, python).
```typescript
{
name: string; // Suite name (shell, python, etc.)
runnerType: string; // Runner type (script, unittest, pytest)
total: number; // Total tests in suite
passed: number; // Passed tests
failed: number; // Failed tests
skipped: number; // Skipped tests
durationMs: number; // Suite duration in milliseconds
testCases: TestCaseResult[]; // Individual test cases
}
```
### TestCaseResult
Individual test case result.
```typescript
{
name: string; // Test case name
status: TestStatus; // "passed" | "failed" | "skipped" | "error"
durationMs: number; // Test duration in milliseconds
errorMessage?: string; // Error message if failed
stdout?: string; // Standard output (if captured)
stderr?: string; // Standard error (if captured)
}
```
### PackTestExecution
Database record of test execution.
```typescript
{
id: number; // Execution ID
packId: number; // Pack ID (foreign key)
packVersion: string; // Pack version
executionTime: string; // When tests were run
triggerReason: string; // "manual" | "install" | "update" | "validation"
totalTests: number; // Total number of tests
passed: number; // Passed tests
failed: number; // Failed tests
skipped: number; // Skipped tests
passRate: number; // Pass rate (0.0 to 1.0)
durationMs: number; // Duration in milliseconds
result: object; // Full PackTestResult JSON
created: string; // Record creation timestamp
}
```
---
## Usage Examples
### 1. Run Tests Before Deployment
```bash
#!/bin/bash
# test-and-deploy.sh
PACK="my_pack"
API_URL="http://localhost:8080/api/v1"
# Execute tests
RESULT=$(curl -s -X POST "$API_URL/packs/$PACK/test" \
-H "Authorization: Bearer $TOKEN")
# Check pass rate
PASS_RATE=$(echo $RESULT | jq -r '.data.passRate')
if (( $(echo "$PASS_RATE >= 1.0" | bc -l) )); then
echo "✅ All tests passed, deploying..."
# Deploy pack
else
echo "❌ Tests failed (pass rate: $PASS_RATE)"
exit 1
fi
```
### 2. Monitor Pack Quality
```bash
#!/bin/bash
# monitor-pack-quality.sh
PACKS=("core" "aws" "kubernetes")
for PACK in "${PACKS[@]}"; do
LATEST=$(curl -s -X GET "$API_URL/packs/$PACK/tests/latest" \
-H "Authorization: Bearer $TOKEN")
PASS_RATE=$(echo $LATEST | jq -r '.data.passRate')
FAILED=$(echo $LATEST | jq -r '.data.failed')
if [ "$FAILED" -gt 0 ]; then
echo "⚠️ $PACK has $FAILED failing tests (pass rate: $PASS_RATE)"
else
echo "✅ $PACK: All tests passing"
fi
done
```
### 3. Get Test Trend Data
```bash
#!/bin/bash
# get-test-trend.sh
PACK="core"
# Get last 10 test executions
HISTORY=$(curl -s -X GET "$API_URL/packs/$PACK/tests?limit=10" \
-H "Authorization: Bearer $TOKEN")
# Extract pass rates
echo $HISTORY | jq -r '.data[] | "\(.executionTime): \(.passRate)"'
```
### 4. JavaScript/TypeScript Integration
```typescript
// pack-test-client.ts
interface PackTestClient {
async executeTests(packRef: string): Promise<PackTestResult>;
async getTestHistory(packRef: string, page?: number): Promise<PaginatedResponse<PackTestExecution>>;
async getLatestTest(packRef: string): Promise<PackTestExecution>;
}
class AttunePackTestClient implements PackTestClient {
constructor(
private apiUrl: string,
private token: string
) {}
async executeTests(packRef: string): Promise<PackTestResult> {
const response = await fetch(
`${this.apiUrl}/packs/${packRef}/test`,
{
method: 'POST',
headers: {
'Authorization': `Bearer ${this.token}`,
'Content-Type': 'application/json'
}
}
);
if (!response.ok) {
throw new Error(`Test execution failed: ${response.statusText}`);
}
const { data } = await response.json();
return data;
}
async getLatestTest(packRef: string): Promise<PackTestExecution> {
const response = await fetch(
`${this.apiUrl}/packs/${packRef}/tests/latest`,
{
headers: {
'Authorization': `Bearer ${this.token}`
}
}
);
if (!response.ok) {
throw new Error(`Failed to get latest test: ${response.statusText}`);
}
const { data } = await response.json();
return data;
}
}
// Usage
const client = new AttunePackTestClient(
'http://localhost:8080/api/v1',
process.env.ATTUNE_TOKEN
);
const result = await client.executeTests('core');
console.log(`Pass rate: ${result.passRate * 100}%`);
```
---
## Best Practices
### 1. Run Tests in CI/CD
Always run pack tests before deploying:
```yaml
# .github/workflows/test-pack.yml
name: Test Pack
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Test Pack
run: |
curl -X POST ${{ secrets.ATTUNE_API }}/packs/my_pack/test \
-H "Authorization: Bearer ${{ secrets.ATTUNE_TOKEN }}" \
-f # Fail on HTTP errors
```
### 2. Monitor Test Trends
Track test pass rates over time:
```javascript
// Store metrics in monitoring system
const latest = await client.getLatestTest('core');
metrics.gauge('pack.test.pass_rate', latest.passRate, {
pack: 'core',
version: latest.packVersion
});
metrics.gauge('pack.test.duration_ms', latest.durationMs, {
pack: 'core'
});
```
### 3. Alert on Test Failures
Set up alerts for failing tests:
```javascript
const latest = await client.getLatestTest('core');
if (latest.failed > 0) {
await slack.sendMessage({
channel: '#alerts',
text: `⚠️ Pack 'core' has ${latest.failed} failing tests!`,
attachments: [{
color: 'danger',
fields: [
{ title: 'Pass Rate', value: `${latest.passRate * 100}%` },
{ title: 'Failed', value: latest.failed },
{ title: 'Duration', value: `${latest.durationMs}ms` }
]
}]
});
}
```
### 4. Use Timeouts
Test execution can take time. Use appropriate timeouts:
```bash
# 5 minute timeout for test execution
timeout 300 curl -X POST "$API_URL/packs/my_pack/test" \
-H "Authorization: Bearer $TOKEN"
```
---
## Troubleshooting
### Tests Always Fail
**Problem**: Tests fail even though they work locally
**Solutions**:
1. Check pack.yaml testing configuration is correct
2. Verify test files exist and are executable
3. Check dependencies are available in API environment
4. Review test logs in database `result` field
### Timeout Errors
**Problem**: Test execution times out
**Solutions**:
1. Increase timeout in pack.yaml runners
2. Split tests into multiple suites
3. Mock slow external dependencies
4. Consider async test execution (future feature)
### Missing Test Results
**Problem**: No test history available
**Solutions**:
1. Run tests at least once: `POST /packs/{ref}/test`
2. Check pack exists in database
3. Verify database migrations have run
---
## Related Documentation
- **Pack Testing Framework**: `docs/pack-testing-framework.md`
- **Pack Testing Guide**: `docs/PACK_TESTING.md`
- **Core Pack Tests**: `packs/core/tests/README.md`
- **API Reference**: `docs/api-reference.md`
- **Database Schema**: `migrations/012_add_pack_test_results.sql`
---
## Future Enhancements
- [ ] Async test execution (return job ID, poll for results)
- [ ] Webhooks for test completion
- [ ] Test result comparison (diff between runs)
- [ ] Test coverage metrics
- [ ] Performance regression detection
- [ ] Scheduled test execution
- [ ] Test result retention policies
---
## Changelog
- **2026-01-22**: Initial implementation
- POST /packs/{ref}/test - Execute tests
- GET /packs/{ref}/tests - Get test history
- GET /packs/{ref}/tests/latest - Get latest test result

402
docs/api/api-pack-workflows.md Normal file
View File

@@ -0,0 +1,402 @@
# Pack Workflow API Documentation
This document describes the pack workflow integration endpoints that enable automatic loading, validation, and synchronization of workflow definitions from pack directories.
## Overview
The Pack Workflow API provides endpoints to:
- Automatically sync workflows from filesystem when packs are created/updated
- Manually trigger workflow synchronization for a pack
- Validate workflow definitions without registering them
- Integrate workflow lifecycle with pack management
## Endpoints
### Sync Pack Workflows
Synchronizes workflow definitions from the filesystem to the database for a specific pack.
**Endpoint:** `POST /api/v1/packs/{ref}/workflows/sync`
**Authentication:** Required (Bearer token)
**Path Parameters:**
- `ref` (string, required): Pack reference identifier
**Response:** `200 OK`
```json
{
"data": {
"pack_ref": "my_pack",
"loaded_count": 5,
"registered_count": 5,
"workflows": [
{
"ref_name": "my_pack.deploy_app",
"created": true,
"workflow_def_id": 123,
"warnings": []
}
],
"errors": []
},
"message": "Pack workflows synced successfully"
}
```
**Response Fields:**
- `pack_ref`: The pack reference that was synced
- `loaded_count`: Number of workflow files found and loaded from filesystem
- `registered_count`: Number of workflows successfully registered/updated in database
- `workflows`: Array of individual workflow sync results
- `ref_name`: Full workflow reference (pack.workflow_name)
- `created`: Whether workflow was created (true) or updated (false)
- `workflow_def_id`: Database ID of the workflow definition
- `warnings`: Any validation warnings (workflow still registered)
- `errors`: Array of error messages (workflows that failed to sync)
**Error Responses:**
- `404 Not Found`: Pack does not exist
- `401 Unauthorized`: Missing or invalid authentication token
- `500 Internal Server Error`: Failed to sync workflows
**Example:**
```bash
curl -X POST http://localhost:8080/api/v1/packs/core/workflows/sync \
-H "Authorization: Bearer $TOKEN"
```
---
### Validate Pack Workflows
Validates workflow definitions from the filesystem without registering them in the database. Useful for checking workflow syntax and structure before deployment.
**Endpoint:** `POST /api/v1/packs/{ref}/workflows/validate`
**Authentication:** Required (Bearer token)
**Path Parameters:**
- `ref` (string, required): Pack reference identifier
**Response:** `200 OK`
```json
{
"data": {
"pack_ref": "my_pack",
"validated_count": 5,
"error_count": 1,
"errors": {
"my_pack.broken_workflow": [
"Missing required field: version",
"Task 'step1' references undefined action 'invalid.action'"
]
}
},
"message": "Pack workflows validated"
}
```
**Response Fields:**
- `pack_ref`: The pack reference that was validated
- `validated_count`: Total number of workflow files validated
- `error_count`: Number of workflows with validation errors
- `errors`: Map of workflow references to their validation error messages
**Error Responses:**
- `404 Not Found`: Pack does not exist
- `401 Unauthorized`: Missing or invalid authentication token
- `500 Internal Server Error`: Failed to validate workflows
**Example:**
```bash
curl -X POST http://localhost:8080/api/v1/packs/core/workflows/validate \
-H "Authorization: Bearer $TOKEN"
```
---
## Automatic Workflow Synchronization
Workflows are automatically synchronized in the following scenarios:
### Pack Creation
When a pack is created via `POST /api/v1/packs`, the system automatically attempts to load and register workflows from the pack's `workflows/` directory.
**Example:**
```bash
curl -X POST http://localhost:8080/api/v1/packs \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "my_pack",
"label": "My Pack",
"version": "1.0.0",
"description": "A custom pack"
}'
```
If workflows exist in `/opt/attune/packs/my_pack/workflows/`, they will be automatically loaded and registered.
### Pack Update
When a pack is updated via `PUT /api/v1/packs/{ref}`, the system automatically resyncs workflows to capture any changes.
**Example:**
```bash
curl -X PUT http://localhost:8080/api/v1/packs/my_pack \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"version": "1.1.0",
"label": "My Pack (Updated)"
}'
```
**Note:** Auto-sync uses `skip_validation_errors: true`, so pack operations won't fail if workflows have validation errors. Use the validate endpoint to check for errors.
---
## Workflow Directory Structure
Workflows must be placed in the `workflows/` subdirectory of the pack directory:
```
/opt/attune/packs/
└── my_pack/
├── actions/
├── sensors/
└── workflows/
├── deploy_app.yaml
├── rollback.yaml
└── health_check.yml
```
**Workflow File Requirements:**
- Must be in YAML format (`.yaml` or `.yml` extension)
- Filename (without extension) becomes the workflow name
- Full workflow reference is `pack_ref.workflow_name`
- Must conform to the workflow schema (see Workflow API documentation)
---
## Configuration
The pack workflows feature uses the following configuration:
**Config File (`config.yaml`):**
```yaml
packs_base_dir: "/opt/attune/packs" # Base directory for pack directories
```
**Environment Variable:**
```bash
export ATTUNE__PACKS_BASE_DIR="/opt/attune/packs"
```
Default: `/opt/attune/packs`
---
## Workflow Lifecycle
1. **Development:** Create workflow YAML files in pack's `workflows/` directory
2. **Validation:** Use validate endpoint to check for errors
3. **Deployment:** Create/update pack via API (auto-syncs workflows)
4. **Manual Sync:** Use sync endpoint to reload workflows after filesystem changes
5. **Execution:** Workflows become available for execution via workflow API
---
## Best Practices
### 1. Validate Before Deploy
Always validate workflows before deploying:
```bash
# Validate workflows
curl -X POST http://localhost:8080/api/v1/packs/my_pack/workflows/validate \
-H "Authorization: Bearer $TOKEN"
# If validation passes, sync
curl -X POST http://localhost:8080/api/v1/packs/my_pack/workflows/sync \
-H "Authorization: Bearer $TOKEN"
```
### 2. Version Control
Keep workflow YAML files in version control alongside pack code:
```
my_pack/
├── actions/
│ └── deploy.py
├── workflows/
│ └── deploy_app.yaml
└── pack.yaml
```
### 3. Naming Conventions
Use descriptive workflow filenames that indicate their purpose:
- `deploy_production.yaml`
- `backup_database.yaml`
- `incident_response.yaml`
### 4. Handle Sync Errors
Check the `errors` field in sync responses:
```json
{
"data": {
"errors": [
"workflow 'my_pack.invalid' failed validation: Missing required field"
]
}
}
```
### 5. Incremental Updates
When updating workflows:
1. Modify YAML files on filesystem
2. Call sync endpoint to reload
3. Previous workflow versions are updated (not duplicated)
---
## Error Handling
### Common Errors
**Pack Not Found (404):**
```json
{
"error": "Pack 'nonexistent_pack' not found"
}
```
**Validation Errors:**
Workflows with validation errors are reported but don't prevent sync:
```json
{
"data": {
"workflows": [...],
"errors": [
"Validation failed for my_pack.broken: Missing tasks field"
]
}
}
```
**Filesystem Access:**
If pack directory doesn't exist on filesystem:
```json
{
"data": {
"loaded_count": 0,
"registered_count": 0,
"errors": ["Failed to load workflows: Directory not found"]
}
}
```
---
## Integration Examples
### CI/CD Pipeline
```bash
#!/bin/bash
# deploy-pack.sh
PACK_NAME="my_pack"
PACK_VERSION="1.0.0"
API_URL="http://localhost:8080"
# 1. Validate workflows locally
echo "Validating workflows..."
response=$(curl -s -X POST "$API_URL/api/v1/packs/$PACK_NAME/workflows/validate" \
-H "Authorization: Bearer $TOKEN")
error_count=$(echo $response | jq -r '.data.error_count')
if [ "$error_count" -gt 0 ]; then
echo "Validation errors found:"
echo $response | jq '.data.errors'
exit 1
fi
# 2. Create or update pack
echo "Deploying pack..."
curl -X POST "$API_URL/api/v1/packs" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d "{
\"ref\": \"$PACK_NAME\",
\"label\": \"My Pack\",
\"version\": \"$PACK_VERSION\"
}"
# 3. Verify sync
echo "Syncing workflows..."
curl -X POST "$API_URL/api/v1/packs/$PACK_NAME/workflows/sync" \
-H "Authorization: Bearer $TOKEN"
echo "Deployment complete!"
```
### Development Workflow
```python
import requests
API_URL = "http://localhost:8080"
TOKEN = "your_access_token"
def sync_workflows(pack_ref: str):
"""Sync workflows after local changes."""
response = requests.post(
f"{API_URL}/api/v1/packs/{pack_ref}/workflows/sync",
headers={"Authorization": f"Bearer {TOKEN}"}
)
data = response.json()["data"]
print(f"Synced {data['registered_count']} workflows")
if data['errors']:
print("Errors:", data['errors'])
return data
# Usage
sync_workflows("my_pack")
```
---
## Related Documentation
- [Workflow API](api-workflows.md) - Workflow definition management
- [Pack API](api-packs.md) - Pack management endpoints
- [Workflow Orchestration](workflow-orchestration.md) - Workflow execution and concepts
---
## Changelog
- **v0.1.0** (2024-01): Initial implementation of pack workflow integration
- Auto-sync on pack create/update
- Manual sync endpoint
- Validation endpoint

752
docs/api/api-packs.md Normal file
View File

@@ -0,0 +1,752 @@
# Pack Management API
This document provides comprehensive documentation for the Pack Management API endpoints in Attune.
## Overview
Packs are containers that bundle related automation components (actions, triggers, rules, and sensors) together. They provide:
- **Organization**: Group related automation components by domain or service
- **Versioning**: Track and manage versions of automation components
- **Configuration**: Define pack-level configuration schemas and defaults
- **Dependencies**: Declare runtime and pack dependencies
- **Metadata**: Store tags, descriptions, and other metadata
## Pack Data Model
### Pack Structure
```json
{
"id": 1,
"ref": "aws.ec2",
"label": "AWS EC2 Pack",
"description": "Actions and triggers for AWS EC2 management",
"version": "1.0.0",
"conf_schema": {
"type": "object",
"properties": {
"region": {"type": "string"},
"access_key_id": {"type": "string"},
"secret_access_key": {"type": "string"}
},
"required": ["region"]
},
"config": {
"region": "us-east-1"
},
"meta": {
"author": "Attune Team",
"license": "MIT"
},
"tags": ["aws", "ec2", "cloud"],
"runtime_deps": [1, 2],
"is_standard": true,
"created": "2024-01-15T10:30:00Z",
"updated": "2024-01-15T10:30:00Z"
}
```
### Field Descriptions
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `id` | integer | auto | Unique pack identifier |
| `ref` | string | yes | Unique reference (e.g., "aws.ec2", "core.http") |
| `label` | string | yes | Human-readable pack name |
| `description` | string | yes | Pack description and purpose |
| `version` | string | no | Semantic version (e.g., "1.0.0") |
| `conf_schema` | object | no | JSON Schema for pack configuration |
| `config` | object | no | Default pack configuration values |
| `meta` | object | no | Additional metadata (author, license, etc.) |
| `tags` | array | no | Tags for categorization and search |
| `runtime_deps` | array | no | Runtime IDs this pack depends on |
| `is_standard` | boolean | no | Whether this is a standard/built-in pack |
| `created` | timestamp | auto | Creation timestamp |
| `updated` | timestamp | auto | Last update timestamp |
## API Endpoints
### 1. List Packs
Retrieve a paginated list of all packs.
**Endpoint:** `GET /api/v1/packs`
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `page` | integer | 1 | Page number (1-based) |
| `per_page` | integer | 50 | Items per page (max: 100) |
**Example Request:**
```bash
curl -X GET "http://localhost:3000/api/v1/packs?page=1&per_page=20" \
-H "Authorization: Bearer YOUR_TOKEN"
```
**Example Response:**
```json
{
"data": [
{
"id": 1,
"ref": "core.http",
"label": "HTTP Core Pack",
"description": "HTTP actions and triggers",
"version": "1.0.0",
"tags": ["http", "core"],
"is_standard": true,
"created": "2024-01-15T10:30:00Z",
"updated": "2024-01-15T10:30:00Z"
},
{
"id": 2,
"ref": "aws.ec2",
"label": "AWS EC2 Pack",
"description": "AWS EC2 management",
"version": "1.0.0",
"tags": ["aws", "ec2", "cloud"],
"is_standard": false,
"created": "2024-01-16T14:20:00Z",
"updated": "2024-01-16T14:20:00Z"
}
],
"pagination": {
"page": 1,
"per_page": 20,
"total": 2,
"total_pages": 1
}
}
```
### 2. Get Pack by Reference
Retrieve a specific pack by its reference identifier.
**Endpoint:** `GET /api/v1/packs/:ref`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Pack reference (e.g., "aws.ec2") |
**Example Request:**
```bash
curl -X GET "http://localhost:3000/api/v1/packs/aws.ec2" \
-H "Authorization: Bearer YOUR_TOKEN"
```
**Example Response:**
```json
{
"data": {
"id": 2,
"ref": "aws.ec2",
"label": "AWS EC2 Pack",
"description": "Actions and triggers for AWS EC2 management",
"version": "1.0.0",
"conf_schema": {
"type": "object",
"properties": {
"region": {"type": "string"},
"access_key_id": {"type": "string"}
}
},
"config": {
"region": "us-east-1"
},
"meta": {
"author": "Attune Team"
},
"tags": ["aws", "ec2", "cloud"],
"runtime_deps": [1],
"is_standard": false,
"created": "2024-01-16T14:20:00Z",
"updated": "2024-01-16T14:20:00Z"
}
}
```
### 3. Create Pack
Create a new pack.
**Endpoint:** `POST /api/v1/packs`
**Request Body:**
```json
{
"ref": "slack.notifications",
"label": "Slack Notifications Pack",
"description": "Actions for sending Slack notifications",
"version": "1.0.0",
"conf_schema": {
"type": "object",
"properties": {
"webhook_url": {"type": "string"},
"default_channel": {"type": "string"}
},
"required": ["webhook_url"]
},
"config": {
"default_channel": "#general"
},
"meta": {
"author": "Your Team",
"license": "MIT"
},
"tags": ["slack", "notifications", "messaging"],
"runtime_deps": [1],
"is_standard": false
}
```
**Example Request:**
```bash
curl -X POST "http://localhost:3000/api/v1/packs" \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "slack.notifications",
"label": "Slack Notifications Pack",
"description": "Actions for sending Slack notifications",
"version": "1.0.0",
"tags": ["slack", "notifications"],
"is_standard": false
}'
```
**Example Response:**
```json
{
"data": {
"id": 3,
"ref": "slack.notifications",
"label": "Slack Notifications Pack",
"description": "Actions for sending Slack notifications",
"version": "1.0.0",
"conf_schema": null,
"config": null,
"meta": null,
"tags": ["slack", "notifications"],
"runtime_deps": [],
"is_standard": false,
"created": "2024-01-17T09:15:00Z",
"updated": "2024-01-17T09:15:00Z"
},
"message": "Pack created successfully"
}
```
**Status Codes:**
- `201 Created`: Pack created successfully
- `400 Bad Request`: Invalid request data or validation error
- `409 Conflict`: Pack with the same ref already exists
### 4. Update Pack
Update an existing pack.
**Endpoint:** `PUT /api/v1/packs/:ref`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Pack reference to update |
**Request Body:** (all fields optional)
```json
{
"label": "Updated Pack Label",
"description": "Updated description",
"version": "1.1.0",
"conf_schema": {...},
"config": {...},
"meta": {...},
"tags": ["updated", "tags"],
"runtime_deps": [1, 2],
"is_standard": false
}
```
**Example Request:**
```bash
curl -X PUT "http://localhost:3000/api/v1/packs/slack.notifications" \
-H "Authorization: Bearer YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"version": "1.1.0",
"description": "Enhanced Slack notifications with rich formatting"
}'
```
**Example Response:**
```json
{
"data": {
"id": 3,
"ref": "slack.notifications",
"label": "Slack Notifications Pack",
"description": "Enhanced Slack notifications with rich formatting",
"version": "1.1.0",
"conf_schema": null,
"config": null,
"meta": null,
"tags": ["slack", "notifications"],
"runtime_deps": [],
"is_standard": false,
"created": "2024-01-17T09:15:00Z",
"updated": "2024-01-17T09:30:00Z"
},
"message": "Pack updated successfully"
}
```
**Status Codes:**
- `200 OK`: Pack updated successfully
- `400 Bad Request`: Invalid request data or validation error
- `404 Not Found`: Pack not found
### 5. Delete Pack
Delete a pack and all its associated components.
**Endpoint:** `DELETE /api/v1/packs/:ref`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Pack reference to delete |
**Example Request:**
```bash
curl -X DELETE "http://localhost:3000/api/v1/packs/slack.notifications" \
-H "Authorization: Bearer YOUR_TOKEN"
```
**Example Response:**
```json
{
"success": true,
"message": "Pack 'slack.notifications' deleted successfully"
}
```
**Status Codes:**
- `200 OK`: Pack deleted successfully
- `404 Not Found`: Pack not found
**Warning:** Deleting a pack will cascade delete all actions, triggers, rules, and sensors that belong to it. This operation cannot be undone.
### 6. List Pack Actions
Retrieve all actions that belong to a specific pack.
**Endpoint:** `GET /api/v1/packs/:ref/actions`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Pack reference |
**Example Request:**
```bash
curl -X GET "http://localhost:3000/api/v1/packs/aws.ec2/actions" \
-H "Authorization: Bearer YOUR_TOKEN"
```
**Example Response:**
```json
{
"data": [
{
"id": 1,
"ref": "aws.ec2.start_instance",
"pack_ref": "aws.ec2",
"label": "Start EC2 Instance",
"description": "Start an EC2 instance",
"runtime": 1,
"created": "2024-01-16T14:30:00Z",
"updated": "2024-01-16T14:30:00Z"
},
{
"id": 2,
"ref": "aws.ec2.stop_instance",
"pack_ref": "aws.ec2",
"label": "Stop EC2 Instance",
"description": "Stop an EC2 instance",
"runtime": 1,
"created": "2024-01-16T14:35:00Z",
"updated": "2024-01-16T14:35:00Z"
}
]
}
```
**Status Codes:**
- `200 OK`: Actions retrieved successfully (empty array if none)
- `404 Not Found`: Pack not found
### 7. List Pack Triggers
Retrieve all triggers that belong to a specific pack.
**Endpoint:** `GET /api/v1/packs/:ref/triggers`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Pack reference |
**Example Request:**
```bash
curl -X GET "http://localhost:3000/api/v1/packs/aws.ec2/triggers" \
-H "Authorization: Bearer YOUR_TOKEN"
```
**Example Response:**
```json
{
"data": [
{
"id": 1,
"ref": "aws.ec2.instance_state_change",
"pack_ref": "aws.ec2",
"label": "EC2 Instance State Change",
"description": "Triggered when EC2 instance state changes",
"enabled": true,
"created": "2024-01-16T14:40:00Z",
"updated": "2024-01-16T14:40:00Z"
}
]
}
```
**Status Codes:**
- `200 OK`: Triggers retrieved successfully (empty array if none)
- `404 Not Found`: Pack not found
### 8. List Pack Rules
Retrieve all rules that belong to a specific pack.
**Endpoint:** `GET /api/v1/packs/:ref/rules`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Pack reference |
**Example Request:**
```bash
curl -X GET "http://localhost:3000/api/v1/packs/aws.ec2/rules" \
-H "Authorization: Bearer YOUR_TOKEN"
```
**Example Response:**
```json
{
"data": [
{
"id": 1,
"ref": "aws.ec2.auto_stop_idle",
"pack_ref": "aws.ec2",
"label": "Auto-stop Idle Instances",
"description": "Automatically stop idle EC2 instances",
"action_ref": "aws.ec2.stop_instance",
"trigger_ref": "aws.ec2.instance_state_change",
"enabled": true,
"created": "2024-01-16T15:00:00Z",
"updated": "2024-01-16T15:00:00Z"
}
]
}
```
**Status Codes:**
- `200 OK`: Rules retrieved successfully (empty array if none)
- `404 Not Found`: Pack not found
## Pack Lifecycle
### 1. Pack Creation Workflow
```
1. Define pack metadata (ref, label, description)
2. Create configuration schema (optional)
3. Set default configuration values (optional)
4. Add metadata and tags
5. Specify runtime dependencies
6. POST to /api/v1/packs
7. Create pack actions, triggers, and rules
```
### 2. Pack Update Workflow
```
1. Retrieve current pack details (GET /api/v1/packs/:ref)
2. Modify desired fields
3. Update pack (PUT /api/v1/packs/:ref)
4. Update version number if making breaking changes
5. Update dependent components if needed
```
### 3. Pack Deletion Workflow
```
1. List all pack components:
- GET /api/v1/packs/:ref/actions
- GET /api/v1/packs/:ref/triggers
- GET /api/v1/packs/:ref/rules
2. Verify no critical dependencies
3. Backup pack configuration if needed
4. DELETE /api/v1/packs/:ref
5. All components cascade deleted automatically
```
## Configuration Schema
Packs can define a JSON Schema for their configuration. This schema validates pack configuration values and provides documentation for users.
### Example Configuration Schema
```json
{
"type": "object",
"properties": {
"api_key": {
"type": "string",
"description": "API key for authentication"
},
"region": {
"type": "string",
"enum": ["us-east-1", "us-west-2", "eu-west-1"],
"default": "us-east-1",
"description": "AWS region"
},
"timeout": {
"type": "integer",
"minimum": 1000,
"maximum": 30000,
"default": 5000,
"description": "Request timeout in milliseconds"
},
"retry_count": {
"type": "integer",
"minimum": 0,
"maximum": 5,
"default": 3,
"description": "Number of retry attempts"
}
},
"required": ["api_key", "region"]
}
```
### Using Configuration in Actions
Actions within a pack can access pack configuration:
```python
# In an action's Python script
pack_config = context.pack.config
api_key = pack_config.get("api_key")
region = pack_config.get("region", "us-east-1")
```
## Best Practices
### Pack Design
1. **Single Responsibility**: Each pack should focus on a specific domain or service
2. **Versioning**: Use semantic versioning (MAJOR.MINOR.PATCH)
3. **Documentation**: Provide clear descriptions for pack and configuration
4. **Dependencies**: Minimize runtime dependencies when possible
5. **Configuration**: Use sensible defaults in configuration schemas
### Naming Conventions
- **Pack Ref**: Use dot notation (e.g., "aws.ec2", "slack.notifications")
- **Labels**: Use title case (e.g., "AWS EC2 Pack")
- **Tags**: Use lowercase, kebab-case if needed (e.g., "aws", "cloud-provider")
### Security
1. **Secrets**: Never store secrets in pack configuration
2. **Schema**: Define strict configuration schemas
3. **Validation**: Validate all configuration values
4. **Access Control**: Limit pack modification to authorized users
### Organization
1. **Standard Packs**: Mark built-in/core packs as `is_standard: true`
2. **Tagging**: Use consistent tags for discoverability
3. **Metadata**: Include author, license, and documentation URLs
4. **Dependencies**: Document runtime requirements clearly
## Error Handling
### Common Error Responses
**404 Not Found:**
```json
{
"error": "Pack 'nonexistent.pack' not found"
}
```
**409 Conflict:**
```json
{
"error": "Pack with ref 'aws.ec2' already exists"
}
```
**400 Bad Request:**
```json
{
"error": "Validation failed",
"details": {
"ref": ["Must be between 1 and 255 characters"],
"label": ["Field is required"]
}
}
```
## Integration Examples
### Creating a Complete Pack
```bash
#!/bin/bash
# 1. Create the pack
PACK_ID=$(curl -X POST "http://localhost:3000/api/v1/packs" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "monitoring.healthcheck",
"label": "Health Check Monitoring",
"description": "Monitor endpoint health",
"version": "1.0.0",
"tags": ["monitoring", "health"],
"is_standard": false
}' | jq -r '.data.id')
# 2. Create a trigger
curl -X POST "http://localhost:3000/api/v1/triggers" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "monitoring.healthcheck.endpoint_down",
"pack_ref": "monitoring.healthcheck",
"label": "Endpoint Down",
"description": "Triggered when endpoint is unreachable"
}'
# 3. Create an action
curl -X POST "http://localhost:3000/api/v1/actions" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "monitoring.healthcheck.send_alert",
"pack_ref": "monitoring.healthcheck",
"label": "Send Alert",
"description": "Send notification about endpoint status",
"entrypoint": "actions/send_alert.py",
"runtime": 1
}'
# 4. Create a rule
curl -X POST "http://localhost:3000/api/v1/rules" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "monitoring.healthcheck.alert_on_down",
"pack_ref": "monitoring.healthcheck",
"label": "Alert on Down",
"description": "Send alert when endpoint goes down",
"action_ref": "monitoring.healthcheck.send_alert",
"trigger_ref": "monitoring.healthcheck.endpoint_down",
"enabled": true
}'
echo "Pack created successfully!"
```
### Listing Pack Components
```bash
#!/bin/bash
PACK_REF="aws.ec2"
echo "=== Pack Details ==="
curl -s "http://localhost:3000/api/v1/packs/$PACK_REF" \
-H "Authorization: Bearer $TOKEN" | jq '.data'
echo -e "\n=== Actions ==="
curl -s "http://localhost:3000/api/v1/packs/$PACK_REF/actions" \
-H "Authorization: Bearer $TOKEN" | jq '.data[] | {ref, label}'
echo -e "\n=== Triggers ==="
curl -s "http://localhost:3000/api/v1/packs/$PACK_REF/triggers" \
-H "Authorization: Bearer $TOKEN" | jq '.data[] | {ref, label}'
echo -e "\n=== Rules ==="
curl -s "http://localhost:3000/api/v1/packs/$PACK_REF/rules" \
-H "Authorization: Bearer $TOKEN" | jq '.data[] | {ref, label, enabled}'
```
## Related Documentation
- [Action Management API](api-actions.md)
- [Trigger & Sensor Management API](api-triggers-sensors.md)
- [Rule Management API](api-rules.md)
- [Authentication Guide](authentication.md)
## Summary
The Pack Management API provides:
- ✅ Full CRUD operations for packs
- ✅ Pack component listing (actions, triggers, rules)
- ✅ Configuration schema support
- ✅ Version management
- ✅ Dependency tracking
- ✅ Comprehensive validation and error handling
Packs are the organizational foundation of Attune, enabling modular and reusable automation components.

840
docs/api/api-rules.md Normal file
View File

@@ -0,0 +1,840 @@
# Rule Management API
This document describes the Rule Management API endpoints for the Attune automation platform.
## Overview
Rules are the core automation logic in Attune that connect triggers to actions. When a trigger fires an event that matches a rule's conditions, the associated action is executed. Rules enable powerful event-driven automation workflows.
**Base Path:** `/api/v1/rules`
## Data Model
### Rule
```json
{
"id": 1,
"ref": "mypack.notify_on_error",
"pack": 1,
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action": 5,
"action_ref": "slack.send_message",
"trigger": 3,
"trigger_ref": "core.error_event",
"conditions": {
"and": [
{"var": "event.severity", ">=": 3},
{"var": "event.status", "==": "error"}
]
},
"action_params": {
"channel": "#alerts",
"message": "Error in {{ trigger.payload.service }}: {{ trigger.payload.message }}",
"severity": "{{ trigger.payload.severity }}"
},
"enabled": true,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
```
**Field Descriptions:**
- `id` (integer) - Unique identifier
- `ref` (string) - Unique reference in format `pack.name`
- `pack` (integer) - Pack ID this rule belongs to
- `pack_ref` (string) - Pack reference
- `label` (string) - Human-readable name
- `description` (string) - Rule description
- `action` (integer) - Action ID to execute
- `action_ref` (string) - Action reference
- `trigger` (integer) - Trigger ID that activates this rule
- `trigger_ref` (string) - Trigger reference
- `conditions` (object) - JSON Logic conditions for rule evaluation
- `action_params` (object) - Parameters to pass to the action (supports dynamic templates)
- `enabled` (boolean) - Whether the rule is active
- `created` (timestamp) - Creation time
- `updated` (timestamp) - Last update time
**Action Parameters:**
The `action_params` field supports both static values and dynamic templates:
- **Static values**: `"channel": "#alerts"`
- **Dynamic from trigger payload**: `"message": "{{ trigger.payload.message }}"`
- **Dynamic from pack config**: `"token": "{{ pack.config.api_token }}"`
- **System variables**: `"timestamp": "{{ system.timestamp }}"`
See [Rule Parameter Mapping](./rule-parameter-mapping.md) for complete documentation.
### Rule Summary (List View)
```json
{
"id": 1,
"ref": "mypack.notify_on_error",
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action_ref": "slack.send_message",
"trigger_ref": "core.error_event",
"enabled": true,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
```
## Endpoints
### List All Rules
Retrieve a paginated list of all rules.
**Endpoint:** `GET /api/v1/rules`
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
```json
{
"data": [
{
"id": 1,
"ref": "mypack.notify_on_error",
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action_ref": "slack.send_message",
"trigger_ref": "core.error_event",
"enabled": true,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
],
"pagination": {
"page": 1,
"per_page": 20,
"total": 1,
"total_pages": 1
}
}
```
---
### List Enabled Rules
Retrieve only rules that are currently enabled.
**Endpoint:** `GET /api/v1/rules/enabled`
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
```json
{
"data": [
{
"id": 1,
"ref": "mypack.notify_on_error",
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action_ref": "slack.send_message",
"trigger_ref": "core.error_event",
"enabled": true,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
],
"pagination": {
"page": 1,
"per_page": 20,
"total": 1,
"total_pages": 1
}
}
```
---
### List Rules by Pack
Retrieve all rules belonging to a specific pack.
**Endpoint:** `GET /api/v1/packs/:pack_ref/rules`
**Path Parameters:**
- `pack_ref` (string): Pack reference identifier
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
**Errors:**
- `404 Not Found`: Pack with the specified ref does not exist
---
### List Rules by Action
Retrieve all rules that execute a specific action.
**Endpoint:** `GET /api/v1/actions/:action_ref/rules`
**Path Parameters:**
- `action_ref` (string): Action reference identifier
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
**Errors:**
- `404 Not Found`: Action with the specified ref does not exist
---
### List Rules by Trigger
Retrieve all rules that are activated by a specific trigger.
**Endpoint:** `GET /api/v1/triggers/:trigger_ref/rules`
**Path Parameters:**
- `trigger_ref` (string): Trigger reference identifier
**Query Parameters:**
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Response:** `200 OK`
**Errors:**
- `404 Not Found`: Trigger with the specified ref does not exist
---
### Get Rule by Reference
Retrieve a single rule by its reference identifier.
**Endpoint:** `GET /api/v1/rules/:ref`
**Path Parameters:**
- `ref` (string): Rule reference identifier (e.g., "mypack.notify_on_error")
**Response:** `200 OK`
```json
{
"data": {
"id": 1,
"ref": "mypack.notify_on_error",
"pack": 1,
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action": 5,
"action_ref": "slack.send_message",
"trigger": 3,
"trigger_ref": "core.error_event",
"conditions": {
"and": [
{"var": "event.severity", ">=": 3},
{"var": "event.status", "==": "error"}
]
},
"enabled": true,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
}
}
```
**Errors:**
- `404 Not Found`: Rule with the specified ref does not exist
---
### Create Rule
Create a new rule in the system.
**Endpoint:** `POST /api/v1/rules`
**Request Body:**
```json
{
"ref": "mypack.notify_on_error",
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action_ref": "slack.send_message",
"trigger_ref": "core.error_event",
"conditions": {
"and": [
{"var": "event.severity", ">=": 3},
{"var": "event.status", "==": "error"}
]
},
"action_params": {
"channel": "#alerts",
"message": "Error detected: {{ trigger.payload.message }}",
"severity": "{{ trigger.payload.severity }}"
},
"enabled": true
}
```
**Required Fields:**
- `ref`: Unique reference identifier (alphanumeric, dots, underscores, hyphens)
- `pack_ref`: Reference to the parent pack (must exist)
- `label`: Human-readable name (1-255 characters)
- `description`: Rule description (min 1 character)
- `action_ref`: Reference to action to execute (must exist)
- `trigger_ref`: Reference to trigger that activates rule (must exist)
**Optional Fields:**
- `conditions`: JSON Logic conditions for rule evaluation (default: `{}`)
- `action_params`: Parameters to pass to the action (default: `{}`)
- Supports static values: `"channel": "#alerts"`
- Supports dynamic templates: `"message": "{{ trigger.payload.message }}"`
- Supports pack config: `"token": "{{ pack.config.api_token }}"`
- `enabled`: Whether the rule is active (default: `true`)
**Response:** `201 Created`
```json
{
"data": {
"id": 1,
"ref": "mypack.notify_on_error",
"pack": 1,
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action": 5,
"action_ref": "slack.send_message",
"trigger": 3,
"trigger_ref": "core.error_event",
"conditions": {
"and": [
{"var": "event.severity", ">=": 3},
{"var": "event.status", "==": "error"}
]
},
"action_params": {
"channel": "#alerts",
"message": "Error detected: {{ trigger.payload.message }}",
"severity": "{{ trigger.payload.severity }}"
},
"enabled": true,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T10:00:00Z"
},
"message": "Rule created successfully"
}
```
**Errors:**
- `400 Bad Request`: Invalid request data or validation failure
- `404 Not Found`: Referenced pack, action, or trigger does not exist
- `409 Conflict`: Rule with the same ref already exists
---
### Update Rule
Update an existing rule's properties.
**Endpoint:** `PUT /api/v1/rules/:ref`
**Path Parameters:**
- `ref` (string): Rule reference identifier
**Request Body:**
All fields are optional. Only provided fields will be updated.
```json
{
"label": "Notify on Critical Errors",
"description": "Enhanced error notification with filtering",
"conditions": {
"and": [
{"var": "event.severity", ">=": 4},
{"var": "event.status", "==": "error"}
]
},
"action_params": {
"channel": "#critical-alerts",
"message": "CRITICAL: {{ trigger.payload.service }} - {{ trigger.payload.message }}",
"priority": "high"
},
"enabled": false
}
```
**Note:** You cannot change `pack_ref`, `action_ref`, or `trigger_ref` via update. Create a new rule instead.
**Response:** `200 OK`
```json
{
"data": {
"id": 1,
"ref": "mypack.notify_on_error",
"pack": 1,
"pack_ref": "mypack",
"label": "Updated Notify on Error",
"description": "Updated description",
"action": 5,
"action_ref": "slack.send_message",
"trigger": 3,
"trigger_ref": "core.error_event",
"conditions": {
"and": [
{"var": "event.severity", ">=": 4},
{"var": "event.status", "==": "error"}
]
},
"action_params": {
"channel": "#critical-alerts",
"message": "CRITICAL: {{ trigger.payload.service }} - {{ trigger.payload.message }}",
"priority": "high"
},
"enabled": false,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T12:00:00Z"
},
"message": "Rule updated successfully"
}
```
**Errors:**
- `400 Bad Request`: Invalid request data or validation failure
- `404 Not Found`: Rule with the specified ref does not exist
---
### Enable Rule
Enable a rule to activate it for event processing.
**Endpoint:** `POST /api/v1/rules/:ref/enable`
**Path Parameters:**
- `ref` (string): Rule reference identifier
**Response:** `200 OK`
```json
{
"data": {
"id": 1,
"ref": "mypack.notify_on_error",
"pack": 1,
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action": 5,
"action_ref": "slack.send_message",
"trigger": 3,
"trigger_ref": "core.error_event",
"conditions": { ... },
"enabled": true,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T12:00:00Z"
},
"message": "Rule enabled successfully"
}
```
**Errors:**
- `404 Not Found`: Rule with the specified ref does not exist
---
### Disable Rule
Disable a rule to prevent it from processing events.
**Endpoint:** `POST /api/v1/rules/:ref/disable`
**Path Parameters:**
- `ref` (string): Rule reference identifier
**Response:** `200 OK`
```json
{
"data": {
"id": 1,
"ref": "mypack.notify_on_error",
"pack": 1,
"pack_ref": "mypack",
"label": "Notify on Error",
"description": "Send notification when error event is detected",
"action": 5,
"action_ref": "slack.send_message",
"trigger": 3,
"trigger_ref": "core.error_event",
"conditions": { ... },
"enabled": false,
"created": "2024-01-13T10:00:00Z",
"updated": "2024-01-13T12:00:00Z"
},
"message": "Rule disabled successfully"
}
```
**Errors:**
- `404 Not Found`: Rule with the specified ref does not exist
---
### Delete Rule
Delete a rule from the system.
**Endpoint:** `DELETE /api/v1/rules/:ref`
**Path Parameters:**
- `ref` (string): Rule reference identifier
**Response:** `200 OK`
```json
{
"success": true,
"message": "Rule 'mypack.notify_on_error' deleted successfully"
}
```
**Errors:**
- `404 Not Found`: Rule with the specified ref does not exist
---
## Rule Conditions
Rules use conditions to determine whether an action should be executed when a trigger fires. Conditions are evaluated against the event payload.
### Condition Format
Attune supports JSON Logic format for conditions:
```json
{
"and": [
{"var": "event.severity", ">=": 3},
{"var": "event.status", "==": "error"}
]
}
```
### Common Operators
- **Comparison:** `==`, `!=`, `<`, `<=`, `>`, `>=`
- **Logical:** `and`, `or`, `not`
- **Membership:** `in`, `contains`
- **Existence:** `var` (check if variable exists)
### Condition Examples
**Simple equality check:**
```json
{
"var": "event.status",
"==": "error"
}
```
**Multiple conditions (AND):**
```json
{
"and": [
{"var": "event.severity", ">=": 3},
{"var": "event.type", "==": "alert"}
]
}
```
**Multiple conditions (OR):**
```json
{
"or": [
{"var": "event.status", "==": "error"},
{"var": "event.status", "==": "critical"}
]
}
```
**Nested conditions:**
```json
{
"and": [
{"var": "event.severity", ">=": 3},
{
"or": [
{"var": "event.type", "==": "alert"},
{"var": "event.type", "==": "warning"}
]
}
]
}
```
**Always match (empty conditions):**
```json
{}
```
---
## Examples
### Creating a Simple Rule
```bash
curl -X POST http://localhost:3000/api/v1/rules \
-H "Content-Type: application/json" \
-d '{
"ref": "mypack.alert_on_high_cpu",
"pack_ref": "mypack",
"label": "Alert on High CPU",
"description": "Send alert when CPU usage exceeds 90%",
"action_ref": "slack.send_message",
"trigger_ref": "system.cpu_monitor",
"conditions": {
"var": "cpu_percent",
">": 90
},
"enabled": true
}'
```
### Creating a Rule with Complex Conditions
```bash
curl -X POST http://localhost:3000/api/v1/rules \
-H "Content-Type: application/json" \
-d '{
"ref": "security.failed_login_alert",
"pack_ref": "security",
"label": "Failed Login Alert",
"description": "Alert on multiple failed login attempts",
"action_ref": "pagerduty.create_incident",
"trigger_ref": "auth.login_failure",
"conditions": {
"and": [
{"var": "failed_attempts", ">=": 5},
{"var": "time_window_minutes", "<=": 10},
{"var": "user.is_admin", "==": true}
]
}
}'
```
### Listing Enabled Rules
```bash
curl http://localhost:3000/api/v1/rules/enabled
```
### Listing Rules by Pack
```bash
curl http://localhost:3000/api/v1/packs/mypack/rules
```
### Updating Rule Conditions
```bash
curl -X PUT http://localhost:3000/api/v1/rules/mypack.alert_on_high_cpu \
-H "Content-Type: application/json" \
-d '{
"conditions": {
"var": "cpu_percent",
">": 95
}
}'
```
### Disabling a Rule
```bash
curl -X POST http://localhost:3000/api/v1/rules/mypack.alert_on_high_cpu/disable
```
### Enabling a Rule
```bash
curl -X POST http://localhost:3000/api/v1/rules/mypack.alert_on_high_cpu/enable
```
### Deleting a Rule
```bash
curl -X DELETE http://localhost:3000/api/v1/rules/mypack.alert_on_high_cpu
```
---
## Validation Rules
### Rule Reference (`ref`)
- Must be unique across all rules
- Can contain alphanumeric characters, dots (.), underscores (_), and hyphens (-)
- Typically follows the pattern: `pack_name.rule_name`
- Example: `mypack.notify_on_error`, `security.failed_login_alert`
### Pack Reference (`pack_ref`)
- Must reference an existing pack
- The pack must exist before creating rules for it
### Action Reference (`action_ref`)
- Must reference an existing action
- The action will be executed when rule conditions match
### Trigger Reference (`trigger_ref`)
- Must reference an existing trigger
- The trigger determines when the rule is evaluated
### Conditions
- Must be valid JSON
- Typically follows JSON Logic format
- Empty object `{}` means rule always matches
- Conditions are evaluated against event payload
---
## Rule Evaluation Flow
1. **Trigger Fires:** An event occurs that activates a trigger
2. **Find Rules:** System finds all enabled rules for that trigger
3. **Evaluate Conditions:** Each rule's conditions are evaluated against the event payload
4. **Execute Action:** If conditions match, the associated action is executed
5. **Record Enforcement:** Execution is logged as an enforcement record
```
Event → Trigger → Rule Evaluation → Condition Match? → Execute Action
↓ ↓
(conditions) (yes/no)
```
---
## Best Practices
1. **Naming Conventions**
- Use descriptive, hierarchical names: `pack.purpose`
- Keep names concise but meaningful
- Use lowercase with dots as separators
2. **Condition Design**
- Start simple, add complexity as needed
- Test conditions with sample event data
- Document complex condition logic
- Consider edge cases and null values
3. **Rule Organization**
- Group related rules in the same pack
- One rule per specific automation task
- Avoid overly complex conditions (split into multiple rules)
4. **Performance**
- Keep conditions efficient
- Disable unused rules rather than deleting
- Use specific conditions to reduce unnecessary action executions
5. **Testing**
- Test rules in development environment first
- Start with rules disabled, enable after testing
- Monitor enforcement records to verify behavior
- Use `/rules/:ref/disable` to quickly stop problematic rules
6. **Maintenance**
- Document rule purpose and expected behavior
- Review and update conditions regularly
- Clean up obsolete rules
- Version your condition logic in comments
---
## Common Patterns
### Alert on Threshold
```json
{
"ref": "monitoring.disk_space_alert",
"trigger_ref": "system.disk_check",
"action_ref": "slack.notify",
"conditions": {
"var": "disk_usage_percent",
">": 85
}
}
```
### Multi-Condition Filter
```json
{
"ref": "security.suspicious_activity",
"trigger_ref": "security.access_log",
"action_ref": "security.investigate",
"conditions": {
"and": [
{"var": "request.method", "==": "POST"},
{"var": "response.status", "==": 401},
{"var": "ip_reputation_score", "<": 50}
]
}
}
```
### Time-Based Rule
```json
{
"ref": "backup.daily_backup",
"trigger_ref": "schedule.daily",
"action_ref": "backup.full_backup",
"conditions": {
"var": "hour",
"==": 2
}
}
```
---
## Related Documentation
- [Pack Management API](./api-packs.md)
- [Action Management API](./api-actions.md)
- [Trigger Management API](./api-triggers.md)
- [Execution API](./api-executions.md)
- [Event System](./events.md)
---
**Last Updated:** January 13, 2026

772
docs/api/api-secrets.md Normal file
View File

@@ -0,0 +1,772 @@
# Secret Management API
The Secret Management API provides secure endpoints for storing, retrieving, and managing sensitive credentials, API keys, tokens, and other secret values in the Attune automation platform. All secret values are encrypted at rest using AES-256-GCM encryption.
## Table of Contents
- [Overview](#overview)
- [Security Model](#security-model)
- [Key Model](#key-model)
- [Authentication](#authentication)
- [Endpoints](#endpoints)
- [List Keys](#list-keys)
- [Get Key by Reference](#get-key-by-reference)
- [Create Key](#create-key)
- [Update Key](#update-key)
- [Delete Key](#delete-key)
- [Use Cases](#use-cases)
- [Security Best Practices](#security-best-practices)
- [Related Resources](#related-resources)
---
## Overview
The Secret Management API enables secure storage and retrieval of sensitive data that actions and sensors need to execute. Common use cases include:
- **API Credentials**: Store API keys, tokens, and secrets for external services
- **Database Credentials**: Store database passwords and connection strings
- **SSH Keys**: Store private SSH keys for remote access
- **OAuth Tokens**: Store OAuth access and refresh tokens
- **Service Account Keys**: Store service account credentials for cloud providers
### Key Features
- **Encryption at Rest**: All secret values are encrypted using AES-256-GCM
- **Value Redaction**: List views never expose actual secret values
- **Owner Association**: Link secrets to identities, packs, actions, or sensors
- **Audit Trail**: Track creation and modification timestamps
- **Flexible Ownership**: Support multiple ownership models (system, identity, pack, action, sensor)
---
## Security Model
### Encryption
All secret values marked as `encrypted: true` are encrypted using AES-256-GCM encryption before being stored in the database. The encryption process:
1. **Key Derivation**: The server's encryption key is hashed using SHA-256 to derive a 256-bit AES key
2. **Random Nonce**: A random 96-bit nonce is generated for each encryption operation
3. **Encryption**: The plaintext is encrypted using AES-256-GCM with the derived key and nonce
4. **Storage**: The encrypted value (nonce + ciphertext + authentication tag) is base64-encoded and stored
### Decryption
When retrieving a secret value via GET `/api/v1/keys/:ref`, the server automatically decrypts the value if it's encrypted:
1. The encrypted value is base64-decoded
2. The nonce is extracted from the beginning of the data
3. The ciphertext is decrypted using the server's encryption key
4. The decrypted plaintext is returned in the API response
### Access Control
- **Authentication Required**: All endpoints require JWT authentication
- **No List Value Exposure**: List endpoints (`GET /keys`) never return actual secret values
- **Individual Retrieval**: Secret values can only be retrieved one at a time via GET `/keys/:ref`
- **Audit Logging**: All access is logged (future enhancement)
### Server Configuration
The server must have an encryption key configured:
```yaml
security:
encryption_key: "your-encryption-key-must-be-at-least-32-characters-long"
```
Or via environment variable:
```bash
ATTUNE__SECURITY__ENCRYPTION_KEY="your-encryption-key-must-be-at-least-32-characters-long"
```
⚠️ **Warning**: The encryption key must be:
- At least 32 characters long
- Kept secret and secure
- Backed up securely
- Never committed to version control
- Rotated periodically (requires re-encrypting all secrets)
---
## Key Model
### Key Object
```json
{
"id": 123,
"ref": "github_api_token",
"owner_type": "pack",
"owner": "github-integration",
"owner_identity": null,
"owner_pack": 456,
"owner_pack_ref": "github",
"owner_action": null,
"owner_action_ref": null,
"owner_sensor": null,
"owner_sensor_ref": null,
"name": "GitHub Personal Access Token",
"encrypted": true,
"value": "ghp_1234567890abcdefghijklmnopqrstuvwxyz",
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
}
```
### Fields
| Field | Type | Description |
|-------|------|-------------|
| `id` | integer | Unique key identifier |
| `ref` | string | Unique reference (e.g., "github_token", "aws_secret_key") |
| `owner_type` | string | Owner type: `system`, `identity`, `pack`, `action`, `sensor` |
| `owner` | string | Optional owner string identifier |
| `owner_identity` | integer | Optional owner identity ID |
| `owner_pack` | integer | Optional owner pack ID |
| `owner_pack_ref` | string | Optional owner pack reference |
| `owner_action` | integer | Optional owner action ID |
| `owner_action_ref` | string | Optional owner action reference |
| `owner_sensor` | integer | Optional owner sensor ID |
| `owner_sensor_ref` | string | Optional owner sensor reference |
| `name` | string | Human-readable name |
| `encrypted` | boolean | Whether the value is encrypted (recommended: true) |
| `value` | string | The secret value (decrypted in single-item GET, omitted in lists) |
| `created` | datetime | Timestamp when key was created |
| `updated` | datetime | Timestamp of last update |
### Owner Types
| Type | Description | Use Case |
|------|-------------|----------|
| `system` | System-wide secret | Global configuration, shared credentials |
| `identity` | User-owned secret | Personal API keys, user-specific tokens |
| `pack` | Pack-scoped secret | Credentials for a specific integration pack |
| `action` | Action-specific secret | Credentials used by a single action |
| `sensor` | Sensor-specific secret | Credentials used by a sensor |
---
## Authentication
All secret management endpoints require authentication. Include a valid JWT access token in the `Authorization` header:
```
Authorization: Bearer <access_token>
```
See the [Authentication Guide](./authentication.md) for details on obtaining tokens.
---
## Endpoints
### List Keys
Retrieve a paginated list of keys with optional filtering. **Values are redacted** in list views for security.
**Endpoint:** `GET /api/v1/keys`
**Query Parameters:**
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `owner_type` | string | - | Filter by owner type (system, identity, pack, action, sensor) |
| `owner` | string | - | Filter by owner string |
| `page` | integer | 1 | Page number (1-indexed) |
| `per_page` | integer | 50 | Items per page (max 100) |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/keys?owner_type=pack&page=1" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": [
{
"id": 123,
"ref": "github_api_token",
"owner_type": "pack",
"owner": "github-integration",
"name": "GitHub Personal Access Token",
"encrypted": true,
"created": "2024-01-15T10:00:00Z"
},
{
"id": 124,
"ref": "aws_secret_key",
"owner_type": "pack",
"owner": "aws-integration",
"name": "AWS Secret Access Key",
"encrypted": true,
"created": "2024-01-15T11:00:00Z"
}
],
"pagination": {
"page": 1,
"page_size": 50,
"total_items": 2,
"total_pages": 1
}
}
```
**Note**: Secret values are **never** included in list responses for security.
---
### Get Key by Reference
Retrieve a single key by its reference. **The secret value is decrypted and returned** in the response.
**Endpoint:** `GET /api/v1/keys/:ref`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Key reference (e.g., "github_token") |
**Example Request:**
```bash
curl -X GET "http://localhost:8080/api/v1/keys/github_api_token" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"data": {
"id": 123,
"ref": "github_api_token",
"owner_type": "pack",
"owner": "github-integration",
"owner_identity": null,
"owner_pack": 456,
"owner_pack_ref": "github",
"owner_action": null,
"owner_action_ref": null,
"owner_sensor": null,
"owner_sensor_ref": null,
"name": "GitHub Personal Access Token",
"encrypted": true,
"value": "ghp_1234567890abcdefghijklmnopqrstuvwxyz",
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
}
}
```
**Security Note**: The `value` field contains the **decrypted plaintext** secret. Handle this data carefully and never log or expose it.
**Error Responses:**
- `404 Not Found`: Key not found
- `500 Internal Server Error`: Decryption failed (wrong encryption key or corrupted data)
---
### Create Key
Create a new secret key with automatic encryption.
**Endpoint:** `POST /api/v1/keys`
**Request Body:**
```json
{
"ref": "github_api_token",
"owner_type": "pack",
"owner": "github-integration",
"owner_pack": 456,
"owner_pack_ref": "github",
"name": "GitHub Personal Access Token",
"value": "ghp_1234567890abcdefghijklmnopqrstuvwxyz",
"encrypted": true
}
```
**Field Validation:**
| Field | Required | Constraints |
|-------|----------|-------------|
| `ref` | Yes | 1-255 characters, must be unique |
| `owner_type` | Yes | Must be: system, identity, pack, action, sensor |
| `owner` | No | Max 255 characters |
| `owner_identity` | No | Valid identity ID |
| `owner_pack` | No | Valid pack ID |
| `owner_pack_ref` | No | Max 255 characters |
| `owner_action` | No | Valid action ID |
| `owner_action_ref` | No | Max 255 characters |
| `owner_sensor` | No | Valid sensor ID |
| `owner_sensor_ref` | No | Max 255 characters |
| `name` | Yes | 1-255 characters |
| `value` | Yes | 1-10,000 characters |
| `encrypted` | No | Boolean (default: true) |
**Example Request:**
```bash
curl -X POST "http://localhost:8080/api/v1/keys" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"ref": "github_api_token",
"owner_type": "pack",
"owner_pack_ref": "github",
"name": "GitHub Personal Access Token",
"value": "ghp_1234567890abcdefghijklmnopqrstuvwxyz",
"encrypted": true
}'
```
**Response:** `201 Created`
```json
{
"data": {
"id": 123,
"ref": "github_api_token",
"owner_type": "pack",
"owner": null,
"owner_identity": null,
"owner_pack": null,
"owner_pack_ref": "github",
"owner_action": null,
"owner_action_ref": null,
"owner_sensor": null,
"owner_sensor_ref": null,
"name": "GitHub Personal Access Token",
"encrypted": true,
"value": "ghp_1234567890abcdefghijklmnopqrstuvwxyz",
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T10:00:00Z"
},
"message": "Key created successfully"
}
```
**Error Responses:**
- `400 Bad Request`: Validation error or encryption key not configured
- `409 Conflict`: Key with same `ref` already exists
---
### Update Key
Update an existing key's name or value. If the value is updated and encryption is enabled, it will be re-encrypted.
**Endpoint:** `PUT /api/v1/keys/:ref`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Key reference |
**Request Body:**
```json
{
"name": "GitHub Token (Updated)",
"value": "ghp_newtoken123456789abcdefghijklmnopqr",
"encrypted": true
}
```
**Updatable Fields:**
| Field | Type | Description |
|-------|------|-------------|
| `name` | string | Update the human-readable name |
| `value` | string | Update the secret value (will be re-encrypted if needed) |
| `encrypted` | boolean | Change encryption status (re-encrypts/decrypts value) |
**Example Request:**
```bash
curl -X PUT "http://localhost:8080/api/v1/keys/github_api_token" \
-H "Authorization: Bearer <access_token>" \
-H "Content-Type: application/json" \
-d '{
"name": "GitHub Token (Production)",
"value": "ghp_newtoken123456789abcdefghijklmnopqr"
}'
```
**Response:** `200 OK`
```json
{
"data": {
"id": 123,
"ref": "github_api_token",
"owner_type": "pack",
"owner": "github-integration",
"owner_identity": null,
"owner_pack": 456,
"owner_pack_ref": "github",
"owner_action": null,
"owner_action_ref": null,
"owner_sensor": null,
"owner_sensor_ref": null,
"name": "GitHub Token (Production)",
"encrypted": true,
"value": "ghp_newtoken123456789abcdefghijklmnopqr",
"created": "2024-01-15T10:00:00Z",
"updated": "2024-01-15T14:30:00Z"
},
"message": "Key updated successfully"
}
```
**Error Responses:**
- `404 Not Found`: Key not found
- `400 Bad Request`: Validation error
---
### Delete Key
Delete a secret key permanently.
**Endpoint:** `DELETE /api/v1/keys/:ref`
**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `ref` | string | Key reference |
**Example Request:**
```bash
curl -X DELETE "http://localhost:8080/api/v1/keys/github_api_token" \
-H "Authorization: Bearer <access_token>"
```
**Response:** `200 OK`
```json
{
"message": "Key deleted successfully",
"success": true
}
```
**Error Responses:**
- `404 Not Found`: Key not found
---
## Use Cases
### Store API Credentials
Store third-party API credentials for use in actions:
```bash
curl -X POST "http://localhost:8080/api/v1/keys" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"ref": "sendgrid_api_key",
"owner_type": "pack",
"owner_pack_ref": "email",
"name": "SendGrid API Key",
"value": "SG.abcdefghijklmnopqrstuvwxyz",
"encrypted": true
}'
```
### Store Database Credentials
Store database connection credentials:
```bash
curl -X POST "http://localhost:8080/api/v1/keys" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"ref": "prod_db_password",
"owner_type": "system",
"name": "Production Database Password",
"value": "supersecretpassword123!",
"encrypted": true
}'
```
### Store OAuth Tokens
Store OAuth access tokens for external services:
```bash
curl -X POST "http://localhost:8080/api/v1/keys" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"ref": "slack_oauth_token",
"owner_type": "identity",
"owner_identity": 789,
"name": "Slack OAuth Token",
"value": "xoxb-1234567890-abcdefghijklmnopqr",
"encrypted": true
}'
```
### Retrieve Secret for Action
Actions can retrieve secrets at runtime:
```bash
# Get secret value
SECRET_VALUE=$(curl -s -X GET "http://localhost:8080/api/v1/keys/github_api_token" \
-H "Authorization: Bearer <token>" | jq -r '.data.value')
# Use in action
curl -X GET "https://api.github.com/user" \
-H "Authorization: token $SECRET_VALUE"
```
### List Secrets by Owner
List all secrets owned by a specific pack:
```bash
curl -X GET "http://localhost:8080/api/v1/keys?owner_type=pack&owner=github-integration" \
-H "Authorization: Bearer <token>"
```
### Update Expired Token
Update a secret when credentials change:
```bash
curl -X PUT "http://localhost:8080/api/v1/keys/github_api_token" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{
"value": "ghp_newtoken_after_rotation"
}'
```
---
## Security Best Practices
### 1. Always Encrypt Sensitive Data
**Always** set `encrypted: true` when creating keys containing sensitive data:
```json
{
"ref": "aws_secret_key",
"name": "AWS Secret Access Key",
"value": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"encrypted": true
}
```
### 2. Use Descriptive References
Use clear, descriptive references that indicate the purpose:
✅ Good:
- `github_api_token`
- `prod_db_password`
- `slack_oauth_token`
❌ Bad:
- `token1`
- `secret`
- `key`
### 3. Associate with Owners
Always associate secrets with appropriate owners for better organization:
```json
{
"ref": "github_deploy_key",
"owner_type": "action",
"owner_action_ref": "deploy_to_production",
"name": "GitHub Deployment Key",
"value": "ssh-rsa AAAAB3NzaC1...",
"encrypted": true
}
```
### 4. Rotate Secrets Regularly
Implement a secret rotation policy:
```bash
# Update secret with new value
curl -X PUT "http://localhost:8080/api/v1/keys/api_key" \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"value": "new_rotated_value"}'
```
### 5. Never Log Secret Values
Never log or print secret values in application code:
```python
# ❌ BAD - Logs secret value
logger.info(f"Using API key: {secret_value}")
# ✅ GOOD - Logs redacted info
logger.info(f"Using API key: {secret_value[:4]}...{secret_value[-4:]}")
```
### 6. Limit Access
Use authentication and authorization to limit who can access secrets:
- Implement RBAC (future enhancement)
- Audit secret access (future enhancement)
- Rotate tokens regularly
- Use least-privilege principle
### 7. Backup Encryption Key
**Critical**: Securely backup your encryption key. If you lose it, encrypted secrets cannot be recovered:
```bash
# Backup encryption key to secure location
echo "$ATTUNE__SECURITY__ENCRYPTION_KEY" | gpg --encrypt > encryption_key.gpg.backup
```
### 8. Use Environment-Specific Secrets
Use different secrets for different environments:
- `dev_api_key` - Development environment
- `staging_api_key` - Staging environment
- `prod_api_key` - Production environment
---
## Error Handling
### Common Error Codes
| Status Code | Description |
|-------------|-------------|
| `400 Bad Request` | Invalid input, validation error, or encryption not configured |
| `401 Unauthorized` | Missing or invalid authentication token |
| `404 Not Found` | Key not found |
| `409 Conflict` | Key with same reference already exists |
| `500 Internal Server Error` | Encryption/decryption error or server error |
### Example Error Responses
**Encryption Key Not Configured:**
```json
{
"error": "Cannot encrypt: encryption key not configured on server",
"status": 400
}
```
**Key Already Exists:**
```json
{
"error": "Key with ref 'github_api_token' already exists",
"status": 409
}
```
**Decryption Failed:**
```json
{
"error": "Failed to decrypt key: Decryption failed",
"status": 500
}
```
---
## Encryption Details
### Algorithm
- **Cipher**: AES-256-GCM (Galois/Counter Mode)
- **Key Size**: 256 bits (32 bytes)
- **Nonce Size**: 96 bits (12 bytes)
- **Authentication**: Built-in AEAD authentication
### Key Derivation
The server's encryption key (configured in `security.encryption_key`) is hashed using SHA-256 to derive the actual AES-256 key:
```
AES_KEY = SHA256(encryption_key)
```
### Encrypted Value Format
Encrypted values are stored as base64-encoded strings containing:
```
BASE64(nonce || ciphertext || authentication_tag)
```
Where:
- `nonce`: 12 bytes (randomly generated for each encryption)
- `ciphertext`: Variable length (encrypted plaintext)
- `authentication_tag`: 16 bytes (GCM authentication tag)
### Security Properties
- **Confidentiality**: AES-256 encryption prevents unauthorized reading
- **Authenticity**: GCM mode prevents tampering and forgery
- **Non-deterministic**: Random nonces ensure same plaintext produces different ciphertexts
- **Forward Security**: Key rotation possible (requires re-encrypting all secrets)
---
## Related Resources
- [Action Management API](./api-actions.md) - Actions that use secrets
- [Pack Management API](./api-packs.md) - Packs that own secrets
- [Authentication Guide](./authentication.md) - API authentication details
- [Configuration Guide](./configuration.md) - Server configuration including encryption key
---
## Future Enhancements
### Planned Features
1. **Key Rotation**: Automatic re-encryption when changing encryption keys
2. **Access Control Lists**: Fine-grained permissions on who can access which secrets
3. **Audit Logging**: Detailed logs of all secret access and modifications
4. **Secret Expiration**: Time-to-live (TTL) for temporary secrets
5. **Secret Versioning**: Keep history of secret value changes
6. **Import/Export**: Secure import/export of secrets with encryption
7. **Secret References**: Reference secrets from other secrets
8. **Integration with Vaults**: Support for HashiCorp Vault, AWS Secrets Manager, etc.
---
**Last Updated:** 2024-01-13
**API Version:** v1
**Security Note:** Always use HTTPS in production to protect secrets in transit

1043
docs/api/api-triggers-sensors.md Normal file
View File

File diff suppressed because it is too large Load Diff

685
docs/api/api-workflows.md Normal file
View File

@@ -0,0 +1,685 @@
# Workflow Management API
This document describes the Workflow Management API endpoints in Attune.
## Overview
Workflows are multi-step automation sequences that orchestrate multiple actions, handle conditional logic, and manage complex execution flows. The Workflow API provides endpoints for creating, managing, and querying workflow definitions.
## Endpoints
### List Workflows
List all workflows with optional filtering and pagination.
**Endpoint**: `GET /api/v1/workflows`
**Authentication**: Required (Bearer token)
**Query Parameters**:
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
- `tags` (string, optional): Filter by tags (comma-separated list)
- `enabled` (boolean, optional): Filter by enabled status
- `search` (string, optional): Search term for label/description (case-insensitive)
- `pack_ref` (string, optional): Filter by pack reference
**Example Request**:
```bash
curl -X GET "http://localhost:8080/api/v1/workflows?page=1&per_page=20&enabled=true&tags=incident,approval" \
-H "Authorization: Bearer ${ACCESS_TOKEN}"
```
**Example Response** (200 OK):
```json
{
"data": [
{
"id": 1,
"ref": "slack.incident_workflow",
"pack_ref": "slack",
"label": "Incident Response Workflow",
"description": "Automated incident response workflow",
"version": "1.0.0",
"tags": ["incident", "approval"],
"enabled": true,
"created": "2024-01-13T10:30:00Z",
"updated": "2024-01-13T10:30:00Z"
}
],
"meta": {
"page": 1,
"per_page": 20,
"total": 1,
"total_pages": 1
}
}
```
---
### Get Workflow by Reference
Get detailed information about a specific workflow.
**Endpoint**: `GET /api/v1/workflows/{ref}`
**Authentication**: Required (Bearer token)
**Path Parameters**:
- `ref` (string): Workflow reference identifier (e.g., "slack.incident_workflow")
**Example Request**:
```bash
curl -X GET "http://localhost:8080/api/v1/workflows/slack.incident_workflow" \
-H "Authorization: Bearer ${ACCESS_TOKEN}"
```
**Example Response** (200 OK):
```json
{
"data": {
"id": 1,
"ref": "slack.incident_workflow",
"pack": 1,
"pack_ref": "slack",
"label": "Incident Response Workflow",
"description": "Automated incident response with notifications and approvals",
"version": "1.0.0",
"param_schema": {
"type": "object",
"properties": {
"severity": {
"type": "string",
"enum": ["low", "medium", "high", "critical"]
},
"channel": {
"type": "string"
}
},
"required": ["severity", "channel"]
},
"out_schema": {
"type": "object",
"properties": {
"incident_id": {
"type": "string"
},
"resolved": {
"type": "boolean"
}
}
},
"definition": {
"tasks": [
{
"name": "notify_team",
"action": "slack.post_message",
"input": {
"channel": "{{ channel }}",
"message": "Incident detected: {{ severity }}"
}
},
{
"name": "create_ticket",
"action": "jira.create_issue",
"input": {
"project": "INC",
"summary": "Incident: {{ severity }}",
"description": "Auto-generated incident"
}
},
{
"name": "await_approval",
"action": "core.inquiry",
"input": {
"timeout": 3600,
"approvers": ["oncall@company.com"]
}
}
]
},
"tags": ["incident", "approval", "slack"],
"enabled": true,
"created": "2024-01-13T10:30:00Z",
"updated": "2024-01-13T10:30:00Z"
}
}
```
**Error Responses**:
- `404 Not Found`: Workflow not found
---
### List Workflows by Pack
List all workflows belonging to a specific pack.
**Endpoint**: `GET /api/v1/packs/{pack_ref}/workflows`
**Authentication**: Required (Bearer token)
**Path Parameters**:
- `pack_ref` (string): Pack reference identifier
**Query Parameters**:
- `page` (integer, optional): Page number (default: 1)
- `per_page` (integer, optional): Items per page (default: 20, max: 100)
**Example Request**:
```bash
curl -X GET "http://localhost:8080/api/v1/packs/slack/workflows" \
-H "Authorization: Bearer ${ACCESS_TOKEN}"
```
**Example Response** (200 OK):
```json
{
"data": [
{
"id": 1,
"ref": "slack.incident_workflow",
"pack_ref": "slack",
"label": "Incident Response Workflow",
"description": "Automated incident response workflow",
"version": "1.0.0",
"tags": ["incident"],
"enabled": true,
"created": "2024-01-13T10:30:00Z",
"updated": "2024-01-13T10:30:00Z"
}
],
"meta": {
"page": 1,
"per_page": 20,
"total": 1,
"total_pages": 1
}
}
```
**Error Responses**:
- `404 Not Found`: Pack not found
---
### Create Workflow
Create a new workflow definition.
**Endpoint**: `POST /api/v1/workflows`
**Authentication**: Required (Bearer token)
**Request Body**:
```json
{
"ref": "slack.incident_workflow",
"pack_ref": "slack",
"label": "Incident Response Workflow",
"description": "Automated incident response with notifications",
"version": "1.0.0",
"param_schema": {
"type": "object",
"properties": {
"severity": {
"type": "string",
"enum": ["low", "medium", "high", "critical"]
},
"channel": {
"type": "string"
}
},
"required": ["severity", "channel"]
},
"out_schema": {
"type": "object",
"properties": {
"incident_id": {
"type": "string"
}
}
},
"definition": {
"tasks": [
{
"name": "notify_team",
"action": "slack.post_message",
"input": {
"channel": "{{ channel }}",
"message": "Incident: {{ severity }}"
}
}
]
},
"tags": ["incident", "approval"],
"enabled": true
}
```
**Field Descriptions**:
- `ref` (string, required): Unique workflow reference (typically `pack_name.workflow_name`)
- `pack_ref` (string, required): Reference to the parent pack
- `label` (string, required): Human-readable workflow name
- `description` (string, optional): Workflow description
- `version` (string, required): Semantic version (e.g., "1.0.0")
- `param_schema` (object, optional): JSON Schema for workflow inputs
- `out_schema` (object, optional): JSON Schema for workflow outputs
- `definition` (object, required): Complete workflow definition (tasks, conditions, etc.)
- `tags` (array, optional): Tags for categorization and search
- `enabled` (boolean, optional): Whether workflow is enabled (default: true)
**Example Request**:
```bash
curl -X POST "http://localhost:8080/api/v1/workflows" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d @workflow.json
```
**Example Response** (201 Created):
```json
{
"data": {
"id": 1,
"ref": "slack.incident_workflow",
"pack": 1,
"pack_ref": "slack",
"label": "Incident Response Workflow",
"description": "Automated incident response with notifications",
"version": "1.0.0",
"param_schema": { ... },
"out_schema": { ... },
"definition": { ... },
"tags": ["incident", "approval"],
"enabled": true,
"created": "2024-01-13T10:30:00Z",
"updated": "2024-01-13T10:30:00Z"
},
"message": "Workflow created successfully"
}
```
**Error Responses**:
- `400 Bad Request`: Validation error (invalid fields, missing required fields)
- `404 Not Found`: Pack not found
- `409 Conflict`: Workflow with same ref already exists
---
### Update Workflow
Update an existing workflow definition.
**Endpoint**: `PUT /api/v1/workflows/{ref}`
**Authentication**: Required (Bearer token)
**Path Parameters**:
- `ref` (string): Workflow reference identifier
**Request Body** (all fields optional):
```json
{
"label": "Updated Incident Response Workflow",
"description": "Enhanced incident response with additional automation",
"version": "1.1.0",
"param_schema": { ... },
"out_schema": { ... },
"definition": { ... },
"tags": ["incident", "approval", "automation"],
"enabled": false
}
```
**Example Request**:
```bash
curl -X PUT "http://localhost:8080/api/v1/workflows/slack.incident_workflow" \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"label": "Updated Incident Response",
"version": "1.1.0",
"enabled": false
}'
```
**Example Response** (200 OK):
```json
{
"data": {
"id": 1,
"ref": "slack.incident_workflow",
"pack": 1,
"pack_ref": "slack",
"label": "Updated Incident Response",
"description": "Automated incident response with notifications",
"version": "1.1.0",
"param_schema": { ... },
"out_schema": { ... },
"definition": { ... },
"tags": ["incident", "approval"],
"enabled": false,
"created": "2024-01-13T10:30:00Z",
"updated": "2024-01-13T11:45:00Z"
},
"message": "Workflow updated successfully"
}
```
**Error Responses**:
- `400 Bad Request`: Validation error
- `404 Not Found`: Workflow not found
---
### Delete Workflow
Delete a workflow definition.
**Endpoint**: `DELETE /api/v1/workflows/{ref}`
**Authentication**: Required (Bearer token)
**Path Parameters**:
- `ref` (string): Workflow reference identifier
**Example Request**:
```bash
curl -X DELETE "http://localhost:8080/api/v1/workflows/slack.incident_workflow" \
-H "Authorization: Bearer ${ACCESS_TOKEN}"
```
**Example Response** (200 OK):
```json
{
"success": true,
"message": "Workflow 'slack.incident_workflow' deleted successfully"
}
```
**Error Responses**:
- `404 Not Found`: Workflow not found
**Note**: Deleting a workflow will cascade delete associated workflow executions and task executions.
---
## Workflow Definition Structure
The `definition` field contains the complete workflow specification. Here's the structure:
### Basic Structure
```json
{
"tasks": [
{
"name": "task_name",
"action": "pack.action_name",
"input": {
"param1": "value1",
"param2": "{{ variable }}"
},
"on_success": "next_task",
"on_failure": "error_handler",
"retry": {
"count": 3,
"delay": 60
},
"timeout": 300
}
],
"variables": {
"initial_var": "value"
}
}
```
### Task Fields
- `name` (string, required): Unique task identifier within the workflow
- `action` (string, required): Action reference to execute
- `input` (object, optional): Input parameters for the action (supports Jinja2 templates)
- `on_success` (string, optional): Next task to execute on success
- `on_failure` (string, optional): Task to execute on failure
- `retry` (object, optional): Retry configuration
- `count` (integer): Number of retry attempts
- `delay` (integer): Delay between retries in seconds
- `timeout` (integer, optional): Task timeout in seconds
### Variable Templating
Use Jinja2 syntax to reference workflow variables:
- `{{ variable_name }}`: Reference workflow variable
- `{{ task_name.output.field }}`: Reference task output
- `{{ workflow.param.field }}`: Reference workflow input parameter
### Example: Complex Workflow
```json
{
"tasks": [
{
"name": "fetch_data",
"action": "core.http",
"input": {
"url": "{{ workflow.param.api_url }}",
"method": "GET"
},
"on_success": "process_data",
"on_failure": "notify_error"
},
{
"name": "process_data",
"action": "core.transform",
"input": {
"data": "{{ fetch_data.output.body }}"
},
"on_success": "store_result"
},
{
"name": "store_result",
"action": "database.insert",
"input": {
"table": "results",
"data": "{{ process_data.output }}"
}
},
{
"name": "notify_error",
"action": "slack.post_message",
"input": {
"channel": "#errors",
"message": "Workflow failed: {{ error.message }}"
}
}
],
"variables": {
"max_retries": 3,
"timeout": 300
}
}
```
---
## Filtering and Search
### Filter by Tags
To find workflows with specific tags:
```bash
GET /api/v1/workflows?tags=incident,approval
```
This returns workflows that have **any** of the specified tags.
### Filter by Enabled Status
To find only enabled workflows:
```bash
GET /api/v1/workflows?enabled=true
```
To find disabled workflows:
```bash
GET /api/v1/workflows?enabled=false
```
### Search by Text
To search workflows by label or description:
```bash
GET /api/v1/workflows?search=incident
```
This performs a case-insensitive search across workflow labels and descriptions.
### Filter by Pack
To find workflows from a specific pack:
```bash
GET /api/v1/workflows?pack_ref=slack
```
This returns only workflows belonging to the specified pack.
### Combine Filters
Filters can be combined:
```bash
GET /api/v1/workflows?enabled=true&tags=incident&search=response
```
---
## Best Practices
### Workflow Naming
- Use dot notation: `pack_name.workflow_name`
- Keep names descriptive but concise
- Use snake_case for workflow names
### Versioning
- Follow semantic versioning (MAJOR.MINOR.PATCH)
- Increment MAJOR for breaking changes
- Increment MINOR for new features
- Increment PATCH for bug fixes
### Task Organization
- Keep tasks focused on single responsibilities
- Use descriptive task names
- Define clear success/failure paths
- Set appropriate timeouts and retries
### Error Handling
- Always define `on_failure` handlers for critical tasks
- Use dedicated error notification tasks
- Log errors for debugging
### Performance
- Minimize task dependencies
- Use parallel execution where possible
- Set reasonable timeouts
- Consider task execution costs
---
## Common Use Cases
### 1. Incident Response
```json
{
"tasks": [
{
"name": "create_incident",
"action": "pagerduty.create_incident"
},
{
"name": "notify_team",
"action": "slack.post_message"
},
{
"name": "create_ticket",
"action": "jira.create_issue"
}
]
}
```
### 2. Approval Workflow
```json
{
"tasks": [
{
"name": "request_approval",
"action": "core.inquiry",
"input": {
"approvers": ["manager@company.com"],
"timeout": 3600
},
"on_success": "execute_action",
"on_failure": "notify_rejection"
},
{
"name": "execute_action",
"action": "aws.deploy"
},
{
"name": "notify_rejection",
"action": "email.send"
}
]
}
```
### 3. Data Pipeline
```json
{
"tasks": [
{
"name": "extract",
"action": "database.query"
},
{
"name": "transform",
"action": "core.transform",
"input": {
"data": "{{ extract.output }}"
}
},
{
"name": "load",
"action": "s3.upload",
"input": {
"data": "{{ transform.output }}"
}
}
]
}
```
---
## Related Documentation
- [Actions API](./api-actions.md)
- [Executions API](./api-executions.md)
- [Packs API](./api-packs.md)
- [Workflow Execution Engine](./workflows.md)

340
docs/api/openapi-client-generation.md Normal file
View File

@@ -0,0 +1,340 @@
# OpenAPI Client Generation
This document describes the auto-generated TypeScript API client for the Attune web frontend.
## Overview
The Attune frontend uses an **auto-generated TypeScript client** created from the backend's OpenAPI specification. This ensures:
- ✅ **Type Safety** - All API calls are fully typed
- ✅ **Schema Validation** - Frontend stays in sync with backend
- ✅ **Auto-completion** - Full IDE support for all endpoints
- ✅ **Reduced Errors** - Catch API mismatches at compile time
- ✅ **Automatic Updates** - Regenerate when backend changes
## Architecture
```
Backend (Rust) Frontend (TypeScript)
────────────── ─────────────────────
OpenAPI Spec ─────────────────> Generated Client
(/api-spec/openapi.json) (web/src/api/)
│
├── models/ (TypeScript types)
├── services/ (API methods)
└── core/ (HTTP client)
```
## Generated Files
All files in `web/src/api/` are **auto-generated** from the OpenAPI spec:
```
web/src/api/
├── core/ # HTTP client internals
│ ├── OpenAPI.ts # Configuration
│ ├── request.ts # Request handler
│ └── ApiError.ts # Error types
├── models/ # TypeScript types (90+ files)
│ ├── PackResponse.ts
│ ├── CreatePackRequest.ts
│ ├── ExecutionStatus.ts (enum)
│ └── ...
├── services/ # API service classes (13 files)
│ ├── AuthService.ts
│ ├── PacksService.ts
│ ├── ActionsService.ts
│ └── ...
└── index.ts # Barrel exports
```
**⚠️ DO NOT EDIT THESE FILES** - They will be overwritten on regeneration.
## Configuration
### 1. OpenAPI Client Config (`web/src/lib/api-config.ts`)
```typescript
import { OpenAPI } from "../api";
// Set base URL
OpenAPI.BASE = import.meta.env.VITE_API_BASE_URL || 'http://localhost:8080';
// Configure JWT token injection
OpenAPI.TOKEN = async (): Promise<string> => {
return localStorage.getItem("access_token") || "";
};
// Optional headers
OpenAPI.HEADERS = {
"Content-Type": "application/json",
};
```
### 2. Import in Entry Point (`web/src/main.tsx`)
```typescript
import "./lib/api-config"; // Initialize OpenAPI client
```
This ensures the client is configured before any API calls are made.
## Usage
### Basic API Calls
```typescript
import { PacksService, AuthService } from '@/api';
// List packs
const packs = await PacksService.listPacks({ page: 1, pageSize: 50 });
// Login
const response = await AuthService.login({
requestBody: { login: 'admin', password: 'secret' }
});
// Create pack
const pack = await PacksService.createPack({
requestBody: {
ref: 'my-pack',
label: 'My Pack',
description: 'A custom pack'
}
});
```
### With React Query
```typescript
import { useQuery, useMutation } from '@tanstack/react-query';
import { PacksService } from '@/api';
import type { CreatePackRequest } from '@/api';
// Query
const { data, isLoading } = useQuery({
queryKey: ['packs'],
queryFn: () => PacksService.listPacks({ page: 1, pageSize: 50 })
});
// Mutation
const { mutate } = useMutation({
mutationFn: (data: CreatePackRequest) =>
PacksService.createPack({ requestBody: data }),
onSuccess: () => {
queryClient.invalidateQueries({ queryKey: ['packs'] });
}
});
```
### Error Handling
```typescript
import { ApiError } from '@/api';
try {
await PacksService.getPack({ ref: 'unknown' });
} catch (error) {
if (error instanceof ApiError) {
console.error(`API Error ${error.status}: ${error.message}`);
console.error('Response:', error.body);
}
}
```
## Regenerating the Client
When the backend API changes, regenerate the client to stay in sync.
### Prerequisites
1. **API server must be running:**
```bash
cd attune/crates/api
cargo run --bin attune-api
```
2. **Server listening on http://localhost:8080**
### Regeneration Steps
```bash
cd attune/web
# Regenerate from OpenAPI spec
npm run generate:api
```
This command:
1. Downloads `openapi.json` from `http://localhost:8080/api-spec/openapi.json`
2. Runs `openapi-typescript-codegen` to generate TypeScript code
3. Overwrites all files in `src/api/`
### After Regeneration
1. **Check for TypeScript errors:**
```bash
npm run build
```
2. **Fix any breaking changes** in your code that uses the API
3. **Test the application:**
```bash
npm run dev
```
## Available Services
| Service | Endpoints | Description |
|---------|-----------|-------------|
| **AuthService** | `/auth/*` | Authentication (login, register, refresh) |
| **PacksService** | `/api/v1/packs` | Pack CRUD operations |
| **ActionsService** | `/api/v1/actions` | Action management |
| **RulesService** | `/api/v1/rules` | Rule configuration |
| **TriggersService** | `/api/v1/triggers` | Trigger definitions |
| **SensorsService** | `/api/v1/sensors` | Sensor monitoring |
| **ExecutionsService** | `/api/v1/executions` | Execution tracking |
| **EventsService** | `/api/v1/events` | Event history |
| **InquiriesService** | `/api/v1/inquiries` | Human-in-the-loop workflows |
| **WorkflowsService** | `/api/v1/workflows` | Workflow orchestration |
| **HealthService** | `/health` | Health checks |
| **SecretsService** | `/api/v1/keys` | Secret management |
| **EnforcementsService** | `/api/v1/enforcements` | Rule enforcements |
## Type Definitions
All backend models have corresponding TypeScript types:
### Request Types
- `CreatePackRequest`
- `UpdatePackRequest`
- `CreateActionRequest`
- `LoginRequest`
- `RegisterRequest`
- etc.
### Response Types
- `PackResponse`
- `ActionResponse`
- `ExecutionResponse`
- `ApiResponse_PackResponse` (wrapped responses)
- `PaginatedResponse_PackSummary` (paginated lists)
- etc.
### Enums
- `ExecutionStatus` (`Requested`, `Running`, `Completed`, etc.)
- `EnforcementStatus`
- `InquiryStatus`
- `OwnerType`
- etc.
## Benefits Over Manual API Calls
| Manual Axios | Generated Client |
|--------------|------------------|
| ❌ No type safety | ✅ Full TypeScript types |
| ❌ Manual type definitions | ✅ Auto-generated from spec |
| ❌ Runtime errors | ✅ Compile-time validation |
| ❌ Out-of-sync schemas | ✅ Always matches backend |
| ❌ No auto-completion | ✅ Full IDE support |
| ❌ More code to write | ✅ Less boilerplate |
**Example - Schema Mismatch Caught at Compile Time:**
```typescript
// Manual call - runtime error!
await apiClient.post('/api/v1/packs', {
name: 'my-pack', // ❌ Wrong field (should be 'ref')
system: false // ❌ Wrong field (should be 'is_standard')
});
// Generated client - compile error!
await PacksService.createPack({
requestBody: {
name: 'my-pack', // ❌ TypeScript error: Property 'name' does not exist
ref: 'my-pack', // ✅ Correct field
is_standard: false // ✅ Correct field
}
});
```
## Troubleshooting
### Build Errors After Regeneration
**Symptom:** TypeScript errors after running `npm run generate:api`
**Cause:** Backend schema changed, breaking existing code
**Solution:**
1. Read error messages carefully
2. Update code to match new schema
3. Check backend OpenAPI spec at http://localhost:8080/docs
### "command not found: openapi-typescript-codegen"
### "openapi-typescript-codegen: command not found"
**Solution:**
```bash
# Ensure dependencies are installed
npm install
# The npm script already uses npx, but you can run manually:
npx openapi-typescript-codegen --input ./openapi.json --output ./src/api --client axios --useOptions
```
### Token Not Sent with Requests
**Symptom:** 401 Unauthorized errors despite being logged in
**Cause:** `api-config.ts` not imported
**Solution:** Ensure `import './lib/api-config'` is in `main.tsx`
### Cannot Fetch OpenAPI Spec
**Symptom:** Error downloading from `http://localhost:8080/api-spec/openapi.json`
**Solution:**
1. Start the API server: `cargo run --bin attune-api`
2. Verify server is running: `curl http://localhost:8080/health`
3. Check OpenAPI endpoint: `curl http://localhost:8080/api-spec/openapi.json`
## Development Workflow
1. **Make backend API changes** in Rust code
2. **Update OpenAPI annotations** (`#[utoipa::path(...)]`)
3. **Test backend** with Swagger UI at http://localhost:8080/docs
4. **Regenerate frontend client:** `npm run generate:api`
5. **Fix TypeScript errors** in frontend code
6. **Test integration** end-to-end
## Best Practices
1. ✅ **Always use generated services** - Don't make manual API calls
2. ✅ **Regenerate frequently** - Stay in sync with backend
3. ✅ **Use generated types** - Import from `@/api`, not manual definitions
4. ✅ **Create custom hooks** - Wrap services in React Query hooks
5. ✅ **Handle errors properly** - Use `ApiError` for typed error handling
6. ❌ **Never edit generated files** - Changes will be overwritten
7. ❌ **Don't duplicate types** - Reuse generated types
## Related Documentation
- **Migration Guide:** `web/MIGRATION-TO-GENERATED-CLIENT.md`
- **Generated Client README:** `web/src/api/README.md`
- **Backend OpenAPI Module:** `crates/api/src/openapi.rs`
- **Interactive API Docs:** http://localhost:8080/docs (Swagger UI)
- **OpenAPI Spec:** http://localhost:8080/api-spec/openapi.json
## Tools Used
- **openapi-typescript-codegen** v0.30.0 - Generates TypeScript client
- **Axios** - HTTP client (configured by generator)
- **utoipa** (Rust) - Generates OpenAPI spec from code annotations
## Summary
The auto-generated OpenAPI client provides **type-safe, schema-validated API access** for the Attune frontend. By regenerating the client whenever the backend changes, we ensure the frontend always matches the backend API contract, catching integration issues at compile time rather than runtime.

238
docs/api/openapi-spec-completion.md Normal file
View File

@@ -0,0 +1,238 @@
# OpenAPI Specification Completion
**Date:** 2024-01-13
**Last Updated:** 2024-01-13
**Status:** ✅ Complete
## Overview
The OpenAPI specification for the Attune API has been fully annotated with `utoipa::path` attributes on all route handlers. The API now provides comprehensive, interactive documentation accessible via Swagger UI at `/docs`.
## Completed Work
### 1. Route Annotations
All API endpoints have been annotated with `#[utoipa::path]` attributes including:
- HTTP method and path
- Tag categorization
- Request/response schemas
- Security requirements (JWT bearer auth where applicable)
- Parameter descriptions
- Example values
### 2. Endpoints Documented (86 operations across 62 paths)
The API has **62 unique paths** with **86 total operations** (HTTP methods). Multiple operations on the same path (e.g., GET, POST, PUT, DELETE) count as separate operations.
#### Health Check (4 endpoints)
- `GET /health` - Basic health check
- `GET /health/detailed` - Detailed health with database connectivity
- `GET /health/ready` - Readiness probe
- `GET /health/live` - Liveness probe
#### Authentication (5 endpoints)
- `POST /auth/login` - User login
- `POST /auth/register` - User registration
- `POST /auth/refresh` - Refresh access token
- `GET /auth/me` - Get current user info
- `POST /auth/change-password` - Change password
#### Packs (7 endpoints)
- `GET /packs` - List all packs
- `POST /packs` - Create new pack
- `GET /packs/{ref}` - Get pack by reference
- `PUT /packs/{ref}` - Update pack
- `DELETE /packs/{ref}` - Delete pack
- `POST /packs/{ref}/sync-workflows` - Sync workflows from pack directory
- `GET /packs/{ref}/validate-workflows` - Validate workflows in pack directory
#### Actions (8 endpoints)
- `GET /actions` - List all actions
- `POST /actions` - Create new action
- `GET /actions/{ref}` - Get action by reference
- `PUT /actions/{ref}` - Update action
- `DELETE /actions/{ref}` - Delete action
- `GET /actions/id/{id}` - Get action by ID
- `GET /packs/{pack_ref}/actions` - List actions by pack
- `GET /actions/{ref}/queue-stats` - Get queue statistics for action
#### Triggers (10 endpoints)
- `GET /triggers` - List all triggers
- `GET /triggers/enabled` - List enabled triggers
- `POST /triggers` - Create new trigger
- `GET /triggers/{ref}` - Get trigger by reference
- `PUT /triggers/{ref}` - Update trigger
- `DELETE /triggers/{ref}` - Delete trigger
- `POST /triggers/{ref}/enable` - Enable trigger
- `POST /triggers/{ref}/disable` - Disable trigger
- `GET /triggers/id/{id}` - Get trigger by ID
- `GET /packs/{pack_ref}/triggers` - List triggers by pack
#### Sensors (11 endpoints)
- `GET /sensors` - List all sensors
- `GET /sensors/enabled` - List enabled sensors
- `POST /sensors` - Create new sensor
- `GET /sensors/{ref}` - Get sensor by reference
- `PUT /sensors/{ref}` - Update sensor
- `DELETE /sensors/{ref}` - Delete sensor
- `POST /sensors/{ref}/enable` - Enable sensor
- `POST /sensors/{ref}/disable` - Disable sensor
- `GET /sensors/id/{id}` - Get sensor by ID
- `GET /packs/{pack_ref}/sensors` - List sensors by pack
- `GET /triggers/{trigger_ref}/sensors` - List sensors by trigger
#### Rules (11 endpoints)
- `GET /rules` - List all rules
- `GET /rules/enabled` - List enabled rules
- `POST /rules` - Create new rule
- `GET /rules/{ref}` - Get rule by reference
- `PUT /rules/{ref}` - Update rule
- `DELETE /rules/{ref}` - Delete rule
- `POST /rules/{ref}/enable` - Enable rule
- `POST /rules/{ref}/disable` - Disable rule
- `GET /rules/id/{id}` - Get rule by ID
- `GET /packs/{pack_ref}/rules` - List rules by pack
- `GET /actions/{action_ref}/rules` - List rules by action
- `GET /triggers/{trigger_ref}/rules` - List rules by trigger
#### Executions (5 endpoints)
- `GET /executions` - List all executions (with filters)
- `GET /executions/{id}` - Get execution by ID
- `GET /executions/stats` - Get execution statistics
- `GET /executions/status/{status}` - List executions by status
- `GET /executions/enforcement/{enforcement_id}` - List executions by enforcement
#### Events (2 endpoints)
- `GET /events` - List all events (with filters)
- `GET /events/{id}` - Get event by ID
#### Enforcements (2 endpoints)
- `GET /enforcements` - List all enforcements (with filters)
- `GET /enforcements/{id}` - Get enforcement by ID
#### Inquiries (8 endpoints)
- `GET /inquiries` - List all inquiries (with filters)
- `POST /inquiries` - Create new inquiry
- `GET /inquiries/{id}` - Get inquiry by ID
- `PUT /inquiries/{id}` - Update inquiry
- `DELETE /inquiries/{id}` - Delete inquiry
- `GET /inquiries/status/{status}` - List inquiries by status
- `GET /executions/{execution_id}/inquiries` - List inquiries for execution
- `POST /inquiries/{id}/respond` - Respond to inquiry
#### Keys/Secrets (5 endpoints)
- `GET /keys` - List all keys
- `POST /keys` - Create new key
- `GET /keys/{ref}` - Get key by reference
- `PUT /keys/{ref}` - Update key
- `DELETE /keys/{ref}` - Delete key
#### Workflows (7 endpoints)
- `GET /workflows` - List all workflows (with filtering by tags, enabled status, search)
- `POST /workflows` - Create new workflow
- `GET /workflows/{ref}` - Get workflow by reference
- `PUT /workflows/{ref}` - Update workflow
- `DELETE /workflows/{ref}` - Delete workflow
- `GET /workflows/id/{id}` - Get workflow by ID
- `GET /packs/{pack_ref}/workflows` - List workflows by pack
### 3. DTO Schemas
All Data Transfer Objects (DTOs) are properly documented with `ToSchema` or `IntoParams` attributes:
**Request DTOs:**
- Authentication: LoginRequest, RegisterRequest, RefreshTokenRequest, ChangePasswordRequest
- Packs: CreatePackRequest, UpdatePackRequest
- Actions: CreateActionRequest, UpdateActionRequest
- Triggers: CreateTriggerRequest, UpdateTriggerRequest
- Sensors: CreateSensorRequest, UpdateSensorRequest
- Rules: CreateRuleRequest, UpdateRuleRequest
- Inquiries: CreateInquiryRequest, UpdateInquiryRequest, RespondToInquiryRequest
- Keys: CreateKeyRequest, UpdateKeyRequest
**Response DTOs:**
- Full responses: PackResponse, ActionResponse, TriggerResponse, SensorResponse, RuleResponse, ExecutionResponse, EventResponse, EnforcementResponse, InquiryResponse, KeyResponse
- Summary responses: PackSummary, ActionSummary, TriggerSummary, SensorSummary, RuleSummary, ExecutionSummary, EventSummary, EnforcementSummary, InquirySummary, KeySummary
- Auth: TokenResponse, CurrentUserResponse
- Workflow: WorkflowResponse, WorkflowSummary
- Queue Stats: QueueStatsResponse
**Query Parameter DTOs (IntoParams):**
- PaginationParams
- EventQueryParams
- EnforcementQueryParams
- ExecutionQueryParams
- InquiryQueryParams
- KeyQueryParams
- WorkflowSearchParams
**Common DTOs:**
- ApiResponse<T>
- PaginatedResponse<T>
- PaginationMeta
- SuccessResponse
### 4. Security Schemes
JWT Bearer authentication is properly configured and referenced in protected endpoints:
- Security scheme: `bearer_auth`
- Format: JWT
- Header: `Authorization: Bearer <token>`
Protected endpoints include the `security(("bearer_auth" = []))` attribute.
### 5. Tags
Endpoints are organized by logical tags:
- `health` - Health check endpoints
- `auth` - Authentication and authorization
- `packs` - Pack management
- `actions` - Action management
- `triggers` - Trigger management
- `sensors` - Sensor management
- `rules` - Rule management
- `executions` - Execution queries
- `events` - Event queries
- `enforcements` - Enforcement queries
- `inquiries` - Inquiry (human-in-the-loop) management
- `secrets` - Secret/key management
## Accessing the Documentation
Once the API server is running, the interactive Swagger UI documentation is available at:
```
http://localhost:8080/docs
```
The raw OpenAPI JSON specification is available at:
```
http://localhost:8080/api-spec/openapi.json
```
## Testing
All OpenAPI annotations have been validated:
- ✅ Compilation succeeds without errors
- ✅ OpenAPI spec generation test passes
- ✅ All DTOs properly implement required traits
- ✅ Path count test confirms 62 unique paths
- ✅ Operation count test confirms 86 total operations
## Benefits
1. **Interactive Documentation**: Developers can explore and test API endpoints directly in the browser
2. **Auto-Generated Client SDKs**: The OpenAPI spec can be used to generate client libraries in multiple languages
3. **API Contract**: Serves as the source of truth for API structure and behavior
4. **Validation**: Request/response schemas are explicitly defined and validated
5. **Discoverability**: All endpoints, parameters, and response formats are self-documented
## Future Enhancements
Potential improvements for future iterations:
- Add more detailed examples for complex request bodies
- Include error response schemas for specific error cases
- Add response headers documentation where relevant
- Document rate limiting headers
- Add webhook documentation if/when implemented