attune/docs/api/api-pack-workflows.md

Pack Workflow API Documentation

This document describes the pack workflow integration endpoints that enable automatic loading, validation, and synchronization of workflow definitions from pack directories.

Overview

The Pack Workflow API provides endpoints to:

• Automatically sync workflows from filesystem when packs are created/updated
• Manually trigger workflow synchronization for a pack
• Validate workflow definitions without registering them
• Integrate workflow lifecycle with pack management

Endpoints

Sync Pack Workflows

Synchronizes workflow definitions from the filesystem to the database for a specific pack.

Endpoint: POST /api/v1/packs/{ref}/workflows/sync

Authentication: Required (Bearer token)

Path Parameters:

• ref (string, required): Pack reference identifier

Response: 200 OK

{
  "data": {
    "pack_ref": "my_pack",
    "loaded_count": 5,
    "registered_count": 5,
    "workflows": [
      {
        "ref_name": "my_pack.deploy_app",
        "created": true,
        "workflow_def_id": 123,
        "warnings": []
      }
    ],
    "errors": []
  },
  "message": "Pack workflows synced successfully"
}

Response Fields:

• pack_ref: The pack reference that was synced
• loaded_count: Number of workflow files found and loaded from filesystem
• registered_count: Number of workflows successfully registered/updated in database
• workflows: Array of individual workflow sync results
  • ref_name: Full workflow reference (pack.workflow_name)
  • created: Whether workflow was created (true) or updated (false)
  • workflow_def_id: Database ID of the workflow definition
  • warnings: Any validation warnings (workflow still registered)
• errors: Array of error messages (workflows that failed to sync)

Error Responses:

• 404 Not Found: Pack does not exist
• 401 Unauthorized: Missing or invalid authentication token
• 500 Internal Server Error: Failed to sync workflows

Example:

curl -X POST http://localhost:8080/api/v1/packs/core/workflows/sync \
  -H "Authorization: Bearer $TOKEN"

---

Validate Pack Workflows

Validates workflow definitions from the filesystem without registering them in the database. Useful for checking workflow syntax and structure before deployment.

Endpoint: POST /api/v1/packs/{ref}/workflows/validate

Authentication: Required (Bearer token)

Path Parameters:

• ref (string, required): Pack reference identifier

Response: 200 OK

{
  "data": {
    "pack_ref": "my_pack",
    "validated_count": 5,
    "error_count": 1,
    "errors": {
      "my_pack.broken_workflow": [
        "Missing required field: version",
        "Task 'step1' references undefined action 'invalid.action'"
      ]
    }
  },
  "message": "Pack workflows validated"
}

Response Fields:

• pack_ref: The pack reference that was validated
• validated_count: Total number of workflow files validated
• error_count: Number of workflows with validation errors
• errors: Map of workflow references to their validation error messages

Error Responses:

• 404 Not Found: Pack does not exist
• 401 Unauthorized: Missing or invalid authentication token
• 500 Internal Server Error: Failed to validate workflows

Example:

curl -X POST http://localhost:8080/api/v1/packs/core/workflows/validate \
  -H "Authorization: Bearer $TOKEN"

---

Automatic Workflow Synchronization

Workflows are automatically synchronized in the following scenarios:

Pack Creation

When a pack is created via POST /api/v1/packs, the system automatically attempts to load and register workflows from the pack's workflows/ directory.

Example:

curl -X POST http://localhost:8080/api/v1/packs \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "ref": "my_pack",
    "label": "My Pack",
    "version": "1.0.0",
    "description": "A custom pack"
  }'

If workflows exist in /opt/attune/packs/my_pack/workflows/, they will be automatically loaded and registered.

Pack Update

When a pack is updated via PUT /api/v1/packs/{ref}, the system automatically resyncs workflows to capture any changes.

Example:

curl -X PUT http://localhost:8080/api/v1/packs/my_pack \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "version": "1.1.0",
    "label": "My Pack (Updated)"
  }'

Note: Auto-sync uses skip_validation_errors: true, so pack operations won't fail if workflows have validation errors. Use the validate endpoint to check for errors.

---

Workflow Directory Structure

Workflows must be placed in the workflows/ subdirectory of the pack directory:

/opt/attune/packs/
  └── my_pack/
      ├── actions/
      ├── sensors/
      └── workflows/
          ├── deploy_app.yaml
          ├── rollback.yaml
          └── health_check.yml

Workflow File Requirements:

• Must be in YAML format (.yaml or .yml extension)
• Filename (without extension) becomes the workflow name
• Full workflow reference is pack_ref.workflow_name
• Must conform to the workflow schema (see Workflow API documentation)

---

Configuration

The pack workflows feature uses the following configuration:

Config File (config.yaml):

packs_base_dir: "/opt/attune/packs"  # Base directory for pack directories

Environment Variable:

export ATTUNE__PACKS_BASE_DIR="/opt/attune/packs"

Default: /opt/attune/packs

---

Workflow Lifecycle

1. Development: Create workflow YAML files in pack's workflows/ directory
2. Validation: Use validate endpoint to check for errors
3. Deployment: Create/update pack via API (auto-syncs workflows)
4. Manual Sync: Use sync endpoint to reload workflows after filesystem changes
5. Execution: Workflows become available for execution via workflow API

---

Best Practices

1. Validate Before Deploy

Always validate workflows before deploying:

# Validate workflows
curl -X POST http://localhost:8080/api/v1/packs/my_pack/workflows/validate \
  -H "Authorization: Bearer $TOKEN"

# If validation passes, sync
curl -X POST http://localhost:8080/api/v1/packs/my_pack/workflows/sync \
  -H "Authorization: Bearer $TOKEN"

2. Version Control

Keep workflow YAML files in version control alongside pack code:

my_pack/
  ├── actions/
  │   └── deploy.py
  ├── workflows/
  │   └── deploy_app.yaml
  └── pack.yaml

3. Naming Conventions

Use descriptive workflow filenames that indicate their purpose:

• deploy_production.yaml
• backup_database.yaml
• incident_response.yaml

4. Handle Sync Errors

Check the errors field in sync responses:

{
  "data": {
    "errors": [
      "workflow 'my_pack.invalid' failed validation: Missing required field"
    ]
  }
}

5. Incremental Updates

When updating workflows:

1. Modify YAML files on filesystem
2. Call sync endpoint to reload
3. Previous workflow versions are updated (not duplicated)

---

Error Handling

Common Errors

Pack Not Found (404):

{
  "error": "Pack 'nonexistent_pack' not found"
}

Validation Errors: Workflows with validation errors are reported but don't prevent sync:

{
  "data": {
    "workflows": [...],
    "errors": [
      "Validation failed for my_pack.broken: Missing tasks field"
    ]
  }
}

Filesystem Access: If pack directory doesn't exist on filesystem:

{
  "data": {
    "loaded_count": 0,
    "registered_count": 0,
    "errors": ["Failed to load workflows: Directory not found"]
  }
}

---

Integration Examples

CI/CD Pipeline

#!/bin/bash
# deploy-pack.sh

PACK_NAME="my_pack"
PACK_VERSION="1.0.0"
API_URL="http://localhost:8080"

# 1. Validate workflows locally
echo "Validating workflows..."
response=$(curl -s -X POST "$API_URL/api/v1/packs/$PACK_NAME/workflows/validate" \
  -H "Authorization: Bearer $TOKEN")

error_count=$(echo $response | jq -r '.data.error_count')
if [ "$error_count" -gt 0 ]; then
  echo "Validation errors found:"
  echo $response | jq '.data.errors'
  exit 1
fi

# 2. Create or update pack
echo "Deploying pack..."
curl -X POST "$API_URL/api/v1/packs" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"ref\": \"$PACK_NAME\",
    \"label\": \"My Pack\",
    \"version\": \"$PACK_VERSION\"
  }"

# 3. Verify sync
echo "Syncing workflows..."
curl -X POST "$API_URL/api/v1/packs/$PACK_NAME/workflows/sync" \
  -H "Authorization: Bearer $TOKEN"

echo "Deployment complete!"

Development Workflow

import requests

API_URL = "http://localhost:8080"
TOKEN = "your_access_token"

def sync_workflows(pack_ref: str):
    """Sync workflows after local changes."""
    response = requests.post(
        f"{API_URL}/api/v1/packs/{pack_ref}/workflows/sync",
        headers={"Authorization": f"Bearer {TOKEN}"}
    )
    
    data = response.json()["data"]
    print(f"Synced {data['registered_count']} workflows")
    
    if data['errors']:
        print("Errors:", data['errors'])
    
    return data

# Usage
sync_workflows("my_pack")

---

Related Documentation

• Workflow API - Workflow definition management
• Pack API - Pack management endpoints
• Workflow Orchestration - Workflow execution and concepts

---

Changelog

• v0.1.0 (2024-01): Initial implementation of pack workflow integration
  • Auto-sync on pack create/update
  • Manual sync endpoint
  • Validation endpoint