attune/docs/packs/pack-structure.md

Pack Structure Documentation

Last Updated: 2024-01-20
Status: Reference Documentation

---

Overview

Attune packs are bundles of automation components (actions, sensors, triggers, rules, workflows) organized in a standardized directory structure. This document defines the canonical pack structure and file formats.

---

Pack Directory Structure

packs/<pack_name>/
├── pack.yaml                    # Pack manifest (required)
├── README.md                    # Pack documentation (recommended)
├── CHANGELOG.md                 # Version history (recommended)
├── LICENSE                      # License file (recommended)
├── requirements.txt             # Python dependencies (optional)
├── package.json                 # Node.js dependencies (optional)
├── actions/                     # Action definitions
│   ├── <action_name>.yaml      # Action metadata
│   ├── <action_name>.sh        # Shell action implementation
│   ├── <action_name>.py        # Python action implementation
│   └── <action_name>.js        # Node.js action implementation
├── sensors/                     # Sensor definitions
│   ├── <sensor_name>.yaml      # Sensor metadata
│   ├── <sensor_name>.py        # Python sensor implementation
│   └── <sensor_name>.js        # Node.js sensor implementation
├── triggers/                    # Trigger type definitions
│   └── <trigger_name>.yaml     # Trigger metadata
├── rules/                       # Rule definitions (optional)
│   └── <rule_name>.yaml        # Rule metadata
├── workflows/                   # Workflow definitions (optional)
│   └── <workflow_name>.yaml    # Workflow metadata
├── policies/                    # Policy definitions (optional)
│   └── <policy_name>.yaml      # Policy metadata
├── config.schema.yaml           # Pack configuration schema (optional)
├── icon.png                     # Pack icon (optional)
└── tests/                       # Tests (optional)
    ├── test_actions.py
    └── test_sensors.py

---

File Formats

Pack Manifest (pack.yaml)

The pack manifest is the main metadata file for a pack. It defines the pack's identity, version, dependencies, and configuration.

Required Fields:

• ref (string): Unique pack reference/identifier (lowercase, alphanumeric, hyphens, underscores)
• label (string): Human-readable pack name
• description (string): Brief description of the pack
• version (string): Semantic version (e.g., "1.0.0")

Optional Fields:

• author (string): Pack author name
• email (string): Author email
• system (boolean): Whether this is a system pack (default: false)
• enabled (boolean): Whether pack is enabled by default (default: true)
• conf_schema (object): JSON Schema for pack configuration
• config (object): Default pack configuration
• meta (object): Additional metadata
• tags (array): Tags for categorization
• runtime_deps (array): Runtime dependencies (e.g., "python3", "nodejs", "shell")

Example:

ref: core
label: "Core Pack"
description: "Built-in core functionality including timer triggers and basic actions"
version: "1.0.0"
author: "Attune Team"
email: "core@attune.io"
system: true
enabled: true

conf_schema:
  type: object
  properties:
    max_action_timeout:
      type: integer
      description: "Maximum timeout for action execution in seconds"
      default: 300
      minimum: 1
      maximum: 3600
  required: []

config:
  max_action_timeout: 300

meta:
  category: "system"
  keywords:
    - "core"
    - "utilities"
  python_dependencies:
    - "requests>=2.28.0"
  documentation_url: "https://docs.attune.io/packs/core"
  repository_url: "https://github.com/attune-io/attune"

tags:
  - core
  - system
  - utilities

runtime_deps:
  - shell
  - python3

---

Action Metadata (actions/<action_name>.yaml)

Action metadata files define the parameters, output schema, and execution details for actions.

Required Fields:

• name (string): Action name (matches filename)
• ref (string): Full action reference (e.g., "core.echo")
• description (string): Action description
• runner_type (string): Execution runtime (shell, python, nodejs, docker)
• entry_point (string): Script filename to execute

Optional Fields:

• enabled (boolean): Whether action is enabled (default: true)
• parameters (object): Parameter definitions (JSON Schema style)
• output_schema (object): Output schema definition
• parameter_delivery (string): How parameters are delivered - env (environment variables), stdin (standard input), or file (temporary file). Default: env. Security Note: Use stdin or file for actions with sensitive parameters.
• parameter_format (string): Parameter serialization format - dotenv (KEY='VALUE'), json (JSON object), or yaml (YAML format). Default: dotenv
• tags (array): Tags for categorization
• timeout (integer): Default timeout in seconds
• examples (array): Usage examples

Example:

name: echo
ref: core.echo
description: "Echo a message to stdout"
enabled: true
runner_type: shell
entry_point: echo.sh

# Parameter delivery (optional, defaults to env/dotenv)
parameter_delivery: env
parameter_format: dotenv

parameters:
  message:
    type: string
    description: "Message to echo"
    required: true
    default: "Hello, World!"
  uppercase:
    type: boolean
    description: "Convert message to uppercase"
    required: false
    default: false

output_schema:
  type: object
  properties:
    stdout:
      type: string
      description: "Standard output from the command"
    exit_code:
      type: integer
      description: "Exit code (0 = success)"

tags:
  - utility
  - testing

---

Action Implementation

Actions receive parameters according to the parameter_delivery method specified in their metadata:

• env (default): Parameters as environment variables prefixed with ATTUNE_ACTION_
• stdin: Parameters via standard input in the specified format
• file: Parameters in a temporary file (path in ATTUNE_PARAMETER_FILE env var)

Security Warning: Environment variables are visible in process listings. Use stdin or file for sensitive data.

Shell Example with Environment Variables (actions/echo.sh):

#!/bin/bash
set -e

# Parse parameters from environment variables
MESSAGE="${ATTUNE_ACTION_MESSAGE:-Hello, World!}"
UPPERCASE="${ATTUNE_ACTION_UPPERCASE:-false}"

# Convert to uppercase if requested
if [ "$UPPERCASE" = "true" ]; then
    MESSAGE=$(echo "$MESSAGE" | tr '[:lower:]' '[:upper:]')
fi

# Echo the message
echo "$MESSAGE"

# Exit successfully
exit 0

Shell Example with Stdin/JSON (more secure):

#!/bin/bash
set -e

# Read parameters from stdin (JSON format)
read -r PARAMS_JSON
MESSAGE=$(echo "$PARAMS_JSON" | jq -r '.message // "Hello, World!"')
UPPERCASE=$(echo "$PARAMS_JSON" | jq -r '.uppercase // "false"')

# Convert to uppercase if requested
if [ "$UPPERCASE" = "true" ]; then
    MESSAGE=$(echo "$MESSAGE" | tr '[:lower:]' '[:upper:]')
fi

echo "$MESSAGE"
exit 0

Python Example with Stdin/JSON (recommended for security):

#!/usr/bin/env python3
import json
import sys

def read_stdin_params():
    """Read parameters from stdin."""
    content = sys.stdin.read()
    parts = content.split('---ATTUNE_PARAMS_END---')
    params = json.loads(parts[0].strip()) if parts[0].strip() else {}
    secrets = json.loads(parts[1].strip()) if len(parts) > 1 and parts[1].strip() else {}
    return {**params, **secrets}

def main():
    params = read_stdin_params()
    url = params.get("url")
    method = params.get("method", "GET")
    
    if not url:
        print(json.dumps({"error": "url parameter required"}))
        sys.exit(1)
    
    # Perform action logic
    result = {
        "url": url,
        "method": method,
        "success": True
    }
    
    # Output result as JSON
    print(json.dumps(result, indent=2))
    sys.exit(0)

if __name__ == "__main__":
    main()

Python Example with Environment Variables (legacy, less secure):

#!/usr/bin/env python3
import json
import os
import sys

def get_env_param(name: str, default=None):
    """Get action parameter from environment variable."""
    env_key = f"ATTUNE_ACTION_{name.upper()}"
    return os.environ.get(env_key, default)

def main():
    url = get_env_param("url")
    method = get_env_param("method", "GET")
    
    if not url:
        print(json.dumps({"error": "url parameter required"}))
        sys.exit(1)
    
    # Perform action logic
    result = {
        "url": url,
        "method": method,
        "success": True
    }
    
    # Output result as JSON
    print(json.dumps(result, indent=2))
    sys.exit(0)

if __name__ == "__main__":
    main()

---

Sensor Metadata (sensors/<sensor_name>.yaml)

Sensor metadata files define sensors that monitor for events and fire triggers.

Required Fields:

• name (string): Sensor name
• ref (string): Full sensor reference (e.g., "core.interval_timer_sensor")
• description (string): Sensor description
• runner_type (string): Execution runtime (python, nodejs)
• entry_point (string): Script filename to execute
• trigger_types (array): List of trigger types this sensor monitors

Optional Fields:

• enabled (boolean): Whether sensor is enabled (default: true)
• parameters (object): Sensor configuration parameters
• poll_interval (integer): Poll interval in seconds
• tags (array): Tags for categorization
• meta (object): Additional metadata

Example:

name: interval_timer_sensor
ref: core.interval_timer_sensor
description: "Monitors time and fires interval timer triggers"
enabled: true
runner_type: python
entry_point: interval_timer_sensor.py

trigger_types:
  - core.intervaltimer

parameters:
  check_interval_seconds:
    type: integer
    description: "How often to check if triggers should fire"
    default: 1
    minimum: 1
    maximum: 60

poll_interval: 1

tags:
  - timer
  - system
  - builtin

meta:
  builtin: true
  system: true

---

Sensor Implementation

Sensors run continuously and emit events to stdout as JSON, one per line.

Python Example (sensors/interval_timer_sensor.py):

#!/usr/bin/env python3
import json
import time
from datetime import datetime

def check_triggers():
    """Check configured triggers and return events to fire."""
    # Load trigger instances from environment
    # Check if any should fire
    # Return list of events
    return []

def main():
    while True:
        events = check_triggers()
        
        # Output events as JSON (one per line)
        for event in events:
            print(json.dumps(event))
            sys.stdout.flush()
        
        # Sleep until next check
        time.sleep(1)

if __name__ == "__main__":
    main()

---

Trigger Metadata (triggers/<trigger_name>.yaml)

Trigger metadata files define event types that sensors can fire.

Required Fields:

• name (string): Trigger name
• ref (string): Full trigger reference (e.g., "core.intervaltimer")
• description (string): Trigger description
• type (string): Trigger type (interval, cron, one_shot, webhook, custom)

Optional Fields:

• enabled (boolean): Whether trigger is enabled (default: true)
• parameters_schema (object): Schema for trigger instance configuration
• payload_schema (object): Schema for event payload
• tags (array): Tags for categorization
• examples (array): Configuration examples

Example:

name: intervaltimer
ref: core.intervaltimer
description: "Fires at regular intervals"
enabled: true
type: interval

parameters_schema:
  type: object
  properties:
    unit:
      type: string
      enum: [seconds, minutes, hours]
      description: "Time unit for the interval"
    interval:
      type: integer
      minimum: 1
      description: "Number of time units between triggers"
  required: [unit, interval]

payload_schema:
  type: object
  properties:
    type:
      type: string
      const: interval
    interval_seconds:
      type: integer
    fired_at:
      type: string
      format: date-time
  required: [type, interval_seconds, fired_at]

tags:
  - timer
  - interval

examples:
  - description: "Fire every 10 seconds"
    parameters:
      unit: "seconds"
      interval: 10

---

Pack Loading Process

When a pack is loaded, Attune performs the following steps:

1. Parse Pack Manifest: Read and validate pack.yaml
2. Register Pack: Insert pack metadata into database
3. Load Actions: Parse all actions/*.yaml files and register actions
4. Load Sensors: Parse all sensors/*.yaml files and register sensors
5. Load Triggers: Parse all triggers/*.yaml files and register triggers
6. Load Rules (optional): Parse all rules/*.yaml files
7. Load Workflows (optional): Parse all workflows/*.yaml files
8. Validate Dependencies: Check that all dependencies are available
9. Apply Configuration: Apply default configuration from pack manifest

---

Pack Types

System Packs

System packs are built-in packs that ship with Attune.

• system: true in pack manifest
• Installed automatically
• Cannot be uninstalled
• Examples: core, system, utils

Community Packs

Community packs are third-party packs installed from repositories.

• system: false in pack manifest
• Installed via CLI or API
• Can be updated and uninstalled
• Examples: slack, aws, github, datadog

Ad-Hoc Packs

Ad-hoc packs are user-created packs without code-based components.

• system: false in pack manifest
• Created via Web UI
• May only contain triggers (no actions/sensors)
• Used for custom webhook integrations

---

Best Practices

Naming Conventions

• Pack refs: lowercase, alphanumeric, hyphens/underscores (e.g., my-pack, aws_ec2)
• Component refs: <pack_ref>.<component_name> (e.g., core.echo, slack.send_message)
• File names: Match component names (e.g., echo.yaml, echo.sh)

Versioning

• Use semantic versioning (MAJOR.MINOR.PATCH)
• Update CHANGELOG.md with each release
• Increment version in pack.yaml when releasing

Documentation

• Include comprehensive README.md with usage examples
• Document all parameters and output schemas
• Provide example configurations

Testing

• Include unit tests in tests/ directory
• Test actions/sensors independently
• Validate parameter schemas

Dependencies

• Specify all runtime dependencies in pack manifest
• Pin dependency versions in requirements.txt or package.json
• Test with minimum required versions

Security

• Use stdin or file parameter delivery for actions with sensitive data (not env)
• Use secret: true for sensitive parameters (passwords, tokens, API keys)
• Mark actions with credentials using parameter_delivery: stdin and parameter_format: json
• Validate all user inputs
• Sanitize command-line arguments to prevent injection
• Use HTTPS for API calls with SSL verification enabled
• Never log sensitive parameters in action output

---

Example Packs

Minimal Pack

my-pack/
├── pack.yaml
├── README.md
└── actions/
    ├── hello.yaml
    └── hello.sh

Full-Featured Pack

slack-pack/
├── pack.yaml
├── README.md
├── CHANGELOG.md
├── LICENSE
├── requirements.txt
├── icon.png
├── actions/
│   ├── send_message.yaml
│   ├── send_message.py
│   ├── upload_file.yaml
│   └── upload_file.py
├── sensors/
│   ├── message_sensor.yaml
│   └── message_sensor.py
├── triggers/
│   ├── message_received.yaml
│   └── reaction_added.yaml
├── config.schema.yaml
└── tests/
    ├── test_actions.py
    └── test_sensors.py

---

Related Documentation

• Pack Management Architecture
• Pack Management API
• Trigger and Sensor Architecture
• Parameter Delivery Methods
• Action Development Guide (future)
• Sensor Development Guide (future)