# Pack Structure Documentation

**Last Updated**: 2024-01-20  
**Status**: Reference Documentation

---

## Overview

Attune packs are bundles of automation components (actions, sensors, triggers, rules, workflows) organized in a standardized directory structure. This document defines the canonical pack structure and file formats.

---

## Pack Directory Structure

```
packs/<pack_name>/
├── pack.yaml                    # Pack manifest (required)
├── README.md                    # Pack documentation (recommended)
├── CHANGELOG.md                 # Version history (recommended)
├── LICENSE                      # License file (recommended)
├── requirements.txt             # Python dependencies (optional)
├── package.json                 # Node.js dependencies (optional)
├── actions/                     # Action definitions
│   ├── <action_name>.yaml      # Action metadata
│   ├── <action_name>.sh        # Shell action implementation
│   ├── <action_name>.py        # Python action implementation
│   └── <action_name>.js        # Node.js action implementation
├── sensors/                     # Sensor definitions
│   ├── <sensor_name>.yaml      # Sensor metadata
│   ├── <sensor_name>.py        # Python sensor implementation
│   └── <sensor_name>.js        # Node.js sensor implementation
├── triggers/                    # Trigger type definitions
│   └── <trigger_name>.yaml     # Trigger metadata
├── rules/                       # Rule definitions (optional)
│   └── <rule_name>.yaml        # Rule metadata
├── workflows/                   # Workflow definitions (optional)
│   └── <workflow_name>.yaml    # Workflow metadata
├── policies/                    # Policy definitions (optional)
│   └── <policy_name>.yaml      # Policy metadata
├── config.schema.yaml           # Pack configuration schema (optional)
├── icon.png                     # Pack icon (optional)
└── tests/                       # Tests (optional)
    ├── test_actions.py
    └── test_sensors.py
```

---

## File Formats

### Pack Manifest (`pack.yaml`)

The pack manifest is the main metadata file for a pack. It defines the pack's identity, version, dependencies, and configuration.

**Required Fields:**
- `ref` (string): Unique pack reference/identifier (lowercase, alphanumeric, hyphens, underscores)
- `label` (string): Human-readable pack name
- `description` (string): Brief description of the pack
- `version` (string): Semantic version (e.g., "1.0.0")

**Optional Fields:**
- `author` (string): Pack author name
- `email` (string): Author email
- `system` (boolean): Whether this is a system pack (default: false)
- `enabled` (boolean): Whether pack is enabled by default (default: true)
- `conf_schema` (object): JSON Schema for pack configuration
- `config` (object): Default pack configuration
- `meta` (object): Additional metadata
- `tags` (array): Tags for categorization
- `runtime_deps` (array): Runtime dependencies (e.g., "python3", "nodejs", "shell")

**Example:**

```yaml
ref: core
label: "Core Pack"
description: "Built-in core functionality including timer triggers and basic actions"
version: "1.0.0"
author: "Attune Team"
email: "core@attune.io"
system: true
enabled: true

conf_schema:
  type: object
  properties:
    max_action_timeout:
      type: integer
      description: "Maximum timeout for action execution in seconds"
      default: 300
      minimum: 1
      maximum: 3600

config:
  max_action_timeout: 300

meta:
  category: "system"
  keywords:
    - "core"
    - "utilities"
  python_dependencies:
    - "requests>=2.28.0"
  documentation_url: "https://docs.attune.io/packs/core"
  repository_url: "https://github.com/attune-io/attune"

tags:
  - core
  - system
  - utilities

runtime_deps:
  - shell
  - python3
```

---

### Action Metadata (`actions/<action_name>.yaml`)

Action metadata files define the parameters, output schema, and execution details for actions.

**Required Fields:**
- `ref` (string): Full action reference (e.g., "core.echo")
- `label` (string): Human-readable action name
- `description` (string): Action description
- `runner_type` (string): Execution runtime (shell, python, nodejs, docker)
- `entry_point` (string): Script filename to execute

**Optional Fields:**
- `enabled` (boolean): Whether action is enabled (default: true)
- `parameters` (object): Parameter definitions (JSON Schema style)
- `output_schema` (object): Output schema definition
- `parameter_delivery` (string): How parameters are delivered - `env` (environment variables), `stdin` (standard input), or `file` (temporary file). Default: `env`. **Security Note**: Use `stdin` or `file` for actions with sensitive parameters.
- `parameter_format` (string): Parameter serialization format - `dotenv` (KEY='VALUE'), `json` (JSON object), or `yaml` (YAML format). Default: `dotenv`
- `tags` (array): Tags for categorization
- `timeout` (integer): Default timeout in seconds
- `examples` (array): Usage examples

**Example:**

```yaml
ref: core.echo
label: "Echo"
description: "Echo a message to stdout"
enabled: true
runner_type: shell
entry_point: echo.sh

# Parameter delivery (optional, defaults to stdin/json)
parameter_delivery: stdin
parameter_format: json

parameters:
  type: object
  properties:
    message:
      type: string
      description: "Message to echo"
      default: "Hello, World!"
    uppercase:
      type: boolean
      description: "Convert message to uppercase"
      default: false

output_schema:
  type: object
  properties:
    stdout:
      type: string
      description: "Standard output from the command"
    exit_code:
      type: integer
      description: "Exit code (0 = success)"

tags:
  - utility
  - testing
```

---

### Action Implementation

Actions receive parameters according to the `parameter_delivery` method specified in their metadata:

- **`env`** (default): Parameters as environment variables prefixed with `ATTUNE_ACTION_`
- **`stdin`**: Parameters via standard input in the specified format
- **`file`**: Parameters in a temporary file (path in `ATTUNE_PARAMETER_FILE` env var)

**Security Warning**: Environment variables are visible in process listings. Use `stdin` or `file` for sensitive data.

**Shell Example with Environment Variables** (`actions/echo.sh`):

```bash
#!/bin/bash
set -e

# Parse parameters from environment variables
MESSAGE="${ATTUNE_ACTION_MESSAGE:-Hello, World!}"
UPPERCASE="${ATTUNE_ACTION_UPPERCASE:-false}"

# Convert to uppercase if requested
if [ "$UPPERCASE" = "true" ]; then
    MESSAGE=$(echo "$MESSAGE" | tr '[:lower:]' '[:upper:]')
fi

# Echo the message
echo "$MESSAGE"

# Exit successfully
exit 0
```

**Shell Example with Stdin/JSON** (more secure):

```bash
#!/bin/bash
set -e

# Read parameters from stdin (JSON format)
read -r PARAMS_JSON
MESSAGE=$(echo "$PARAMS_JSON" | jq -r '.message // "Hello, World!"')
UPPERCASE=$(echo "$PARAMS_JSON" | jq -r '.uppercase // "false"')

# Convert to uppercase if requested
if [ "$UPPERCASE" = "true" ]; then
    MESSAGE=$(echo "$MESSAGE" | tr '[:lower:]' '[:upper:]')
fi

echo "$MESSAGE"
exit 0
```

**Python Example with Stdin/JSON** (recommended for security):

```python
#!/usr/bin/env python3
import json
import sys

def read_stdin_params():
    """Read parameters from stdin. Secrets are already merged into parameters."""
    content = sys.stdin.read().strip()
    return json.loads(content) if content else {}

def main():
    params = read_stdin_params()
    url = params.get("url")
    method = params.get("method", "GET")
    
    if not url:
        print(json.dumps({"error": "url parameter required"}))
        sys.exit(1)
    
    # Perform action logic
    result = {
        "url": url,
        "method": method,
        "success": True
    }
    
    # Output result as JSON
    print(json.dumps(result, indent=2))
    sys.exit(0)

if __name__ == "__main__":
    main()
```

**Python Example with Environment Variables** (legacy, less secure):

```python
#!/usr/bin/env python3
import json
import os
import sys

def get_env_param(name: str, default=None):
    """Get action parameter from environment variable."""
    env_key = f"ATTUNE_ACTION_{name.upper()}"
    return os.environ.get(env_key, default)

def main():
    url = get_env_param("url")
    method = get_env_param("method", "GET")
    
    if not url:
        print(json.dumps({"error": "url parameter required"}))
        sys.exit(1)
    
    # Perform action logic
    result = {
        "url": url,
        "method": method,
        "success": True
    }
    
    # Output result as JSON
    print(json.dumps(result, indent=2))
    sys.exit(0)

if __name__ == "__main__":
    main()
```

---

### Sensor Metadata (`sensors/<sensor_name>.yaml`)

Sensor metadata files define sensors that monitor for events and fire triggers.

**Required Fields:**
- `ref` (string): Full sensor reference (e.g., "core.interval_timer_sensor")
- `label` (string): Human-readable sensor name
- `description` (string): Sensor description
- `runner_type` (string): Execution runtime (python, nodejs)
- `entry_point` (string): Script filename to execute
- `trigger_types` (array): List of trigger types this sensor monitors

**Optional Fields:**
- `enabled` (boolean): Whether sensor is enabled (default: true)
- `parameters` (object): Sensor configuration parameters
- `poll_interval` (integer): Poll interval in seconds
- `tags` (array): Tags for categorization
- `meta` (object): Additional metadata

**Example:**

```yaml
ref: core.interval_timer_sensor
label: "Interval Timer Sensor"
description: "Monitors time and fires interval timer triggers"
enabled: true
runner_type: python
entry_point: interval_timer_sensor.py

trigger_types:
  - core.intervaltimer

parameters:
  check_interval_seconds:
    type: integer
    description: "How often to check if triggers should fire"
    default: 1
    minimum: 1
    maximum: 60

poll_interval: 1

tags:
  - timer
  - system
  - builtin

meta:
  builtin: true
  system: true
```

---

### Sensor Implementation

Sensors run continuously and emit events to stdout as JSON, one per line.

**Python Example (`sensors/interval_timer_sensor.py`):**

```python
#!/usr/bin/env python3
import json
import time
from datetime import datetime

def check_triggers():
    """Check configured triggers and return events to fire."""
    # Load trigger instances from environment
    # Check if any should fire
    # Return list of events
    return []

def main():
    while True:
        events = check_triggers()
        
        # Output events as JSON (one per line)
        for event in events:
            print(json.dumps(event))
            sys.stdout.flush()
        
        # Sleep until next check
        time.sleep(1)

if __name__ == "__main__":
    main()
```

---

### Trigger Metadata (`triggers/<trigger_name>.yaml`)

Trigger metadata files define event types that sensors can fire.

**Required Fields:**
- `ref` (string): Full trigger reference (e.g., "core.intervaltimer")
- `label` (string): Human-readable trigger name
- `description` (string): Trigger description
- `type` (string): Trigger type (interval, cron, one_shot, webhook, custom)

**Optional Fields:**
- `enabled` (boolean): Whether trigger is enabled (default: true)
- `parameters_schema` (object): Schema for trigger instance configuration
- `payload_schema` (object): Schema for event payload
- `tags` (array): Tags for categorization
- `examples` (array): Configuration examples

**Example:**

```yaml
ref: core.intervaltimer
label: "Interval Timer"
description: "Fires at regular intervals"
enabled: true
type: interval

parameters_schema:
  type: object
  properties:
    unit:
      type: string
      enum: [seconds, minutes, hours]
      description: "Time unit for the interval"
    interval:
      type: integer
      minimum: 1
      description: "Number of time units between triggers"
  required: [unit, interval]

payload_schema:
  type: object
  properties:
    type:
      type: string
      const: interval
    interval_seconds:
      type: integer
    fired_at:
      type: string
      format: date-time
  required: [type, interval_seconds, fired_at]

tags:
  - timer
  - interval

examples:
  - description: "Fire every 10 seconds"
    parameters:
      unit: "seconds"
      interval: 10
```

---

## Pack Loading Process

When a pack is loaded, Attune performs the following steps:

1. **Parse Pack Manifest**: Read and validate `pack.yaml`
2. **Register Pack**: Insert pack metadata into database
3. **Load Actions**: Parse all `actions/*.yaml` files and register actions
4. **Load Sensors**: Parse all `sensors/*.yaml` files and register sensors
5. **Load Triggers**: Parse all `triggers/*.yaml` files and register triggers
6. **Load Rules** (optional): Parse all `rules/*.yaml` files
7. **Load Workflows** (optional): Parse all `workflows/*.yaml` files
8. **Validate Dependencies**: Check that all dependencies are available
9. **Apply Configuration**: Apply default configuration from pack manifest

---

## Pack Types

### System Packs

System packs are built-in packs that ship with Attune.

- `system: true` in pack manifest
- Installed automatically
- Cannot be uninstalled
- Examples: `core`, `system`, `utils`

### Community Packs

Community packs are third-party packs installed from repositories.

- `system: false` in pack manifest
- Installed via CLI or API
- Can be updated and uninstalled
- Examples: `slack`, `aws`, `github`, `datadog`

### Ad-Hoc Packs

Ad-hoc packs are user-created packs without code-based components.

- `system: false` in pack manifest
- Created via Web UI
- May only contain triggers (no actions/sensors)
- Used for custom webhook integrations

---

## Best Practices

### Naming Conventions

- **Pack refs**: lowercase, alphanumeric, hyphens/underscores (e.g., `my-pack`, `aws_ec2`)
- **Component refs**: `<pack_ref>.<component_name>` (e.g., `core.echo`, `slack.send_message`)
- **File names**: Match component names (e.g., `echo.yaml`, `echo.sh`)

### Versioning

- Use semantic versioning (MAJOR.MINOR.PATCH)
- Update `CHANGELOG.md` with each release
- Increment version in `pack.yaml` when releasing

### Documentation

- Include comprehensive `README.md` with usage examples
- Document all parameters and output schemas
- Provide example configurations

### Testing

- Include unit tests in `tests/` directory
- Test actions/sensors independently
- Validate parameter schemas

### Dependencies

- Specify all runtime dependencies in pack manifest
- Pin dependency versions in `requirements.txt` or `package.json`
- Test with minimum required versions

### Security

- **Use `stdin` or `file` parameter delivery for actions with sensitive data** (not `env`)
- Use `secret: true` for sensitive parameters (passwords, tokens, API keys)
- Mark actions with credentials using `parameter_delivery: stdin` and `parameter_format: json`
- Validate all user inputs
- Sanitize command-line arguments to prevent injection
- Use HTTPS for API calls with SSL verification enabled
- Never log sensitive parameters in action output

---

## Example Packs

### Minimal Pack

```
my-pack/
├── pack.yaml
├── README.md
└── actions/
    ├── hello.yaml
    └── hello.sh
```

### Full-Featured Pack

```
slack-pack/
├── pack.yaml
├── README.md
├── CHANGELOG.md
├── LICENSE
├── requirements.txt
├── icon.png
├── actions/
│   ├── send_message.yaml
│   ├── send_message.py
│   ├── upload_file.yaml
│   └── upload_file.py
├── sensors/
│   ├── message_sensor.yaml
│   └── message_sensor.py
├── triggers/
│   ├── message_received.yaml
│   └── reaction_added.yaml
├── config.schema.yaml
└── tests/
    ├── test_actions.py
    └── test_sensors.py
```

---

## Related Documentation

- [Pack Management Architecture](./pack-management-architecture.md)
- [Pack Management API](./api-packs.md)
- [Trigger and Sensor Architecture](./trigger-sensor-architecture.md)
- [Parameter Delivery Methods](../actions/parameter-delivery.md)
- [Action Development Guide](./action-development-guide.md) (future)
- [Sensor Development Guide](./sensor-development-guide.md) (future)