working out the worker/execution interface
This commit is contained in:
364
docs/actions/QUICKREF-parameter-delivery.md
Normal file
364
docs/actions/QUICKREF-parameter-delivery.md
Normal file
@@ -0,0 +1,364 @@
|
||||
# Parameter Delivery Quick Reference
|
||||
|
||||
**Quick guide for choosing and implementing secure parameter passing in actions**
|
||||
|
||||
---
|
||||
|
||||
## TL;DR - Security First
|
||||
|
||||
**DEFAULT**: `stdin` + `json` (secure by default as of 2025-02-05)
|
||||
|
||||
**KEY DESIGN**: Parameters and environment variables are separate!
|
||||
- **Parameters** = Action data (always secure: stdin or file)
|
||||
- **Environment Variables** = Execution context (separate: `execution.env_vars`)
|
||||
|
||||
```yaml
|
||||
# ✅ DEFAULT (no need to specify) - secure for all actions
|
||||
# parameter_delivery: stdin
|
||||
# parameter_format: json
|
||||
|
||||
# For large payloads only:
|
||||
parameter_delivery: file
|
||||
parameter_format: yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Decision Matrix
|
||||
|
||||
| Your Action Has... | Use This |
|
||||
|--------------------|----------|
|
||||
| 🔑 API keys, passwords, tokens | Default (`stdin` + `json`) |
|
||||
| 📦 Large config files (>1MB) | `file` + `yaml` |
|
||||
| 🐚 Shell scripts | Default (`stdin` + `json` or `dotenv`) |
|
||||
| 🐍 Python/Node.js actions | Default (`stdin` + `json`) |
|
||||
| 📝 Most actions | Default (`stdin` + `json`) |
|
||||
|
||||
---
|
||||
|
||||
## Two Delivery Methods
|
||||
|
||||
### 1. Standard Input (`stdin`)
|
||||
|
||||
**Security**: ✅ HIGH - Not in process list
|
||||
**When**: Credentials, API keys, structured data (DEFAULT)
|
||||
|
||||
```yaml
|
||||
# This is the DEFAULT (no need to specify)
|
||||
# parameter_delivery: stdin
|
||||
# parameter_format: json
|
||||
```
|
||||
|
||||
```python
|
||||
# Read from stdin
|
||||
import sys, json
|
||||
content = sys.stdin.read()
|
||||
params_str = content.split('---ATTUNE_PARAMS_END---')[0]
|
||||
params = json.loads(params_str)
|
||||
api_key = params['api_key'] # Secure!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Temporary File (`file`)
|
||||
|
||||
**Security**: ✅ HIGH - Restrictive permissions (0400)
|
||||
**When**: Large payloads, complex configs
|
||||
|
||||
```yaml
|
||||
# Explicitly use file for large payloads
|
||||
parameter_delivery: file
|
||||
parameter_format: yaml
|
||||
```
|
||||
|
||||
```python
|
||||
# Read from file
|
||||
import os, yaml
|
||||
param_file = os.environ['ATTUNE_PARAMETER_FILE']
|
||||
with open(param_file) as f:
|
||||
params = yaml.safe_load(f)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Format Options
|
||||
|
||||
| Format | Best For | Example |
|
||||
|--------|----------|---------|
|
||||
| `json` (default) | Python/Node.js, structured data | `{"key": "value"}` |
|
||||
| `dotenv` | Simple key-value when needed | `KEY='value'` |
|
||||
| `yaml` | Human-readable configs | `key: value` |
|
||||
|
||||
---
|
||||
|
||||
## Copy-Paste Templates
|
||||
|
||||
### Python Action (Secure with Stdin/JSON)
|
||||
|
||||
```yaml
|
||||
# action.yaml
|
||||
name: my_action
|
||||
ref: mypack.my_action
|
||||
runner_type: python
|
||||
entry_point: my_action.py
|
||||
parameter_delivery: stdin
|
||||
parameter_format: json
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
api_key:
|
||||
type: string
|
||||
secret: true
|
||||
```
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
# my_action.py
|
||||
import sys
|
||||
import json
|
||||
|
||||
def read_params():
|
||||
content = sys.stdin.read()
|
||||
parts = content.split('---ATTUNE_PARAMS_END---')
|
||||
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
|
||||
secrets = json.loads(parts[1].strip()) if len(parts) > 1 and parts[1].strip() else {}
|
||||
return {**params, **secrets}
|
||||
|
||||
params = read_params()
|
||||
api_key = params['api_key']
|
||||
# Use api_key securely...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Shell Action (Secure with Stdin/JSON)
|
||||
|
||||
```yaml
|
||||
# action.yaml
|
||||
name: my_script
|
||||
ref: mypack.my_script
|
||||
runner_type: shell
|
||||
entry_point: my_script.sh
|
||||
parameter_delivery: stdin
|
||||
parameter_format: json
|
||||
```
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# my_script.sh
|
||||
set -e
|
||||
|
||||
# Read params from stdin (requires jq)
|
||||
read -r PARAMS_JSON
|
||||
API_KEY=$(echo "$PARAMS_JSON" | jq -r '.api_key')
|
||||
|
||||
# Use API_KEY securely...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Shell Action (Using Stdin with Dotenv)
|
||||
|
||||
```yaml
|
||||
name: simple_script
|
||||
ref: mypack.simple_script
|
||||
runner_type: shell
|
||||
entry_point: simple.sh
|
||||
# Can use dotenv format with stdin for simple shell scripts
|
||||
parameter_delivery: stdin
|
||||
parameter_format: dotenv
|
||||
```
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# simple.sh
|
||||
# Read dotenv from stdin
|
||||
eval "$(cat)"
|
||||
echo "$MESSAGE"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
**System Variables** (always set):
|
||||
- `ATTUNE_EXECUTION_ID` - Execution ID
|
||||
- `ATTUNE_ACTION_REF` - Action reference
|
||||
- `ATTUNE_PARAMETER_DELIVERY` - Method used (stdin/file, default: stdin)
|
||||
- `ATTUNE_PARAMETER_FORMAT` - Format used (json/dotenv/yaml, default: json)
|
||||
- `ATTUNE_PARAMETER_FILE` - Path to temp file (file delivery only)
|
||||
|
||||
**Custom Variables** (from `execution.env_vars`):
|
||||
- Set any custom environment variables via `execution.env_vars` when creating execution
|
||||
- These are separate from parameters
|
||||
- Use for execution context, configuration, non-sensitive metadata
|
||||
|
||||
---
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Detect Delivery Method
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
delivery = os.environ.get('ATTUNE_PARAMETER_DELIVERY', 'env')
|
||||
if delivery == 'stdin':
|
||||
params = read_from_stdin()
|
||||
elif delivery == 'file':
|
||||
params = read_from_file()
|
||||
else:
|
||||
params = read_from_env()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Mark Sensitive Parameters
|
||||
|
||||
```yaml
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
api_key:
|
||||
type: string
|
||||
secret: true # Mark as sensitive
|
||||
password:
|
||||
type: string
|
||||
secret: true
|
||||
public_url:
|
||||
type: string # Not marked - not sensitive
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Validate Required Parameters
|
||||
|
||||
```python
|
||||
params = read_params()
|
||||
if not params.get('api_key'):
|
||||
print(json.dumps({"error": "api_key required"}))
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Checklist
|
||||
|
||||
- [ ] Identified all sensitive parameters
|
||||
- [ ] Marked sensitive params with `secret: true`
|
||||
- [ ] Set `parameter_delivery: stdin` or `file` (not `env`)
|
||||
- [ ] Set appropriate `parameter_format`
|
||||
- [ ] Updated action script to read from stdin/file
|
||||
- [ ] Tested that secrets don't appear in `ps aux`
|
||||
- [ ] Don't log sensitive parameters
|
||||
- [ ] Handle missing parameters gracefully
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Run action and check process list
|
||||
./attune execution start mypack.my_action --params '{"api_key":"secret123"}' &
|
||||
|
||||
# In another terminal
|
||||
ps aux | grep attune-worker
|
||||
# Should NOT see "secret123" in output!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Design Change (2025-02-05)
|
||||
|
||||
**Parameters and Environment Variables Are Separate**
|
||||
|
||||
**Parameters** (always secure):
|
||||
- Passed via `stdin` (default) or `file` (large payloads)
|
||||
- Never passed as environment variables
|
||||
- Read from stdin or parameter file
|
||||
|
||||
```python
|
||||
# Read parameters from stdin
|
||||
import sys, json
|
||||
content = sys.stdin.read()
|
||||
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
|
||||
api_key = params['api_key'] # Secure!
|
||||
```
|
||||
|
||||
**Environment Variables** (execution context):
|
||||
- Set via `execution.env_vars` when creating execution
|
||||
- Separate from parameters
|
||||
- Read from environment
|
||||
|
||||
```python
|
||||
# Read environment variables (context, not parameters)
|
||||
import os
|
||||
log_level = os.environ.get('LOG_LEVEL', 'info')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Don't Do This
|
||||
|
||||
```python
|
||||
# ❌ Don't log sensitive parameters
|
||||
logger.debug(f"Params: {params}") # May contain secrets!
|
||||
|
||||
# ❌ Don't confuse parameters with env vars
|
||||
# Parameters come from stdin/file, not environment
|
||||
|
||||
# ❌ Don't forget to mark secrets
|
||||
# api_key:
|
||||
# type: string
|
||||
# # Missing: secret: true
|
||||
|
||||
# ❌ Don't put sensitive data in execution.env_vars
|
||||
# Use parameters for sensitive data, env_vars for context
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Do This Instead
|
||||
|
||||
```python
|
||||
# ✅ Log only non-sensitive data
|
||||
logger.info(f"Calling endpoint: {params['endpoint']}")
|
||||
|
||||
# ✅ Use stdin for parameters (the default!)
|
||||
# parameter_delivery: stdin # No need to specify
|
||||
|
||||
# ✅ Mark all secrets
|
||||
# api_key:
|
||||
# type: string
|
||||
# secret: true
|
||||
|
||||
# ✅ Use env_vars for execution context
|
||||
# Set when creating execution:
|
||||
# {"env_vars": {"LOG_LEVEL": "debug"}}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Help & Support
|
||||
|
||||
**Full Documentation**: `docs/actions/parameter-delivery.md`
|
||||
|
||||
**Examples**: See `packs/core/actions/http_request.yaml`
|
||||
|
||||
**Questions**:
|
||||
- Parameters: Check `ATTUNE_PARAMETER_DELIVERY` env var
|
||||
- Env vars: Set via `execution.env_vars` when creating execution
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
1. **Default is `stdin` + `json` - secure by default! 🎉**
|
||||
2. **Parameters and environment variables are separate concepts**
|
||||
3. **Parameters are always secure (stdin or file, never env)**
|
||||
4. **Mark sensitive parameters with `secret: true`**
|
||||
5. **Use `execution.env_vars` for execution context, not parameters**
|
||||
6. **Test that secrets aren't in process list**
|
||||
|
||||
**Remember**: Parameters are secure by design - they're never in environment variables! 🔒
|
||||
163
docs/actions/README.md
Normal file
163
docs/actions/README.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# Action Parameter Delivery
|
||||
|
||||
This directory contains documentation for Attune's secure parameter passing system for actions.
|
||||
|
||||
## Quick Links
|
||||
|
||||
- **[Parameter Delivery Guide](./parameter-delivery.md)** - Complete guide to parameter delivery methods, formats, and best practices (568 lines)
|
||||
- **[Quick Reference](./QUICKREF-parameter-delivery.md)** - Quick decision matrix and copy-paste templates (365 lines)
|
||||
|
||||
## Overview
|
||||
|
||||
Attune provides three methods for delivering parameters to actions, with **stdin + JSON as the secure default** (as of 2025-02-05):
|
||||
|
||||
### Delivery Methods
|
||||
|
||||
| Method | Security | Use Case |
|
||||
|--------|----------|----------|
|
||||
| **stdin** (default) | ✅ High | Credentials, structured data, most actions |
|
||||
| **env** (explicit) | ⚠️ Low | Simple non-sensitive shell scripts only |
|
||||
| **file** | ✅ High | Large payloads, complex configurations |
|
||||
|
||||
### Serialization Formats
|
||||
|
||||
| Format | Best For | Example |
|
||||
|--------|----------|---------|
|
||||
| **json** (default) | Python/Node.js, structured data | `{"key": "value"}` |
|
||||
| **dotenv** | Shell scripts, simple key-value | `KEY='value'` |
|
||||
| **yaml** | Human-readable configs | `key: value` |
|
||||
|
||||
## Security Warning
|
||||
|
||||
⚠️ **Environment variables are visible in process listings** (`ps aux`, `/proc/<pid>/environ`)
|
||||
|
||||
**Never use `env` delivery for sensitive parameters** like passwords, API keys, or tokens.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Secure Action (Default - No Configuration Needed)
|
||||
|
||||
```yaml
|
||||
# action.yaml
|
||||
name: my_action
|
||||
ref: mypack.my_action
|
||||
runner_type: python
|
||||
entry_point: my_action.py
|
||||
# Uses default stdin + json (no need to specify)
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
api_key:
|
||||
type: string
|
||||
secret: true
|
||||
```
|
||||
|
||||
```python
|
||||
# my_action.py
|
||||
import sys, json
|
||||
|
||||
# Read from stdin (the default)
|
||||
content = sys.stdin.read()
|
||||
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
|
||||
api_key = params['api_key'] # Secure - not in process list!
|
||||
```
|
||||
|
||||
### Simple Shell Script (Non-Sensitive - Explicit env)
|
||||
|
||||
```yaml
|
||||
# action.yaml
|
||||
name: simple_script
|
||||
ref: mypack.simple_script
|
||||
runner_type: shell
|
||||
entry_point: simple.sh
|
||||
# Explicitly use env for non-sensitive data
|
||||
parameter_delivery: env
|
||||
parameter_format: dotenv
|
||||
```
|
||||
|
||||
```bash
|
||||
# simple.sh
|
||||
MESSAGE="${ATTUNE_ACTION_MESSAGE:-Hello}"
|
||||
echo "$MESSAGE"
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
- ✅ **Secure by default** - stdin prevents process listing exposure
|
||||
- ✅ **Type preservation** - JSON format maintains data types
|
||||
- ✅ **Automatic cleanup** - Temporary files auto-deleted
|
||||
- ✅ **Flexible formats** - Choose JSON, YAML, or dotenv
|
||||
- ✅ **Explicit opt-in** - Only use env when you really need it
|
||||
|
||||
## Environment Variables
|
||||
|
||||
All actions receive these metadata variables:
|
||||
|
||||
- `ATTUNE_PARAMETER_DELIVERY` - Method used (stdin/env/file)
|
||||
- `ATTUNE_PARAMETER_FORMAT` - Format used (json/dotenv/yaml)
|
||||
- `ATTUNE_PARAMETER_FILE` - File path (file delivery only)
|
||||
- `ATTUNE_ACTION_<KEY>` - Individual parameters (env delivery only)
|
||||
|
||||
## Breaking Change Notice
|
||||
|
||||
**As of 2025-02-05**, the default parameter delivery changed from `env` to `stdin` for security.
|
||||
|
||||
Actions that need environment variable delivery must **explicitly opt-in** by setting:
|
||||
|
||||
```yaml
|
||||
parameter_delivery: env
|
||||
parameter_format: dotenv
|
||||
```
|
||||
|
||||
This is allowed because Attune is in pre-production with no users or deployments (per AGENTS.md policy).
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. ✅ **Use default stdin + json** for most actions
|
||||
2. ✅ **Mark sensitive parameters** with `secret: true`
|
||||
3. ✅ **Only use env explicitly** for simple, non-sensitive shell scripts
|
||||
4. ✅ **Test credentials don't appear** in `ps aux` output
|
||||
5. ✅ **Never log sensitive parameters**
|
||||
|
||||
## Example Actions
|
||||
|
||||
See the core pack for examples:
|
||||
|
||||
- `packs/core/actions/http_request.yaml` - Uses stdin + json (handles API tokens)
|
||||
- `packs/core/actions/echo.yaml` - Uses env + dotenv (no secrets)
|
||||
- `packs/core/actions/sleep.yaml` - Uses env + dotenv (no secrets)
|
||||
|
||||
## Documentation Structure
|
||||
|
||||
```
|
||||
docs/actions/
|
||||
├── README.md # This file - Overview and quick links
|
||||
├── parameter-delivery.md # Complete guide (568 lines)
|
||||
│ ├── Security concerns
|
||||
│ ├── Detailed method descriptions
|
||||
│ ├── Format specifications
|
||||
│ ├── Configuration syntax
|
||||
│ ├── Best practices
|
||||
│ ├── Migration guide
|
||||
│ └── Complete examples
|
||||
└── QUICKREF-parameter-delivery.md # Quick reference (365 lines)
|
||||
├── TL;DR
|
||||
├── Decision matrix
|
||||
├── Copy-paste templates
|
||||
├── Common patterns
|
||||
└── Testing tips
|
||||
```
|
||||
|
||||
## Getting Help
|
||||
|
||||
1. **Quick decisions**: See [QUICKREF-parameter-delivery.md](./QUICKREF-parameter-delivery.md)
|
||||
2. **Detailed guide**: See [parameter-delivery.md](./parameter-delivery.md)
|
||||
3. **Check delivery method**: Look at `ATTUNE_PARAMETER_DELIVERY` env var
|
||||
4. **Test security**: Run `ps aux | grep attune-worker` to verify secrets aren't visible
|
||||
|
||||
## Summary
|
||||
|
||||
**Default**: `stdin` + `json` - Secure, structured, type-preserving parameter passing.
|
||||
|
||||
**Remember**: stdin is the default. Environment variables require explicit opt-in! 🔒
|
||||
576
docs/actions/parameter-delivery.md
Normal file
576
docs/actions/parameter-delivery.md
Normal file
@@ -0,0 +1,576 @@
|
||||
# Parameter Delivery Methods
|
||||
|
||||
**Last Updated**: 2025-02-05
|
||||
**Status**: Active Feature
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Attune provides secure parameter passing for actions with two delivery methods: **stdin** (default) and **file** (for large payloads). This document describes parameter delivery, formats, and best practices.
|
||||
|
||||
**Key Design Principle**: Action parameters and environment variables are completely separate:
|
||||
- **Parameters** - Data the action operates on (always secure: stdin or file)
|
||||
- **Environment Variables** - Execution context/configuration (set as env vars, stored in `execution.env_vars`)
|
||||
|
||||
---
|
||||
|
||||
## Security by Design
|
||||
|
||||
### Parameters Are Always Secure
|
||||
|
||||
Action parameters are **never** passed as environment variables. They are always delivered via:
|
||||
- **stdin** (default) - Secure, not visible in process listings
|
||||
- **file** - Secure temporary file with restrictive permissions (0400)
|
||||
|
||||
This ensures parameters (including sensitive data like passwords, API keys, tokens) are never exposed in process listings.
|
||||
|
||||
### Environment Variables Are Separate
|
||||
|
||||
Environment variables provide execution context and configuration:
|
||||
- Stored in `execution.env_vars` (JSONB key-value pairs)
|
||||
- Set as environment variables by the worker
|
||||
- Examples: `ATTUNE_EXECUTION_ID`, custom config values, feature flags
|
||||
- Typically non-sensitive (visible in process environment)
|
||||
|
||||
---
|
||||
|
||||
## Parameter Delivery Methods
|
||||
|
||||
### 1. Standard Input (`stdin`)
|
||||
|
||||
**Security**: ✅ **High** - Not visible in process listings
|
||||
**Use Case**: Sensitive data, structured parameters, credentials
|
||||
|
||||
Parameters are serialized in the specified format and passed via stdin. A delimiter `---ATTUNE_PARAMS_END---` separates parameters from secrets.
|
||||
|
||||
**Example** (this is the default):
|
||||
```yaml
|
||||
parameter_delivery: stdin
|
||||
parameter_format: json
|
||||
```
|
||||
|
||||
**Environment variables set**:
|
||||
- `ATTUNE_PARAMETER_DELIVERY=stdin`
|
||||
- `ATTUNE_PARAMETER_FORMAT=json`
|
||||
|
||||
**Stdin content (JSON format)**:
|
||||
```
|
||||
{"message":"Hello","count":42,"enabled":true}
|
||||
---ATTUNE_PARAMS_END---
|
||||
{"api_key":"secret123","db_password":"pass456"}
|
||||
```
|
||||
|
||||
**Python script example**:
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import json
|
||||
|
||||
def read_stdin_params():
|
||||
"""Read parameters and secrets from stdin."""
|
||||
content = sys.stdin.read()
|
||||
parts = content.split('---ATTUNE_PARAMS_END---')
|
||||
|
||||
# Parse parameters
|
||||
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
|
||||
|
||||
# Parse secrets (if present)
|
||||
secrets = {}
|
||||
if len(parts) > 1 and parts[1].strip():
|
||||
secrets = json.loads(parts[1].strip())
|
||||
|
||||
return params, secrets
|
||||
|
||||
params, secrets = read_stdin_params()
|
||||
message = params.get('message', 'default')
|
||||
api_key = secrets.get('api_key')
|
||||
print(f"Message: {message}")
|
||||
```
|
||||
|
||||
**Shell script example**:
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
# Read parameters from stdin (JSON format)
|
||||
read -r PARAMS_JSON
|
||||
# Parse JSON (requires jq)
|
||||
MESSAGE=$(echo "$PARAMS_JSON" | jq -r '.message // "default"')
|
||||
COUNT=$(echo "$PARAMS_JSON" | jq -r '.count // 0')
|
||||
|
||||
echo "Message: $MESSAGE, Count: $COUNT"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Temporary File (`file`)
|
||||
|
||||
**Security**: ✅ **High** - File has restrictive permissions (owner read-only)
|
||||
**Use Case**: Large parameter payloads, sensitive data, actions that need random access to parameters
|
||||
|
||||
Parameters are written to a temporary file with restrictive permissions (`0400` on Unix). The file path is provided via the `ATTUNE_PARAMETER_FILE` environment variable.
|
||||
|
||||
**Example**:
|
||||
```yaml
|
||||
# Explicitly set to file
|
||||
parameter_delivery: file
|
||||
parameter_format: yaml
|
||||
```
|
||||
|
||||
**Environment variables set**:
|
||||
- `ATTUNE_PARAMETER_DELIVERY=file`
|
||||
- `ATTUNE_PARAMETER_FORMAT=yaml`
|
||||
- `ATTUNE_PARAMETER_FILE=/tmp/attune-params-abc123.yaml`
|
||||
|
||||
**File content (YAML format)**:
|
||||
```yaml
|
||||
message: Hello
|
||||
count: 42
|
||||
enabled: true
|
||||
```
|
||||
|
||||
**Python script example**:
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import yaml
|
||||
|
||||
def read_file_params():
|
||||
"""Read parameters from temporary file."""
|
||||
param_file = os.environ.get('ATTUNE_PARAMETER_FILE')
|
||||
if not param_file:
|
||||
return {}
|
||||
|
||||
with open(param_file, 'r') as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
params = read_file_params()
|
||||
message = params.get('message', 'default')
|
||||
count = params.get('count', 0)
|
||||
print(f"Message: {message}, Count: {count}")
|
||||
```
|
||||
|
||||
**Shell script example**:
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
# Read from parameter file
|
||||
PARAM_FILE="${ATTUNE_PARAMETER_FILE}"
|
||||
if [ -f "$PARAM_FILE" ]; then
|
||||
# Parse YAML (requires yq or similar)
|
||||
MESSAGE=$(yq eval '.message // "default"' "$PARAM_FILE")
|
||||
COUNT=$(yq eval '.count // 0' "$PARAM_FILE")
|
||||
echo "Message: $MESSAGE, Count: $COUNT"
|
||||
fi
|
||||
```
|
||||
|
||||
**Note**: The temporary file is automatically deleted after the action completes.
|
||||
|
||||
---
|
||||
|
||||
## Parameter Formats
|
||||
|
||||
### 1. JSON (`json`)
|
||||
|
||||
**Format**: JSON object
|
||||
**Best For**: Structured data, Python/Node.js actions, complex parameters
|
||||
**Type Preservation**: Yes (strings, numbers, booleans, arrays, objects)
|
||||
|
||||
**Example**:
|
||||
```json
|
||||
{
|
||||
"message": "Hello, World!",
|
||||
"count": 42,
|
||||
"enabled": true,
|
||||
"tags": ["prod", "api"],
|
||||
"config": {
|
||||
"timeout": 30,
|
||||
"retries": 3
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Dotenv (`dotenv`)
|
||||
|
||||
**Format**: `KEY='VALUE'` (one per line)
|
||||
**Best For**: Simple key-value pairs when needed
|
||||
**Type Preservation**: No (all values are strings)
|
||||
|
||||
**Example**:
|
||||
```
|
||||
MESSAGE='Hello, World!'
|
||||
COUNT='42'
|
||||
ENABLED='true'
|
||||
```
|
||||
|
||||
**Escaping**: Single quotes in values are escaped as `'\''`
|
||||
|
||||
---
|
||||
|
||||
### 3. YAML (`yaml`)
|
||||
|
||||
**Format**: YAML document
|
||||
**Best For**: Human-readable structured data, complex configurations
|
||||
**Type Preservation**: Yes (strings, numbers, booleans, arrays, objects)
|
||||
|
||||
**Example**:
|
||||
```yaml
|
||||
message: Hello, World!
|
||||
count: 42
|
||||
enabled: true
|
||||
tags:
|
||||
- prod
|
||||
- api
|
||||
config:
|
||||
timeout: 30
|
||||
retries: 3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration in Action YAML
|
||||
|
||||
Add these fields to your action metadata file:
|
||||
|
||||
```yaml
|
||||
name: my_action
|
||||
ref: mypack.my_action
|
||||
description: "My secure action"
|
||||
runner_type: python
|
||||
entry_point: my_action.py
|
||||
|
||||
# Parameter delivery configuration (optional - these are the defaults)
|
||||
# parameter_delivery: stdin # Options: stdin, file (default: stdin)
|
||||
# parameter_format: json # Options: json, dotenv, yaml (default: json)
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
api_key:
|
||||
type: string
|
||||
description: "API key for authentication"
|
||||
secret: true # Mark sensitive parameters
|
||||
message:
|
||||
type: string
|
||||
description: "Message to process"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Choose the Right Delivery Method
|
||||
|
||||
| Scenario | Recommended Delivery | Recommended Format |
|
||||
|----------|---------------------|-------------------|
|
||||
| Most actions (default) | `stdin` | `json` |
|
||||
| Sensitive credentials | `stdin` (default) | `json` (default) |
|
||||
| Large parameter payloads (>1MB) | `file` | `json` or `yaml` |
|
||||
| Complex structured data | `stdin` (default) | `json` (default) |
|
||||
| Shell scripts | `stdin` (default) | `json` or `dotenv` |
|
||||
| Python/Node.js actions | `stdin` (default) | `json` (default) |
|
||||
|
||||
### 2. Mark Sensitive Parameters
|
||||
|
||||
Always mark sensitive parameters with `secret: true` in the parameter schema:
|
||||
|
||||
```yaml
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
password:
|
||||
type: string
|
||||
secret: true
|
||||
api_token:
|
||||
type: string
|
||||
secret: true
|
||||
```
|
||||
|
||||
### 3. Handle Missing Parameters Gracefully
|
||||
|
||||
```python
|
||||
# Python example
|
||||
params = read_params()
|
||||
api_key = params.get('api_key')
|
||||
if not api_key:
|
||||
print("ERROR: api_key parameter is required", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
```
|
||||
|
||||
```bash
|
||||
# Shell example
|
||||
if [ -z "$ATTUNE_ACTION_API_KEY" ]; then
|
||||
echo "ERROR: api_key parameter is required" >&2
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### 4. Validate Parameter Format
|
||||
|
||||
Check the `ATTUNE_PARAMETER_DELIVERY` environment variable to determine how parameters were delivered:
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
delivery_method = os.environ.get('ATTUNE_PARAMETER_DELIVERY', 'env')
|
||||
param_format = os.environ.get('ATTUNE_PARAMETER_FORMAT', 'dotenv')
|
||||
|
||||
if delivery_method == 'env':
|
||||
# Read from environment variables
|
||||
params = read_env_params()
|
||||
elif delivery_method == 'stdin':
|
||||
# Read from stdin
|
||||
params = read_stdin_params()
|
||||
elif delivery_method == 'file':
|
||||
# Read from file
|
||||
params = read_file_params()
|
||||
```
|
||||
|
||||
### 5. Clean Up Sensitive Data
|
||||
|
||||
For file-based delivery, the system automatically deletes the temporary file. For stdin/env, ensure sensitive data doesn't leak into logs:
|
||||
|
||||
```python
|
||||
# Don't log sensitive parameters
|
||||
logger.info(f"Processing request for user: {params['username']}")
|
||||
# Don't do this:
|
||||
# logger.debug(f"Full params: {params}") # May contain secrets!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
### Parameters vs Environment Variables
|
||||
|
||||
**Action Parameters** (`stdin` or `file`):
|
||||
- Data the action operates on
|
||||
- Always secure (never in environment)
|
||||
- Examples: API payloads, credentials, business data
|
||||
- Stored in `execution.config` → `parameters`
|
||||
- Passed via stdin or temporary file
|
||||
|
||||
**Environment Variables** (`execution.env_vars`):
|
||||
- Execution context and configuration
|
||||
- Set as environment variables by worker
|
||||
- Examples: `ATTUNE_EXECUTION_ID`, custom config, feature flags
|
||||
- Stored in `execution.env_vars` JSONB
|
||||
- Typically non-sensitive
|
||||
|
||||
### Default Behavior (Secure by Default)
|
||||
|
||||
**As of 2025-02-05**: Parameters default to:
|
||||
- `parameter_delivery: stdin`
|
||||
- `parameter_format: json`
|
||||
|
||||
All action parameters are secure by design. There is no option to pass parameters as environment variables.
|
||||
|
||||
### Migration from Environment Variables
|
||||
|
||||
If you were previously passing data as environment variables, you now have two options:
|
||||
|
||||
**Option 1: Move to Parameters** (for action data):
|
||||
```python
|
||||
# Read from stdin
|
||||
import sys, json
|
||||
content = sys.stdin.read()
|
||||
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
|
||||
value = params.get('key')
|
||||
```
|
||||
|
||||
**Option 2: Use execution.env_vars** (for execution context):
|
||||
Store non-sensitive configuration in `execution.env_vars` when creating the execution:
|
||||
```json
|
||||
{
|
||||
"action_ref": "mypack.myaction",
|
||||
"parameters": {"data": "value"},
|
||||
"env_vars": {"CUSTOM_CONFIG": "value"}
|
||||
}
|
||||
```
|
||||
|
||||
Then read from environment in action:
|
||||
```python
|
||||
import os
|
||||
config = os.environ.get('CUSTOM_CONFIG')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Complete Python Action with Stdin/JSON
|
||||
|
||||
**Action YAML** (`mypack/actions/secure_action.yaml`):
|
||||
```yaml
|
||||
name: secure_action
|
||||
ref: mypack.secure_action
|
||||
description: "Secure action with stdin parameter delivery"
|
||||
runner_type: python
|
||||
entry_point: secure_action.py
|
||||
# Uses default stdin + json (no need to specify)
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
api_token:
|
||||
type: string
|
||||
secret: true
|
||||
endpoint:
|
||||
type: string
|
||||
data:
|
||||
type: object
|
||||
required:
|
||||
- api_token
|
||||
- endpoint
|
||||
```
|
||||
|
||||
**Action Script** (`mypack/actions/secure_action.py`):
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import sys
|
||||
import json
|
||||
import requests
|
||||
|
||||
def read_stdin_params():
|
||||
"""Read parameters and secrets from stdin."""
|
||||
content = sys.stdin.read()
|
||||
parts = content.split('---ATTUNE_PARAMS_END---')
|
||||
|
||||
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
|
||||
secrets = {}
|
||||
if len(parts) > 1 and parts[1].strip():
|
||||
secrets = json.loads(parts[1].strip())
|
||||
|
||||
return {**params, **secrets}
|
||||
|
||||
def main():
|
||||
params = read_stdin_params()
|
||||
|
||||
api_token = params.get('api_token')
|
||||
endpoint = params.get('endpoint')
|
||||
data = params.get('data', {})
|
||||
|
||||
if not api_token or not endpoint:
|
||||
print(json.dumps({"error": "Missing required parameters"}))
|
||||
sys.exit(1)
|
||||
|
||||
headers = {"Authorization": f"Bearer {api_token}"}
|
||||
response = requests.post(endpoint, json=data, headers=headers)
|
||||
|
||||
result = {
|
||||
"status_code": response.status_code,
|
||||
"response": response.json() if response.ok else None,
|
||||
"success": response.ok
|
||||
}
|
||||
|
||||
print(json.dumps(result))
|
||||
sys.exit(0 if response.ok else 1)
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
### Complete Shell Action with File/YAML
|
||||
|
||||
**Action YAML** (`mypack/actions/process_config.yaml`):
|
||||
```yaml
|
||||
name: process_config
|
||||
ref: mypack.process_config
|
||||
description: "Process configuration with file-based parameter delivery"
|
||||
runner_type: shell
|
||||
entry_point: process_config.sh
|
||||
# Explicitly use file delivery for large configs
|
||||
parameter_delivery: file
|
||||
parameter_format: yaml
|
||||
|
||||
parameters:
|
||||
type: object
|
||||
properties:
|
||||
config:
|
||||
type: object
|
||||
description: "Configuration object"
|
||||
environment:
|
||||
type: string
|
||||
enum: [dev, staging, prod]
|
||||
required:
|
||||
- config
|
||||
```
|
||||
|
||||
**Action Script** (`mypack/actions/process_config.sh`):
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
# Check if parameter file exists
|
||||
if [ -z "$ATTUNE_PARAMETER_FILE" ]; then
|
||||
echo "ERROR: No parameter file provided" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Read configuration from YAML file (requires yq)
|
||||
ENVIRONMENT=$(yq eval '.environment // "dev"' "$ATTUNE_PARAMETER_FILE")
|
||||
CONFIG=$(yq eval '.config' "$ATTUNE_PARAMETER_FILE")
|
||||
|
||||
echo "Processing configuration for environment: $ENVIRONMENT"
|
||||
echo "Config: $CONFIG"
|
||||
|
||||
# Process configuration...
|
||||
# Your logic here
|
||||
|
||||
echo "Configuration processed successfully"
|
||||
exit 0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables Reference
|
||||
|
||||
Actions automatically receive these environment variables:
|
||||
|
||||
**System Variables** (always set):
|
||||
- `ATTUNE_EXECUTION_ID` - Current execution ID
|
||||
- `ATTUNE_ACTION_REF` - Action reference (e.g., "mypack.myaction")
|
||||
- `ATTUNE_PARAMETER_DELIVERY` - Delivery method (stdin/file)
|
||||
- `ATTUNE_PARAMETER_FORMAT` - Format used (json/dotenv/yaml)
|
||||
- `ATTUNE_PARAMETER_FILE` - File path (only for file delivery)
|
||||
|
||||
**Custom Variables** (from `execution.env_vars`):
|
||||
Any key-value pairs in `execution.env_vars` are set as environment variables.
|
||||
|
||||
Example:
|
||||
```json
|
||||
{
|
||||
"env_vars": {
|
||||
"LOG_LEVEL": "debug",
|
||||
"RETRY_COUNT": "3"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Action receives:
|
||||
```bash
|
||||
LOG_LEVEL=debug
|
||||
RETRY_COUNT=3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Pack Structure](../packs/pack-structure.md)
|
||||
- [Action Development Guide](./action-development-guide.md) (future)
|
||||
- [Secrets Management](../authentication/secrets-management.md)
|
||||
- [Security Best Practices](../authentication/security-review-2024-01-02.md)
|
||||
- [Execution API](../api/api-executions.md)
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
For questions or issues related to parameter delivery:
|
||||
1. Check the action logs for parameter delivery metadata
|
||||
2. Verify the `ATTUNE_PARAMETER_DELIVERY` and `ATTUNE_PARAMETER_FORMAT` environment variables
|
||||
3. Test with a simple action first before implementing complex parameter handling
|
||||
4. Review the example actions in the `core` pack for reference implementations
|
||||
Reference in New Issue
Block a user