669 lines
19 KiB
Markdown
669 lines
19 KiB
Markdown
# Secure Parameter Delivery - Final Implementation Summary
|
|
|
|
**Date**: 2025-02-05
|
|
**Status**: ✅ Complete
|
|
**Type**: Security Enhancement + Architecture Improvement
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
Implemented a **secure-by-design** parameter passing system for Attune actions that:
|
|
|
|
1. **Eliminates security vulnerability** - Parameters never passed as environment variables
|
|
2. **Separates concerns** - Action parameters vs execution environment variables
|
|
3. **Secure by default** - stdin + JSON for all parameters
|
|
4. **Simple choices** - Just two delivery methods: stdin (default) or file (large payloads)
|
|
|
|
**Key Achievement**: It is now **impossible** to accidentally expose sensitive parameters in process listings.
|
|
|
|
---
|
|
|
|
## Problem Statement
|
|
|
|
### Original Security Vulnerability
|
|
|
|
Environment variables are visible to any user who can inspect running processes:
|
|
- `ps aux` command
|
|
- `/proc/<pid>/environ` file
|
|
- System monitoring tools
|
|
|
|
**Impact**: Passwords, API keys, and credentials were exposed in process listings when passed as environment variables.
|
|
|
|
### Design Confusion
|
|
|
|
The original approach mixed two concepts:
|
|
- **Action Parameters** (data the action operates on)
|
|
- **Environment Variables** (execution context/configuration)
|
|
|
|
This led to unclear usage patterns and security risks.
|
|
|
|
---
|
|
|
|
## Solution Architecture
|
|
|
|
### Core Design Principle
|
|
|
|
**Parameters and Environment Variables Are Separate**:
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ EXECUTION │
|
|
├─────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────────────┐ ┌──────────────────────┐ │
|
|
│ │ PARAMETERS │ │ ENV VARS │ │
|
|
│ │ (action data) │ │ (execution context) │ │
|
|
│ ├──────────────────────┤ ├──────────────────────┤ │
|
|
│ │ • Always secure │ │ • Set as env vars │ │
|
|
│ │ • stdin or file │ │ • From env_vars JSON │ │
|
|
│ │ • Never in env │ │ • Non-sensitive │ │
|
|
│ │ • API payloads │ │ • Configuration │ │
|
|
│ │ • Credentials │ │ • Feature flags │ │
|
|
│ │ • Business data │ │ • Context metadata │ │
|
|
│ └──────────────────────┘ └──────────────────────┘ │
|
|
│ ▼ ▼ │
|
|
│ Via stdin/file Set in process env │
|
|
└─────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Parameter Delivery Methods
|
|
|
|
**Only Two Options** (env removed entirely):
|
|
|
|
1. **stdin** (DEFAULT)
|
|
- Secure, not visible in process listings
|
|
- Good for most actions
|
|
- Supports JSON, dotenv, YAML formats
|
|
|
|
2. **file**
|
|
- Secure temporary file (mode 0400)
|
|
- Good for large payloads (>1MB)
|
|
- Automatic cleanup after execution
|
|
|
|
### Environment Variables (Separate)
|
|
|
|
- Stored in `execution.env_vars` (JSONB in database)
|
|
- Set as environment variables by worker
|
|
- Used for execution context, not sensitive data
|
|
- Examples: `ATTUNE_EXECUTION_ID`, custom config values
|
|
|
|
---
|
|
|
|
## Implementation Details
|
|
|
|
### 1. Database Schema
|
|
|
|
**Migration 1**: `20250205000001_action_parameter_delivery.sql`
|
|
```sql
|
|
ALTER TABLE action
|
|
ADD COLUMN parameter_delivery TEXT NOT NULL DEFAULT 'stdin'
|
|
CHECK (parameter_delivery IN ('stdin', 'file'));
|
|
|
|
ALTER TABLE action
|
|
ADD COLUMN parameter_format TEXT NOT NULL DEFAULT 'json'
|
|
CHECK (parameter_format IN ('dotenv', 'json', 'yaml'));
|
|
```
|
|
|
|
**Migration 2**: `20250205000002_execution_env_vars.sql`
|
|
```sql
|
|
ALTER TABLE execution
|
|
ADD COLUMN env_vars JSONB;
|
|
|
|
CREATE INDEX idx_execution_env_vars_gin ON execution USING GIN (env_vars);
|
|
```
|
|
|
|
### 2. Data Models
|
|
|
|
**ParameterDelivery Enum** (crates/common/src/models.rs):
|
|
```rust
|
|
pub enum ParameterDelivery {
|
|
Stdin, // Standard input (DEFAULT)
|
|
File, // Temporary file
|
|
// NO Env option - removed for security
|
|
}
|
|
|
|
impl Default for ParameterDelivery {
|
|
fn default() -> Self {
|
|
Self::Stdin
|
|
}
|
|
}
|
|
```
|
|
|
|
**ParameterFormat Enum**:
|
|
```rust
|
|
pub enum ParameterFormat {
|
|
Json, // JSON object (DEFAULT)
|
|
Dotenv, // KEY='VALUE' format
|
|
Yaml, // YAML document
|
|
}
|
|
|
|
impl Default for ParameterFormat {
|
|
fn default() -> Self {
|
|
Self::Json
|
|
}
|
|
}
|
|
```
|
|
|
|
**Action Model** (updated):
|
|
```rust
|
|
pub struct Action {
|
|
// ... existing fields
|
|
pub parameter_delivery: ParameterDelivery,
|
|
pub parameter_format: ParameterFormat,
|
|
}
|
|
```
|
|
|
|
**Execution Model** (updated):
|
|
```rust
|
|
pub struct Execution {
|
|
// ... existing fields
|
|
pub env_vars: Option<JsonDict>, // NEW: separate from parameters
|
|
}
|
|
```
|
|
|
|
### 3. Parameter Passing Module
|
|
|
|
**File**: `crates/worker/src/runtime/parameter_passing.rs` (NEW, 384 lines)
|
|
|
|
**Key Functions**:
|
|
- `format_parameters()` - Serializes parameters in specified format
|
|
- `format_json()`, `format_dotenv()`, `format_yaml()` - Format converters
|
|
- `create_parameter_file()` - Creates secure temp file (mode 0400)
|
|
- `prepare_parameters()` - Main entry point for parameter preparation
|
|
|
|
**PreparedParameters Enum**:
|
|
```rust
|
|
pub enum PreparedParameters {
|
|
Stdin(String), // Parameters as formatted string for stdin
|
|
File { // Parameters in temporary file
|
|
path: PathBuf,
|
|
temp_file: NamedTempFile,
|
|
},
|
|
}
|
|
```
|
|
|
|
**Security Features**:
|
|
- Temporary files created with restrictive permissions (0400 on Unix)
|
|
- Automatic cleanup of temporary files
|
|
- Delimiter separation (`---ATTUNE_PARAMS_END---`) for parameters and secrets
|
|
|
|
### 4. Runtime Integration
|
|
|
|
**Shell Runtime** (crates/worker/src/runtime/shell.rs):
|
|
```rust
|
|
async fn execute(&self, context: ExecutionContext) -> RuntimeResult<ExecutionResult> {
|
|
// Prepare parameters according to delivery method
|
|
let mut env = context.env.clone();
|
|
let config = ParameterDeliveryConfig {
|
|
delivery: context.parameter_delivery,
|
|
format: context.parameter_format,
|
|
};
|
|
|
|
let prepared_params = parameter_passing::prepare_parameters(
|
|
&context.parameters,
|
|
&mut env,
|
|
config,
|
|
)?;
|
|
|
|
// Get stdin content if using stdin delivery
|
|
let parameters_stdin = prepared_params.stdin_content();
|
|
|
|
// Execute with parameters via stdin or file
|
|
self.execute_shell_file(
|
|
code_path,
|
|
&context.secrets,
|
|
&env,
|
|
parameters_stdin,
|
|
// ... other args
|
|
).await
|
|
}
|
|
```
|
|
|
|
**Native Runtime** (crates/worker/src/runtime/native.rs):
|
|
- Similar updates to support stdin and file parameter delivery
|
|
- Writes parameters to stdin before secrets
|
|
- All test contexts updated with new required fields
|
|
|
|
### 5. Pack Loader
|
|
|
|
**File**: `scripts/load_core_pack.py` (updated)
|
|
|
|
```python
|
|
# Parameter delivery and format (defaults: stdin + json for security)
|
|
parameter_delivery = action_data.get("parameter_delivery", "stdin").lower()
|
|
parameter_format = action_data.get("parameter_format", "json").lower()
|
|
|
|
# Validate parameter delivery method (only stdin and file allowed)
|
|
if parameter_delivery not in ["stdin", "file"]:
|
|
print(f" ⚠ Invalid parameter_delivery '{parameter_delivery}', defaulting to 'stdin'")
|
|
parameter_delivery = "stdin"
|
|
```
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
### Action YAML Syntax
|
|
|
|
```yaml
|
|
name: my_action
|
|
ref: mypack.my_action
|
|
description: "Secure action with credential handling"
|
|
runner_type: python
|
|
entry_point: my_action.py
|
|
|
|
# Parameter delivery (optional - these are the defaults)
|
|
# parameter_delivery: stdin # Options: stdin, file (default: stdin)
|
|
# parameter_format: json # Options: json, dotenv, yaml (default: json)
|
|
|
|
parameters:
|
|
type: object
|
|
properties:
|
|
api_key:
|
|
type: string
|
|
secret: true # Mark sensitive parameters
|
|
```
|
|
|
|
### Execution Configuration
|
|
|
|
When creating an execution, parameters and environment variables are separate:
|
|
|
|
```json
|
|
{
|
|
"action_ref": "mypack.my_action",
|
|
"parameters": {
|
|
"api_key": "secret123",
|
|
"data": {"foo": "bar"}
|
|
},
|
|
"env_vars": {
|
|
"LOG_LEVEL": "debug",
|
|
"FEATURE_FLAG": "enabled"
|
|
}
|
|
}
|
|
```
|
|
|
|
**Result**:
|
|
- `api_key` and `data` passed via stdin (secure, not visible in `ps`)
|
|
- `LOG_LEVEL` and `FEATURE_FLAG` set as environment variables
|
|
|
|
---
|
|
|
|
## Code Examples
|
|
|
|
### Python Action (Default stdin + json)
|
|
|
|
**Action YAML**:
|
|
```yaml
|
|
name: secure_action
|
|
ref: mypack.secure_action
|
|
runner_type: python
|
|
entry_point: secure_action.py
|
|
# Uses default stdin + json (no need to specify)
|
|
```
|
|
|
|
**Action Script**:
|
|
```python
|
|
#!/usr/bin/env python3
|
|
import sys
|
|
import json
|
|
import os
|
|
|
|
def read_stdin_params():
|
|
"""Read parameters from stdin."""
|
|
content = sys.stdin.read()
|
|
parts = content.split('---ATTUNE_PARAMS_END---')
|
|
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
|
|
secrets = json.loads(parts[1].strip()) if len(parts) > 1 and parts[1].strip() else {}
|
|
return {**params, **secrets}
|
|
|
|
def main():
|
|
# Read parameters (secure)
|
|
params = read_stdin_params()
|
|
api_key = params.get('api_key') # Not in process list!
|
|
|
|
# Read environment variables (context)
|
|
log_level = os.environ.get('LOG_LEVEL', 'info')
|
|
|
|
# Use parameters and env vars...
|
|
print(json.dumps({"success": True}))
|
|
|
|
if __name__ == "__main__":
|
|
main()
|
|
```
|
|
|
|
### Shell Action (stdin + dotenv format)
|
|
|
|
**Action YAML**:
|
|
```yaml
|
|
name: shell_script
|
|
ref: mypack.shell_script
|
|
runner_type: shell
|
|
entry_point: script.sh
|
|
parameter_delivery: stdin
|
|
parameter_format: dotenv
|
|
```
|
|
|
|
**Action Script**:
|
|
```bash
|
|
#!/bin/bash
|
|
set -e
|
|
|
|
# Read dotenv from stdin
|
|
eval "$(cat)"
|
|
|
|
# Use parameters (from stdin)
|
|
echo "Message: $MESSAGE"
|
|
|
|
# Use environment variables (from execution context)
|
|
echo "Log Level: $LOG_LEVEL"
|
|
```
|
|
|
|
### File-Based Delivery (Large Payloads)
|
|
|
|
**Action YAML**:
|
|
```yaml
|
|
name: large_config
|
|
ref: mypack.large_config
|
|
runner_type: python
|
|
entry_point: process.py
|
|
parameter_delivery: file
|
|
parameter_format: yaml
|
|
```
|
|
|
|
**Action Script**:
|
|
```python
|
|
#!/usr/bin/env python3
|
|
import os
|
|
import yaml
|
|
|
|
# Read from parameter file
|
|
param_file = os.environ['ATTUNE_PARAMETER_FILE']
|
|
with open(param_file, 'r') as f:
|
|
params = yaml.safe_load(f)
|
|
|
|
# File has mode 0400 - only owner can read
|
|
# File automatically deleted after execution
|
|
```
|
|
|
|
---
|
|
|
|
## Security Improvements
|
|
|
|
### Before This Implementation
|
|
|
|
```bash
|
|
# Parameters visible to anyone with ps access
|
|
$ ps aux | grep attune-worker
|
|
... ATTUNE_ACTION_DB_PASSWORD=secret123 ...
|
|
```
|
|
|
|
**Risk**: Credentials exposed in process listings
|
|
|
|
### After This Implementation
|
|
|
|
```bash
|
|
# Parameters NOT visible in process list
|
|
$ ps aux | grep attune-worker
|
|
... ATTUNE_PARAMETER_DELIVERY=stdin ATTUNE_PARAMETER_FORMAT=json ...
|
|
```
|
|
|
|
**Security**: Parameters delivered securely via stdin or temporary files
|
|
|
|
### Security Guarantees
|
|
|
|
1. **Parameters Never in Environment** - No option to pass as env vars
|
|
2. **Stdin Not Visible** - Not exposed in process listings
|
|
3. **File Permissions** - Temporary files mode 0400 (owner read-only)
|
|
4. **Automatic Cleanup** - Temp files deleted after execution
|
|
5. **Separation of Concerns** - Parameters vs env vars clearly separated
|
|
|
|
---
|
|
|
|
## Breaking Changes
|
|
|
|
### What Changed
|
|
|
|
1. **Removed `env` delivery option** - Parameters can no longer be passed as environment variables
|
|
2. **Added `execution.env_vars`** - Separate field for environment variables
|
|
3. **Defaults changed** - stdin + json (was env + dotenv)
|
|
|
|
### Justification
|
|
|
|
Per `AGENTS.md`: "Breaking changes are explicitly allowed and encouraged when they improve the architecture, API design, or developer experience. This project is under active development with no users, deployments, or stable releases."
|
|
|
|
**Why This Is Better**:
|
|
- **Secure by design** - Impossible to accidentally expose parameters
|
|
- **Clear separation** - Parameters (data) vs env vars (context)
|
|
- **Simpler choices** - Only 2 delivery methods instead of 3
|
|
- **Better defaults** - Secure by default (stdin + json)
|
|
|
|
---
|
|
|
|
## Documentation
|
|
|
|
### Created
|
|
|
|
- `docs/actions/parameter-delivery.md` (568 lines) - Complete guide
|
|
- `docs/actions/QUICKREF-parameter-delivery.md` (365 lines) - Quick reference
|
|
- `docs/actions/README.md` (163 lines) - Directory overview
|
|
|
|
### Updated
|
|
|
|
- `docs/packs/pack-structure.md` - Parameter delivery examples
|
|
- `work-summary/2025-02-05-secure-parameter-delivery.md` (542 lines)
|
|
- `work-summary/changelogs/CHANGELOG.md` - Feature entry
|
|
|
|
---
|
|
|
|
## Testing
|
|
|
|
### Unit Tests
|
|
|
|
Added comprehensive tests in `parameter_passing.rs`:
|
|
- ✅ `test_format_dotenv()` - Dotenv formatting with escaping
|
|
- ✅ `test_format_json()` - JSON serialization
|
|
- ✅ `test_format_yaml()` - YAML serialization
|
|
- ✅ `test_create_parameter_file()` - Temp file creation
|
|
- ✅ `test_prepare_parameters_stdin()` - Stdin delivery
|
|
- ✅ `test_prepare_parameters_file()` - File delivery
|
|
|
|
### Integration Testing
|
|
|
|
All runtime tests updated:
|
|
- Shell runtime tests - All ExecutionContext structures updated
|
|
- Native runtime tests - Use test_context helper (already updated)
|
|
- All tests pass with new required fields
|
|
|
|
---
|
|
|
|
## Migration Guide
|
|
|
|
### For New Actions
|
|
|
|
**No changes needed!** - Defaults are secure:
|
|
```yaml
|
|
# This is all you need (or omit - it's the default)
|
|
parameter_delivery: stdin
|
|
parameter_format: json
|
|
```
|
|
|
|
Write action to read from stdin:
|
|
```python
|
|
import sys, json
|
|
content = sys.stdin.read()
|
|
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
|
|
```
|
|
|
|
### For Execution Context
|
|
|
|
**Use env_vars for non-sensitive context**:
|
|
```json
|
|
{
|
|
"action_ref": "mypack.action",
|
|
"parameters": {"data": "value"},
|
|
"env_vars": {"LOG_LEVEL": "debug"}
|
|
}
|
|
```
|
|
|
|
Read in action:
|
|
```python
|
|
import os
|
|
log_level = os.environ.get('LOG_LEVEL', 'info')
|
|
```
|
|
|
|
---
|
|
|
|
## Environment Variables Reference
|
|
|
|
### System Variables (Always Set)
|
|
|
|
- `ATTUNE_EXECUTION_ID` - Current execution ID
|
|
- `ATTUNE_ACTION_REF` - Action reference (e.g., "mypack.action")
|
|
- `ATTUNE_PARAMETER_DELIVERY` - Method used (stdin/file)
|
|
- `ATTUNE_PARAMETER_FORMAT` - Format used (json/dotenv/yaml)
|
|
- `ATTUNE_PARAMETER_FILE` - File path (only for file delivery)
|
|
|
|
### Custom Variables (From execution.env_vars)
|
|
|
|
Any key-value pairs in `execution.env_vars` are set as environment variables:
|
|
|
|
```json
|
|
{
|
|
"env_vars": {
|
|
"LOG_LEVEL": "debug",
|
|
"RETRY_COUNT": "3",
|
|
"FEATURE_ENABLED": "true"
|
|
}
|
|
}
|
|
```
|
|
|
|
Action receives:
|
|
```bash
|
|
LOG_LEVEL=debug
|
|
RETRY_COUNT=3
|
|
FEATURE_ENABLED=true
|
|
```
|
|
|
|
---
|
|
|
|
## Performance Impact
|
|
|
|
### Minimal Overhead
|
|
|
|
- **stdin delivery**: Negligible (milliseconds for JSON/YAML parsing)
|
|
- **file delivery**: Slight overhead for I/O, beneficial for large payloads
|
|
- **Memory usage**: Unchanged (parameters were already in memory)
|
|
|
|
### Resource Cleanup
|
|
|
|
- Temporary files automatically deleted after execution
|
|
- No resource leaks
|
|
- GIN index on env_vars for efficient querying
|
|
|
|
---
|
|
|
|
## Compliance & Security Standards
|
|
|
|
### Standards Addressed
|
|
|
|
- ✅ **OWASP** - "Sensitive Data Exposure" vulnerability eliminated
|
|
- ✅ **CWE-214** - Information Exposure Through Process Environment (fixed)
|
|
- ✅ **PCI DSS Requirement 3** - Protect stored cardholder data
|
|
- ✅ **Principle of Least Privilege** - Parameters not visible to other processes
|
|
|
|
### Security Posture Improvements
|
|
|
|
1. **Defense in Depth** - Multiple layers prevent exposure
|
|
2. **Secure by Default** - No insecure options available
|
|
3. **Fail-Safe Defaults** - Default to most secure option
|
|
4. **Clear Separation** - Sensitive data vs configuration clearly separated
|
|
|
|
---
|
|
|
|
## Best Practices for Developers
|
|
|
|
### ✅ Do This
|
|
|
|
1. **Use default stdin + json** for most actions
|
|
2. **Mark sensitive parameters** with `secret: true`
|
|
3. **Use execution.env_vars** for execution context
|
|
4. **Test parameters not in `ps aux`** output
|
|
5. **Never log sensitive parameters**
|
|
|
|
### ❌ Don't Do This
|
|
|
|
1. Don't put sensitive data in `execution.env_vars` - use parameters
|
|
2. Don't log full parameter objects (may contain secrets)
|
|
3. Don't confuse parameters with environment variables
|
|
4. Don't try to read parameters from environment (they're not there!)
|
|
|
|
---
|
|
|
|
## Future Enhancements
|
|
|
|
### Potential Improvements
|
|
|
|
1. **Encrypted Parameter Files** - Encrypt temp files for additional security
|
|
2. **Parameter Validation** - Validate against schema before delivery
|
|
3. **Audit Logging** - Log parameter access for compliance
|
|
4. **Per-Parameter Delivery** - Different methods for different parameters
|
|
5. **Memory-Only Delivery** - Pass via shared memory (no disk I/O)
|
|
|
|
---
|
|
|
|
## Related Files
|
|
|
|
### New Files
|
|
- `migrations/20250205000001_action_parameter_delivery.sql`
|
|
- `migrations/20250205000002_execution_env_vars.sql`
|
|
- `crates/worker/src/runtime/parameter_passing.rs`
|
|
- `docs/actions/parameter-delivery.md`
|
|
- `docs/actions/QUICKREF-parameter-delivery.md`
|
|
- `docs/actions/README.md`
|
|
- `work-summary/2025-02-05-secure-parameter-delivery.md`
|
|
- `work-summary/2025-02-05-FINAL-secure-parameters.md` (this file)
|
|
|
|
### Modified Files
|
|
- `crates/common/src/models.rs` (ParameterDelivery, ParameterFormat enums, Execution model)
|
|
- `crates/worker/src/runtime/mod.rs` (ExecutionContext, exports)
|
|
- `crates/worker/src/runtime/shell.rs` (parameter passing integration)
|
|
- `crates/worker/src/runtime/native.rs` (parameter passing integration)
|
|
- `crates/worker/src/executor.rs` (prepare_execution_context)
|
|
- `crates/worker/Cargo.toml` (dependencies)
|
|
- `scripts/load_core_pack.py` (parameter delivery validation)
|
|
- `packs/core/actions/*.yaml` (updated to use defaults)
|
|
- `docs/packs/pack-structure.md` (examples and documentation)
|
|
- `work-summary/changelogs/CHANGELOG.md` (feature entry)
|
|
|
|
---
|
|
|
|
## Conclusion
|
|
|
|
This implementation provides **secure-by-design** parameter passing for Attune actions:
|
|
|
|
### Key Achievements
|
|
|
|
1. ✅ **Eliminated security vulnerability** - Parameters never in process listings
|
|
2. ✅ **Clear separation of concerns** - Parameters vs environment variables
|
|
3. ✅ **Secure by default** - stdin + json for all actions
|
|
4. ✅ **Impossible to misconfigure** - No insecure options available
|
|
5. ✅ **Simple to use** - Just read from stdin (default)
|
|
6. ✅ **Comprehensive documentation** - 1100+ lines of docs
|
|
7. ✅ **Full test coverage** - Unit and integration tests
|
|
8. ✅ **Zero compilation warnings** - Clean build
|
|
|
|
### Impact
|
|
|
|
**Before**: Credentials could be accidentally exposed via environment variables
|
|
**After**: Parameters are secure by design - no way to expose them accidentally
|
|
|
|
This provides a strong security foundation for the Attune platform from day one, eliminating an entire class of security vulnerabilities before they can affect any production deployments.
|
|
|
|
---
|
|
|
|
**Implementation Date**: 2025-02-05
|
|
**Status**: ✅ Complete and Ready for Use
|
|
**Build Status**: ✅ All packages compile successfully
|
|
**Test Status**: ✅ All tests pass
|
|
**Documentation**: ✅ Comprehensive (1100+ lines) |