attune-system/attune
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

re-uploading work

Browse Source
This commit is contained in:
David Culbreth
2026-02-04 17:46:30 -06:00
commit 3b14c65998
1388 changed files with 381262 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

367
docs/authentication/secrets-management.md Normal file
View File

@@ -0,0 +1,367 @@
# Secrets Management in Attune Worker Service
## Overview
The Attune Worker Service includes a robust secrets management system that securely stores, retrieves, and injects secrets into action execution environments. Secrets are encrypted at rest in the database and decrypted on-demand during execution.
## Architecture
### Components
1. **SecretManager** (`crates/worker/src/secrets.rs`)
- Core component responsible for secret operations
- Handles fetching, decryption, and environment variable preparation
- Integrated into `ActionExecutor` for seamless secret injection
2. **Database Storage** (`attune.key` table)
- Stores secrets with ownership scoping (system, pack, action, sensor, identity)
- Supports both encrypted and plaintext values
- Tracks encryption key hash for validation
3. **Encryption System**
- Uses **AES-256-GCM** for authenticated encryption
- Derives encryption key from configured password using SHA-256
- Generates random nonces for each encryption operation
## Secret Ownership Hierarchy
Secrets are organized in a hierarchical ownership model with increasing specificity:
### 1. System-Level Secrets
- **Owner Type**: `system`
- **Scope**: Available to all actions across all packs
- **Use Case**: Global configuration (API endpoints, common credentials)
### 2. Pack-Level Secrets
- **Owner Type**: `pack`
- **Scope**: Available to all actions within a specific pack
- **Use Case**: Pack-specific credentials, service endpoints
### 3. Action-Level Secrets
- **Owner Type**: `action`
- **Scope**: Available only to a specific action
- **Use Case**: Action-specific credentials, sensitive parameters
### Override Behavior
When an action is executed, secrets are fetched in the following order:
1. System secrets
2. Pack secrets (override system secrets with same name)
3. Action secrets (override pack/system secrets with same name)
This allows for flexible secret management where more specific secrets override less specific ones.
## Encryption Format
### Encrypted Value Format
```
nonce:ciphertext
```
Both components are Base64-encoded:
- **Nonce**: 12-byte random value (96 bits) for AES-GCM
- **Ciphertext**: Encrypted payload with authentication tag
Example:
```
Xk3mP9qRsT6uVwYz:SGVsbG8gV29ybGQhIFRoaXMgaXMgYW4gZW5jcnlwdGVkIG1lc3NhZ2U=
```
### Encryption Key Derivation
The encryption key is derived from the configured password using SHA-256:
```
encryption_key = SHA256(password)
```
This produces a 32-byte (256-bit) key suitable for AES-256.
### Key Hash Validation
Each encrypted secret can optionally store the hash of the encryption key used to encrypt it:
```
key_hash = SHA256(encryption_key)
```
This allows validation that the correct key is being used for decryption.
## Configuration
### Security Configuration
Add to your `config.yaml`:
```yaml
security:
# Encryption key for secrets (REQUIRED for encrypted secrets)
encryption_key: "your-secret-encryption-password-here"
# Or use environment variable
# ATTUNE__SECURITY__ENCRYPTION_KEY=your-secret-encryption-password-here
```
⚠️ **Important Security Notes:**
- The encryption key should be a strong, random password (minimum 32 characters recommended)
- Store the encryption key securely (e.g., using a secrets manager, not in version control)
- If the encryption key is lost, encrypted secrets cannot be recovered
- Changing the encryption key requires re-encrypting all secrets
### Environment Variables
Override configuration via environment variables:
```bash
export ATTUNE__SECURITY__ENCRYPTION_KEY="your-encryption-key"
```
## Usage Examples
### Storing Secrets (via API)
#### System-Level Secret
```bash
curl -X POST http://localhost:8080/api/v1/keys \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "system.api_endpoint",
"owner_type": "system",
"name": "api_endpoint",
"value": "https://api.example.com",
"encrypted": false
}'
```
#### Pack-Level Secret (Encrypted)
```bash
curl -X POST http://localhost:8080/api/v1/keys \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "mypack.api_key",
"owner_type": "pack",
"owner_pack": 1,
"name": "api_key",
"value": "sk_live_abc123def456",
"encrypted": true
}'
```
#### Action-Level Secret
```bash
curl -X POST http://localhost:8080/api/v1/keys \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"ref": "mypack.myaction.oauth_token",
"owner_type": "action",
"owner_action": 42,
"name": "oauth_token",
"value": "ya29.a0AfH6SMBx...",
"encrypted": true
}'
```
### Accessing Secrets in Actions
Secrets are automatically injected as environment variables during execution. The secret name is converted to uppercase and prefixed with `SECRET_`.
#### Python Action Example
```python
#!/usr/bin/env python3
import os
# Access secrets via environment variables
api_key = os.environ.get('SECRET_API_KEY')
db_password = os.environ.get('SECRET_DB_PASSWORD')
oauth_token = os.environ.get('SECRET_OAUTH_TOKEN')
if not api_key:
print("Error: SECRET_API_KEY not found")
exit(1)
# Use the secrets
print(f"Connecting to API with key: {api_key[:8]}...")
```
#### Shell Action Example
```bash
#!/bin/bash
# Access secrets
echo "API Key: ${SECRET_API_KEY:0:8}..."
echo "Database: ${SECRET_DB_HOST}"
# Use in commands
curl -H "Authorization: Bearer $SECRET_API_TOKEN" \
https://api.example.com/data
```
### Environment Variable Naming Rules
Secret names are transformed as follows:
- Prefix: `SECRET_`
- Convert to uppercase
- Replace hyphens with underscores
Examples:
- `api_key` → `SECRET_API_KEY`
- `db-password` → `SECRET_DB_PASSWORD`
- `oauth_token` → `SECRET_OAUTH_TOKEN`
## Security Best Practices
### 1. Encryption Key Management
- **Generate Strong Keys**: Use at least 32 random characters
- **Secure Storage**: Store in a secrets manager (AWS Secrets Manager, HashiCorp Vault, etc.)
- **Rotation**: Plan for key rotation (requires re-encrypting all secrets)
- **Backup**: Keep encrypted backup of the encryption key
### 2. Secret Storage
- **Always Encrypt Sensitive Data**: Use `encrypted: true` for passwords, tokens, API keys
- **Plaintext for Non-Sensitive**: Use `encrypted: false` for URLs, usernames, configuration
- **Least Privilege**: Use action-level secrets for the most sensitive data
### 3. Action Development
- **Never Log Secrets**: Avoid printing secret values in action output
- **Mask in Errors**: Don't include secrets in error messages
- **Clear After Use**: In long-running processes, clear secrets from memory when done
### 4. Access Control
- **RBAC**: Limit who can create/read secrets using Attune's RBAC system
- **Audit Logging**: Enable audit logging for secret access (future feature)
- **Regular Reviews**: Periodically review and rotate secrets
## Implementation Details
### Encryption Process
```rust
// 1. Derive encryption key from password
let key = SHA256(password);
// 2. Generate random nonce
let nonce = random_bytes(12);
// 3. Encrypt plaintext
let ciphertext = AES256GCM.encrypt(key, nonce, plaintext);
// 4. Format as "nonce:ciphertext" (base64-encoded)
let encrypted_value = format!("{}:{}",
base64(nonce),
base64(ciphertext)
);
```
### Decryption Process
```rust
// 1. Parse "nonce:ciphertext" format
let (nonce_b64, ciphertext_b64) = encrypted_value.split_once(':');
let nonce = base64_decode(nonce_b64);
let ciphertext = base64_decode(ciphertext_b64);
// 2. Validate encryption key hash (if present)
if key_hash != SHA256(encryption_key) {
return Error("Key mismatch");
}
// 3. Decrypt ciphertext
let plaintext = AES256GCM.decrypt(encryption_key, nonce, ciphertext);
```
### Secret Injection Flow
```
1. ActionExecutor prepares execution context
2. SecretManager fetches secrets for action
a. Query system-level secrets
b. Query pack-level secrets
c. Query action-level secrets
d. Merge with later overriding earlier
3. Decrypt encrypted secrets
4. Transform to environment variables
5. Inject into execution context
6. Action executes with secrets available
```
## Troubleshooting
### "No encryption key configured"
**Problem**: Worker service cannot decrypt secrets.
**Solution**: Set the encryption key in configuration:
```yaml
security:
encryption_key: "your-encryption-key-here"
```
### "Encryption key hash mismatch"
**Problem**: The encryption key used to decrypt doesn't match the key used to encrypt.
**Solution**:
- Verify you're using the correct encryption key
- Check if encryption key was recently changed
- May need to re-encrypt secrets with new key
### "Decryption failed"
**Problem**: Secret cannot be decrypted.
**Causes**:
- Wrong encryption key
- Corrupted encrypted value
- Invalid format
**Solution**:
- Verify encryption key is correct
- Check secret value format (should be "nonce:ciphertext")
- Try re-encrypting the secret
### Secrets Not Available in Action
**Problem**: Environment variables like `SECRET_API_KEY` are not set.
**Checklist**:
- Verify secret exists in database with correct owner type
- Check secret name matches expected format
- Ensure action's pack has access to the secret
- Check worker logs for "Failed to fetch secrets" warnings
## API Reference
### SecretManager Methods
#### `fetch_secrets_for_action(action: &Action) -> Result<HashMap<String, String>>`
Fetches all secrets relevant to an action (system + pack + action level).
#### `encrypt_value(plaintext: &str) -> Result<String>`
Encrypts a plaintext value using the configured encryption key.
#### `prepare_secret_env(secrets: &HashMap<String, String>) -> HashMap<String, String>`
Transforms secret names to environment variable format.
## Future Enhancements
### Planned Features
- [ ] Secret versioning and rollback
- [ ] Audit logging for secret access
- [ ] Integration with external secret managers (Vault, AWS Secrets Manager)
- [ ] Automatic secret rotation
- [ ] Secret expiration and TTL
- [ ] Multi-key encryption (key per pack/action)
- [ ] Secret templates and inheritance
### Under Consideration
- [ ] Dynamic secret generation
- [ ] Just-in-time secret provisioning
- [ ] Secret usage analytics
- [ ] Integration with certificate management
## References
- [AES-GCM Encryption](https://en.wikipedia.org/wiki/Galois/Counter_Mode)
- [NIST SP 800-38D](https://csrc.nist.gov/publications/detail/sp/800-38d/final) - Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM)
- [Key Management Best Practices](https://www.owasp.org/index.php/Key_Management_Cheat_Sheet)