[wip] cli capability parity
Some checks failed
CI / Rustfmt (push) Successful in 23s
CI / Cargo Audit & Deny (push) Successful in 30s
CI / Web Blocking Checks (push) Successful in 48s
CI / Security Blocking Checks (push) Successful in 8s
CI / Clippy (push) Failing after 1m55s
CI / Web Advisory Checks (push) Successful in 35s
CI / Security Advisory Checks (push) Successful in 37s
CI / Tests (push) Successful in 8m5s

This commit is contained in:
2026-03-06 16:58:50 -06:00
parent 48b6ca6bd7
commit 87d830f952
94 changed files with 3694 additions and 734 deletions

View File

@@ -186,7 +186,7 @@ pub enum PreparedParameters {
**Security Features**:
- Temporary files created with restrictive permissions (0400 on Unix)
- Automatic cleanup of temporary files
- Delimiter separation (`---ATTUNE_PARAMS_END---`) for parameters and secrets
- Single-document delivery (secrets merged into parameters)
### 4. Runtime Integration
@@ -310,12 +310,9 @@ import json
import os
def read_stdin_params():
"""Read parameters from stdin."""
content = sys.stdin.read()
parts = content.split('---ATTUNE_PARAMS_END---')
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
secrets = json.loads(parts[1].strip()) if len(parts) > 1 and parts[1].strip() else {}
return {**params, **secrets}
"""Read parameters from stdin. Secrets are already merged into parameters."""
content = sys.stdin.read().strip()
return json.loads(content) if content else {}
def main():
# Read parameters (secure)
@@ -491,8 +488,8 @@ parameter_format: json
Write action to read from stdin:
```python
import sys, json
content = sys.stdin.read()
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
content = sys.stdin.read().strip()
params = json.loads(content) if content else {}
```
### For Execution Context

View File

@@ -139,7 +139,7 @@ New utility module providing:
- Temporary files created with restrictive permissions (owner read-only)
- Automatic cleanup of temporary files
- Proper escaping of special characters in dotenv format
- Delimiter (`---ATTUNE_PARAMS_END---`) separates parameters from secrets in stdin
- Single-document delivery (secrets merged into parameters)
**Test Coverage**: Comprehensive unit tests for all formatting and delivery methods
@@ -252,11 +252,9 @@ import sys
import json
def read_stdin_params():
content = sys.stdin.read()
parts = content.split('---ATTUNE_PARAMS_END---')
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
secrets = json.loads(parts[1].strip()) if len(parts) > 1 and parts[1].strip() else {}
return {**params, **secrets}
"""Read parameters from stdin. Secrets are already merged into parameters."""
content = sys.stdin.read().strip()
return json.loads(content) if content else {}
params = read_stdin_params()
api_key = params.get('api_key') # Secure - not in process list!

View File

@@ -42,7 +42,7 @@ All four API wrapper actions have been converted from bash scripts using `jq` fo
All converted scripts now follow the pattern established by `core.echo`:
- **Shebang:** `#!/bin/sh` (POSIX shell, not bash)
- **Parameter Parsing:** DOTENV format from stdin with delimiter `---ATTUNE_PARAMS_END---`
- **Parameter Parsing:** DOTENV format from stdin until EOF
- **JSON Construction:** Manual string construction with proper escaping
- **HTTP Requests:** Using `curl` with response written to temp files
- **Response Parsing:** Simple sed/case pattern matching for JSON field extraction
@@ -54,10 +54,8 @@ All converted scripts now follow the pattern established by `core.echo`:
#### DOTENV Parameter Parsing
```sh
while IFS= read -r line; do
case "$line" in
*"---ATTUNE_PARAMS_END---"*) break ;;
esac
[ -z "$line" ] && continue
key="${line%%=*}"
value="${line#*=}"

View File

@@ -174,7 +174,6 @@ attune action execute core.http_request \
headers.Content-Type='application/json'
query_params.page='1'
url='https://example.com'
---ATTUNE_PARAMS_END---
```
3. Verify execution succeeds with correct HTTP request/response

View File

@@ -105,12 +105,9 @@ error: "/opt/attune/packs/core/actions/echo.sh: line 12: jq: command not found"
**Parsing Logic**:
```sh
# Read DOTENV-formatted parameters from stdin until delimiter
# Read DOTENV-formatted parameters from stdin until EOF
while IFS= read -r line; do
case "$line" in
*"---ATTUNE_PARAMS_END---"*)
break
;;
message=*)
message="${line#message=}"
# Remove quotes if present
@@ -176,9 +173,7 @@ Manual testing verified:
Example test:
```sh
echo "message='Hello World!'" > test.txt
echo "---ATTUNE_PARAMS_END---" >> test.txt
cat test.txt | sh packs/core/actions/echo.sh
echo "message='Hello World!'" | sh packs/core/actions/echo.sh
# Output: Hello World!
```

View File

@@ -0,0 +1,80 @@
# Remove `---ATTUNE_PARAMS_END---` Delimiter Antipattern
**Date:** 2026-02-10
## Summary
Removed all instances of the `---ATTUNE_PARAMS_END---` stdin delimiter from the entire project — source code, shell scripts, and documentation. This was an antipattern from the old two-phase stdin protocol where parameters and secrets were delivered as separate documents separated by this delimiter. The current protocol merges secrets into parameters as a single JSON document delivered via one `readline()`, making the delimiter unnecessary.
## Background
The original stdin protocol wrote parameters and secrets in two phases:
1. Parameters JSON + `\n---ATTUNE_PARAMS_END---\n`
2. Secrets JSON + `\n`
This was already fixed in `process_executor.rs` and `shell.rs` (which write a single merged document followed by `\n`), but `native.rs` still had the old protocol, and all shell scripts and documentation still referenced it.
## Changes Made
### Source Code (1 file)
**`crates/worker/src/runtime/native.rs`**:
- Removed the `---ATTUNE_PARAMS_END---` delimiter write from `execute_binary()`
- Removed the separate secrets-writing block (matching the fix already applied to `shell.rs` and `process_executor.rs`)
- Added secrets-into-parameters merge in `execute()` before `prepare_parameters()` is called
- Now passes `&std::collections::HashMap::new()` for secrets to `execute_binary()`
- Stdin protocol is now: `{merged_params}\n` then close — consistent across all runtimes
### Shell Scripts (9 files)
Updated all pack action scripts to read stdin until EOF instead of looking for the delimiter:
- `packs/core/actions/echo.sh`
- `packs/core/actions/sleep.sh`
- `packs/core/actions/noop.sh`
- `packs/core/actions/http_request.sh`
- `packs/core/actions/build_pack_envs.sh`
- `packs/core/actions/download_packs.sh`
- `packs/core/actions/get_pack_dependencies.sh`
- `packs/core/actions/register_packs.sh`
- `packs/examples/actions/list_example.sh`
In each script, removed the `*"---ATTUNE_PARAMS_END---"*) break ;;` case pattern. The `while IFS= read -r line` loop now terminates naturally at EOF when stdin is closed.
### Documentation (9 files)
- `docs/QUICKREF-dotenv-shell-actions.md` — Updated template, format spec, and parsing examples
- `docs/action-development-guide.md` — Updated stdin protocol description, all Python/Node.js/Shell examples, troubleshooting section
- `docs/actions/QUICKREF-parameter-delivery.md` — Updated copy-paste templates and design change section
- `docs/actions/README.md` — Updated quick start Python example
- `docs/actions/parameter-delivery.md` — Updated protocol description, stdin content example, all code examples
- `docs/packs/pack-structure.md` — Updated Python stdin example
- `docs/parameters/dotenv-parameter-format.md` — Updated parsing examples, secret handling docs, troubleshooting
### Work Summaries (5 files)
- `work-summary/2025-02-05-FINAL-secure-parameters.md`
- `work-summary/2025-02-05-secure-parameter-delivery.md`
- `work-summary/2026-02-09-core-pack-jq-elimination.md`
- `work-summary/2026-02-09-dotenv-parameter-flattening.md`
- `work-summary/2026-02-action-execution-fixes.md`
- `work-summary/changelogs/CHANGELOG.md`
## Verification
- `cargo check --all-targets --workspace` — zero errors, zero warnings
- `cargo test -p attune-worker` — all 15 tests pass (including 7 security tests)
- Manual shell script testing — `echo.sh`, `sleep.sh`, `noop.sh` all work correctly with EOF-based reading
- `grep -r ATTUNE_PARAMS_END` — zero matches remaining in entire project
## Current Stdin Protocol (All Runtimes)
```
{merged_parameters_json}\n
<stdin closed>
```
- Secrets are merged into the parameters map by the caller before formatting
- Actions receive a single document via `readline()` or `read()`
- Shell scripts using DOTENV format read `key='value'` lines until EOF
- No delimiter, no two-phase protocol, no separate secrets document

View File

@@ -99,8 +99,8 @@ parameter_format: dotenv
**Action scripts should read from stdin** (default):
```python
import sys, json
content = sys.stdin.read()
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
content = sys.stdin.read().strip()
params = json.loads(content) if content else {}
```
**For env delivery** (explicit opt-in):