[wip] cli capability parity
Some checks failed
CI / Rustfmt (push) Successful in 23s
CI / Cargo Audit & Deny (push) Successful in 30s
CI / Web Blocking Checks (push) Successful in 48s
CI / Security Blocking Checks (push) Successful in 8s
CI / Clippy (push) Failing after 1m55s
CI / Web Advisory Checks (push) Successful in 35s
CI / Security Advisory Checks (push) Successful in 37s
CI / Tests (push) Successful in 8m5s
Some checks failed
CI / Rustfmt (push) Successful in 23s
CI / Cargo Audit & Deny (push) Successful in 30s
CI / Web Blocking Checks (push) Successful in 48s
CI / Security Blocking Checks (push) Successful in 8s
CI / Clippy (push) Failing after 1m55s
CI / Web Advisory Checks (push) Successful in 35s
CI / Security Advisory Checks (push) Successful in 37s
CI / Tests (push) Successful in 8m5s
This commit is contained in:
@@ -186,7 +186,7 @@ pub enum PreparedParameters {
|
||||
**Security Features**:
|
||||
- Temporary files created with restrictive permissions (0400 on Unix)
|
||||
- Automatic cleanup of temporary files
|
||||
- Delimiter separation (`---ATTUNE_PARAMS_END---`) for parameters and secrets
|
||||
- Single-document delivery (secrets merged into parameters)
|
||||
|
||||
### 4. Runtime Integration
|
||||
|
||||
@@ -310,12 +310,9 @@ import json
|
||||
import os
|
||||
|
||||
def read_stdin_params():
|
||||
"""Read parameters from stdin."""
|
||||
content = sys.stdin.read()
|
||||
parts = content.split('---ATTUNE_PARAMS_END---')
|
||||
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
|
||||
secrets = json.loads(parts[1].strip()) if len(parts) > 1 and parts[1].strip() else {}
|
||||
return {**params, **secrets}
|
||||
"""Read parameters from stdin. Secrets are already merged into parameters."""
|
||||
content = sys.stdin.read().strip()
|
||||
return json.loads(content) if content else {}
|
||||
|
||||
def main():
|
||||
# Read parameters (secure)
|
||||
@@ -491,8 +488,8 @@ parameter_format: json
|
||||
Write action to read from stdin:
|
||||
```python
|
||||
import sys, json
|
||||
content = sys.stdin.read()
|
||||
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
|
||||
content = sys.stdin.read().strip()
|
||||
params = json.loads(content) if content else {}
|
||||
```
|
||||
|
||||
### For Execution Context
|
||||
|
||||
@@ -139,7 +139,7 @@ New utility module providing:
|
||||
- Temporary files created with restrictive permissions (owner read-only)
|
||||
- Automatic cleanup of temporary files
|
||||
- Proper escaping of special characters in dotenv format
|
||||
- Delimiter (`---ATTUNE_PARAMS_END---`) separates parameters from secrets in stdin
|
||||
- Single-document delivery (secrets merged into parameters)
|
||||
|
||||
**Test Coverage**: Comprehensive unit tests for all formatting and delivery methods
|
||||
|
||||
@@ -252,11 +252,9 @@ import sys
|
||||
import json
|
||||
|
||||
def read_stdin_params():
|
||||
content = sys.stdin.read()
|
||||
parts = content.split('---ATTUNE_PARAMS_END---')
|
||||
params = json.loads(parts[0].strip()) if parts[0].strip() else {}
|
||||
secrets = json.loads(parts[1].strip()) if len(parts) > 1 and parts[1].strip() else {}
|
||||
return {**params, **secrets}
|
||||
"""Read parameters from stdin. Secrets are already merged into parameters."""
|
||||
content = sys.stdin.read().strip()
|
||||
return json.loads(content) if content else {}
|
||||
|
||||
params = read_stdin_params()
|
||||
api_key = params.get('api_key') # Secure - not in process list!
|
||||
|
||||
@@ -42,7 +42,7 @@ All four API wrapper actions have been converted from bash scripts using `jq` fo
|
||||
All converted scripts now follow the pattern established by `core.echo`:
|
||||
|
||||
- **Shebang:** `#!/bin/sh` (POSIX shell, not bash)
|
||||
- **Parameter Parsing:** DOTENV format from stdin with delimiter `---ATTUNE_PARAMS_END---`
|
||||
- **Parameter Parsing:** DOTENV format from stdin until EOF
|
||||
- **JSON Construction:** Manual string construction with proper escaping
|
||||
- **HTTP Requests:** Using `curl` with response written to temp files
|
||||
- **Response Parsing:** Simple sed/case pattern matching for JSON field extraction
|
||||
@@ -54,10 +54,8 @@ All converted scripts now follow the pattern established by `core.echo`:
|
||||
#### DOTENV Parameter Parsing
|
||||
```sh
|
||||
while IFS= read -r line; do
|
||||
case "$line" in
|
||||
*"---ATTUNE_PARAMS_END---"*) break ;;
|
||||
esac
|
||||
|
||||
[ -z "$line" ] && continue
|
||||
|
||||
key="${line%%=*}"
|
||||
value="${line#*=}"
|
||||
|
||||
|
||||
@@ -174,7 +174,6 @@ attune action execute core.http_request \
|
||||
headers.Content-Type='application/json'
|
||||
query_params.page='1'
|
||||
url='https://example.com'
|
||||
---ATTUNE_PARAMS_END---
|
||||
```
|
||||
|
||||
3. Verify execution succeeds with correct HTTP request/response
|
||||
|
||||
@@ -105,12 +105,9 @@ error: "/opt/attune/packs/core/actions/echo.sh: line 12: jq: command not found"
|
||||
|
||||
**Parsing Logic**:
|
||||
```sh
|
||||
# Read DOTENV-formatted parameters from stdin until delimiter
|
||||
# Read DOTENV-formatted parameters from stdin until EOF
|
||||
while IFS= read -r line; do
|
||||
case "$line" in
|
||||
*"---ATTUNE_PARAMS_END---"*)
|
||||
break
|
||||
;;
|
||||
message=*)
|
||||
message="${line#message=}"
|
||||
# Remove quotes if present
|
||||
@@ -176,9 +173,7 @@ Manual testing verified:
|
||||
|
||||
Example test:
|
||||
```sh
|
||||
echo "message='Hello World!'" > test.txt
|
||||
echo "---ATTUNE_PARAMS_END---" >> test.txt
|
||||
cat test.txt | sh packs/core/actions/echo.sh
|
||||
echo "message='Hello World!'" | sh packs/core/actions/echo.sh
|
||||
# Output: Hello World!
|
||||
```
|
||||
|
||||
|
||||
80
work-summary/2026-02-remove-params-end-delimiter.md
Normal file
80
work-summary/2026-02-remove-params-end-delimiter.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# Remove `---ATTUNE_PARAMS_END---` Delimiter Antipattern
|
||||
|
||||
**Date:** 2026-02-10
|
||||
|
||||
## Summary
|
||||
|
||||
Removed all instances of the `---ATTUNE_PARAMS_END---` stdin delimiter from the entire project — source code, shell scripts, and documentation. This was an antipattern from the old two-phase stdin protocol where parameters and secrets were delivered as separate documents separated by this delimiter. The current protocol merges secrets into parameters as a single JSON document delivered via one `readline()`, making the delimiter unnecessary.
|
||||
|
||||
## Background
|
||||
|
||||
The original stdin protocol wrote parameters and secrets in two phases:
|
||||
1. Parameters JSON + `\n---ATTUNE_PARAMS_END---\n`
|
||||
2. Secrets JSON + `\n`
|
||||
|
||||
This was already fixed in `process_executor.rs` and `shell.rs` (which write a single merged document followed by `\n`), but `native.rs` still had the old protocol, and all shell scripts and documentation still referenced it.
|
||||
|
||||
## Changes Made
|
||||
|
||||
### Source Code (1 file)
|
||||
|
||||
**`crates/worker/src/runtime/native.rs`**:
|
||||
- Removed the `---ATTUNE_PARAMS_END---` delimiter write from `execute_binary()`
|
||||
- Removed the separate secrets-writing block (matching the fix already applied to `shell.rs` and `process_executor.rs`)
|
||||
- Added secrets-into-parameters merge in `execute()` before `prepare_parameters()` is called
|
||||
- Now passes `&std::collections::HashMap::new()` for secrets to `execute_binary()`
|
||||
- Stdin protocol is now: `{merged_params}\n` then close — consistent across all runtimes
|
||||
|
||||
### Shell Scripts (9 files)
|
||||
|
||||
Updated all pack action scripts to read stdin until EOF instead of looking for the delimiter:
|
||||
|
||||
- `packs/core/actions/echo.sh`
|
||||
- `packs/core/actions/sleep.sh`
|
||||
- `packs/core/actions/noop.sh`
|
||||
- `packs/core/actions/http_request.sh`
|
||||
- `packs/core/actions/build_pack_envs.sh`
|
||||
- `packs/core/actions/download_packs.sh`
|
||||
- `packs/core/actions/get_pack_dependencies.sh`
|
||||
- `packs/core/actions/register_packs.sh`
|
||||
- `packs/examples/actions/list_example.sh`
|
||||
|
||||
In each script, removed the `*"---ATTUNE_PARAMS_END---"*) break ;;` case pattern. The `while IFS= read -r line` loop now terminates naturally at EOF when stdin is closed.
|
||||
|
||||
### Documentation (9 files)
|
||||
|
||||
- `docs/QUICKREF-dotenv-shell-actions.md` — Updated template, format spec, and parsing examples
|
||||
- `docs/action-development-guide.md` — Updated stdin protocol description, all Python/Node.js/Shell examples, troubleshooting section
|
||||
- `docs/actions/QUICKREF-parameter-delivery.md` — Updated copy-paste templates and design change section
|
||||
- `docs/actions/README.md` — Updated quick start Python example
|
||||
- `docs/actions/parameter-delivery.md` — Updated protocol description, stdin content example, all code examples
|
||||
- `docs/packs/pack-structure.md` — Updated Python stdin example
|
||||
- `docs/parameters/dotenv-parameter-format.md` — Updated parsing examples, secret handling docs, troubleshooting
|
||||
|
||||
### Work Summaries (5 files)
|
||||
|
||||
- `work-summary/2025-02-05-FINAL-secure-parameters.md`
|
||||
- `work-summary/2025-02-05-secure-parameter-delivery.md`
|
||||
- `work-summary/2026-02-09-core-pack-jq-elimination.md`
|
||||
- `work-summary/2026-02-09-dotenv-parameter-flattening.md`
|
||||
- `work-summary/2026-02-action-execution-fixes.md`
|
||||
- `work-summary/changelogs/CHANGELOG.md`
|
||||
|
||||
## Verification
|
||||
|
||||
- `cargo check --all-targets --workspace` — zero errors, zero warnings
|
||||
- `cargo test -p attune-worker` — all 15 tests pass (including 7 security tests)
|
||||
- Manual shell script testing — `echo.sh`, `sleep.sh`, `noop.sh` all work correctly with EOF-based reading
|
||||
- `grep -r ATTUNE_PARAMS_END` — zero matches remaining in entire project
|
||||
|
||||
## Current Stdin Protocol (All Runtimes)
|
||||
|
||||
```
|
||||
{merged_parameters_json}\n
|
||||
<stdin closed>
|
||||
```
|
||||
|
||||
- Secrets are merged into the parameters map by the caller before formatting
|
||||
- Actions receive a single document via `readline()` or `read()`
|
||||
- Shell scripts using DOTENV format read `key='value'` lines until EOF
|
||||
- No delimiter, no two-phase protocol, no separate secrets document
|
||||
@@ -99,8 +99,8 @@ parameter_format: dotenv
|
||||
**Action scripts should read from stdin** (default):
|
||||
```python
|
||||
import sys, json
|
||||
content = sys.stdin.read()
|
||||
params = json.loads(content.split('---ATTUNE_PARAMS_END---')[0])
|
||||
content = sys.stdin.read().strip()
|
||||
params = json.loads(content) if content else {}
|
||||
```
|
||||
|
||||
**For env delivery** (explicit opt-in):
|
||||
|
||||
Reference in New Issue
Block a user