attune/work-summary/2026-02-09-core-pack-jq-elimination.md

# Core Pack: jq Dependency Elimination

**Date:** 2026-02-09  
**Objective:** Remove all `jq` dependencies from the core pack to minimize external runtime requirements and ensure maximum portability.

## Overview

The core pack previously relied on `jq` (a JSON command-line processor) for parsing JSON parameters in several action scripts. This created an unnecessary external dependency that could cause issues in minimal environments or containers without `jq` installed.

## Changes Made

### 1. Converted API Wrapper Actions from bash+jq to Pure POSIX Shell

All four API wrapper actions have been converted from bash scripts using `jq` for JSON parsing to pure POSIX shell scripts using DOTENV parameter format:

#### `get_pack_dependencies` (bash+jq → POSIX shell)
- **File:** Renamed from `get_pack_dependencies.py` to `get_pack_dependencies.sh`
- **YAML:** Updated `parameter_format: json` → `parameter_format: dotenv`
- **Entry Point:** Already configured as `get_pack_dependencies.sh`
- **Functionality:** API wrapper for POST `/api/v1/packs/dependencies`

#### `download_packs` (bash+jq → POSIX shell)
- **File:** Renamed from `download_packs.py` to `download_packs.sh`
- **YAML:** Updated `parameter_format: json` → `parameter_format: dotenv`
- **Entry Point:** Already configured as `download_packs.sh`
- **Functionality:** API wrapper for POST `/api/v1/packs/download`

#### `register_packs` (bash+jq → POSIX shell)
- **File:** Renamed from `register_packs.py` to `register_packs.sh`
- **YAML:** Updated `parameter_format: json` → `parameter_format: dotenv`
- **Entry Point:** Already configured as `register_packs.sh`
- **Functionality:** API wrapper for POST `/api/v1/packs/register-batch`

#### `build_pack_envs` (bash+jq → POSIX shell)
- **File:** Renamed from `build_pack_envs.py` to `build_pack_envs.sh`
- **YAML:** Updated `parameter_format: json` → `parameter_format: dotenv`
- **Entry Point:** Already configured as `build_pack_envs.sh`
- **Functionality:** API wrapper for POST `/api/v1/packs/build-envs`

### 2. Implementation Approach

All converted scripts now follow the pattern established by `core.echo`:

- **Shebang:** `#!/bin/sh` (POSIX shell, not bash)
- **Parameter Parsing:** DOTENV format from stdin until EOF
- **JSON Construction:** Manual string construction with proper escaping
- **HTTP Requests:** Using `curl` with response written to temp files
- **Response Parsing:** Simple sed/case pattern matching for JSON field extraction
- **Error Handling:** Graceful error messages without external tools
- **Cleanup:** Trap handlers for temporary file cleanup

### 3. Key Techniques Used

#### DOTENV Parameter Parsing
```sh
while IFS= read -r line; do
    [ -z "$line" ] && continue

    key="${line%%=*}"
    value="${line#*=}"
    
    # Remove quotes
    case "$value" in
        \"*\") value="${value#\"}"; value="${value%\"}" ;;
        \'*\') value="${value#\'}"; value="${value%\'}" ;;
    esac
    
    case "$key" in
        param_name) param_name="$value" ;;
    esac
done
```

#### JSON Construction (without jq)
```sh
# Escape special characters for JSON
value_escaped=$(printf '%s' "$value" | sed 's/\\/\\\\/g; s/"/\\"/g')

# Build JSON body
request_body=$(cat <<EOF
{
  "field": "$value_escaped",
  "boolean": $bool_value
}
EOF
)
```

#### API Response Extraction (without jq)
```sh
# Extract .data field using sed pattern matching
case "$response_body" in
    *'"data":'*)
        data_content=$(printf '%s' "$response_body" | sed -n 's/.*"data":\s*\(.*\)}/\1/p')
        ;;
esac
```

#### Boolean Normalization
```sh
case "$verify_ssl" in
    true|True|TRUE|yes|Yes|YES|1) verify_ssl="true" ;;
    *) verify_ssl="false" ;;
esac
```

### 4. Files Modified

**Action Scripts (renamed and rewritten):**
- `packs/core/actions/get_pack_dependencies.py` → `packs/core/actions/get_pack_dependencies.sh`
- `packs/core/actions/download_packs.py` → `packs/core/actions/download_packs.sh`
- `packs/core/actions/register_packs.py` → `packs/core/actions/register_packs.sh`
- `packs/core/actions/build_pack_envs.py` → `packs/core/actions/build_pack_envs.sh`

**YAML Metadata (updated parameter_format):**
- `packs/core/actions/get_pack_dependencies.yaml`
- `packs/core/actions/download_packs.yaml`
- `packs/core/actions/register_packs.yaml`
- `packs/core/actions/build_pack_envs.yaml`

### 5. Previously Completed Actions

The following actions were already using pure POSIX shell without `jq`:
- ✅ `echo.sh` - Simple message output
- ✅ `sleep.sh` - Delay execution
- ✅ `noop.sh` - No-operation placeholder
- ✅ `http_request.sh` - HTTP client (already jq-free)

## Verification

### All Actions Now Use Shell Runtime
```bash
$ grep -H "runner_type:" packs/core/actions/*.yaml | sort -u
# All show: runner_type: shell
```

### All Actions Use DOTENV Parameter Format
```bash
$ grep -H "parameter_format:" packs/core/actions/*.yaml
# All show: parameter_format: dotenv
```

### No jq Command Usage
```bash
$ grep -E "^\s*[^#]*jq\s+" packs/core/actions/*.sh
# No results (only comments mention jq)
```

### All Scripts Use POSIX Shell
```bash
$ head -n 1 packs/core/actions/*.sh
# All show: #!/bin/sh
```

### All Scripts Are Executable
```bash
$ ls -l packs/core/actions/*.sh | awk '{print $1}'
# All show: -rwxrwxr-x
```

## Benefits

1. **Zero External Dependencies:** Core pack now requires only POSIX shell and `curl` (universally available)
2. **Improved Portability:** Works in minimal containers (Alpine, scratch-based, distroless)
3. **Faster Execution:** No process spawning for `jq`, direct shell parsing
4. **Reduced Attack Surface:** Fewer binaries to audit/update
5. **Consistency:** All actions follow the same parameter parsing pattern
6. **Maintainability:** Single, clear pattern for all shell actions

## Core Pack Runtime Requirements

**Required:**
- POSIX-compliant shell (`/bin/sh`)
- `curl` (for HTTP requests)
- Standard POSIX utilities: `sed`, `mktemp`, `cat`, `printf`, `sleep`

**Not Required:**
- ❌ `jq` - Eliminated
- ❌ `yq` - Never used
- ❌ Python - Not used in core pack
- ❌ Node.js - Not used in core pack
- ❌ bash-specific features - Scripts are POSIX-compliant

## Testing Recommendations

1. **Basic Functionality:** Test all 8 core actions with various parameters
2. **Parameter Parsing:** Verify DOTENV format handling (quotes, special characters)
3. **API Integration:** Test API wrapper actions against running API service
4. **Error Handling:** Verify graceful failures with malformed input/API errors
5. **Cross-Platform:** Test on Alpine Linux (minimal environment)
6. **Special Characters:** Test with values containing quotes, backslashes, newlines

## Future Considerations

- Consider adding integration tests specifically for DOTENV parameter parsing
- Document the DOTENV format specification for pack developers
- Consider adding parameter validation helpers to reduce code duplication
- Monitor for any edge cases in JSON construction/parsing

## Conclusion

The core pack is now completely free of `jq` dependencies and relies only on standard POSIX utilities. This significantly improves portability and reduces the maintenance burden, aligning with the project goal of minimal external dependencies.

All actions follow a consistent, well-documented pattern that can serve as a reference for future pack development.