attune/work-summary/2026-02-09-core-pack-jq-elimination.md

Core Pack: jq Dependency Elimination

Date: 2026-02-09
Objective: Remove all jq dependencies from the core pack to minimize external runtime requirements and ensure maximum portability.

Overview

The core pack previously relied on jq (a JSON command-line processor) for parsing JSON parameters in several action scripts. This created an unnecessary external dependency that could cause issues in minimal environments or containers without jq installed.

Changes Made

1. Converted API Wrapper Actions from bash+jq to Pure POSIX Shell

All four API wrapper actions have been converted from bash scripts using jq for JSON parsing to pure POSIX shell scripts using DOTENV parameter format:

get_pack_dependencies (bash+jq → POSIX shell)

• File: Renamed from get_pack_dependencies.py to get_pack_dependencies.sh
• YAML: Updated parameter_format: json → parameter_format: dotenv
• Entry Point: Already configured as get_pack_dependencies.sh
• Functionality: API wrapper for POST /api/v1/packs/dependencies

download_packs (bash+jq → POSIX shell)

• File: Renamed from download_packs.py to download_packs.sh
• YAML: Updated parameter_format: json → parameter_format: dotenv
• Entry Point: Already configured as download_packs.sh
• Functionality: API wrapper for POST /api/v1/packs/download

register_packs (bash+jq → POSIX shell)

• File: Renamed from register_packs.py to register_packs.sh
• YAML: Updated parameter_format: json → parameter_format: dotenv
• Entry Point: Already configured as register_packs.sh
• Functionality: API wrapper for POST /api/v1/packs/register-batch

build_pack_envs (bash+jq → POSIX shell)

• File: Renamed from build_pack_envs.py to build_pack_envs.sh
• YAML: Updated parameter_format: json → parameter_format: dotenv
• Entry Point: Already configured as build_pack_envs.sh
• Functionality: API wrapper for POST /api/v1/packs/build-envs

2. Implementation Approach

All converted scripts now follow the pattern established by core.echo:

• Shebang: #!/bin/sh (POSIX shell, not bash)
• Parameter Parsing: DOTENV format from stdin until EOF
• JSON Construction: Manual string construction with proper escaping
• HTTP Requests: Using curl with response written to temp files
• Response Parsing: Simple sed/case pattern matching for JSON field extraction
• Error Handling: Graceful error messages without external tools
• Cleanup: Trap handlers for temporary file cleanup

3. Key Techniques Used

DOTENV Parameter Parsing

while IFS= read -r line; do
    [ -z "$line" ] && continue

    key="${line%%=*}"
    value="${line#*=}"
    
    # Remove quotes
    case "$value" in
        \"*\") value="${value#\"}"; value="${value%\"}" ;;
        \'*\') value="${value#\'}"; value="${value%\'}" ;;
    esac
    
    case "$key" in
        param_name) param_name="$value" ;;
    esac
done

JSON Construction (without jq)

# Escape special characters for JSON
value_escaped=$(printf '%s' "$value" | sed 's/\\/\\\\/g; s/"/\\"/g')

# Build JSON body
request_body=$(cat <<EOF
{
  "field": "$value_escaped",
  "boolean": $bool_value
}
EOF
)

API Response Extraction (without jq)

# Extract .data field using sed pattern matching
case "$response_body" in
    *'"data":'*)
        data_content=$(printf '%s' "$response_body" | sed -n 's/.*"data":\s*\(.*\)}/\1/p')
        ;;
esac

Boolean Normalization

case "$verify_ssl" in
    true|True|TRUE|yes|Yes|YES|1) verify_ssl="true" ;;
    *) verify_ssl="false" ;;
esac

4. Files Modified

Action Scripts (renamed and rewritten):

• packs/core/actions/get_pack_dependencies.py → packs/core/actions/get_pack_dependencies.sh
• packs/core/actions/download_packs.py → packs/core/actions/download_packs.sh
• packs/core/actions/register_packs.py → packs/core/actions/register_packs.sh
• packs/core/actions/build_pack_envs.py → packs/core/actions/build_pack_envs.sh

YAML Metadata (updated parameter_format):

• packs/core/actions/get_pack_dependencies.yaml
• packs/core/actions/download_packs.yaml
• packs/core/actions/register_packs.yaml
• packs/core/actions/build_pack_envs.yaml

5. Previously Completed Actions

The following actions were already using pure POSIX shell without jq:

• ✅ echo.sh - Simple message output
• ✅ sleep.sh - Delay execution
• ✅ noop.sh - No-operation placeholder
• ✅ http_request.sh - HTTP client (already jq-free)

Verification

All Actions Now Use Shell Runtime

$ grep -H "runner_type:" packs/core/actions/*.yaml | sort -u
# All show: runner_type: shell

All Actions Use DOTENV Parameter Format

$ grep -H "parameter_format:" packs/core/actions/*.yaml
# All show: parameter_format: dotenv

No jq Command Usage

$ grep -E "^\s*[^#]*jq\s+" packs/core/actions/*.sh
# No results (only comments mention jq)

All Scripts Use POSIX Shell

$ head -n 1 packs/core/actions/*.sh
# All show: #!/bin/sh

All Scripts Are Executable

$ ls -l packs/core/actions/*.sh | awk '{print $1}'
# All show: -rwxrwxr-x

Benefits

1. Zero External Dependencies: Core pack now requires only POSIX shell and curl (universally available)
2. Improved Portability: Works in minimal containers (Alpine, scratch-based, distroless)
3. Faster Execution: No process spawning for jq, direct shell parsing
4. Reduced Attack Surface: Fewer binaries to audit/update
5. Consistency: All actions follow the same parameter parsing pattern
6. Maintainability: Single, clear pattern for all shell actions

Core Pack Runtime Requirements

Required:

• POSIX-compliant shell (/bin/sh)
• curl (for HTTP requests)
• Standard POSIX utilities: sed, mktemp, cat, printf, sleep

Not Required:

• ❌ jq - Eliminated
• ❌ yq - Never used
• ❌ Python - Not used in core pack
• ❌ Node.js - Not used in core pack
• ❌ bash-specific features - Scripts are POSIX-compliant

Testing Recommendations

1. Basic Functionality: Test all 8 core actions with various parameters
2. Parameter Parsing: Verify DOTENV format handling (quotes, special characters)
3. API Integration: Test API wrapper actions against running API service
4. Error Handling: Verify graceful failures with malformed input/API errors
5. Cross-Platform: Test on Alpine Linux (minimal environment)
6. Special Characters: Test with values containing quotes, backslashes, newlines

Future Considerations

• Consider adding integration tests specifically for DOTENV parameter parsing
• Document the DOTENV format specification for pack developers
• Consider adding parameter validation helpers to reduce code duplication
• Monitor for any edge cases in JSON construction/parsing

Conclusion

The core pack is now completely free of jq dependencies and relies only on standard POSIX utilities. This significantly improves portability and reduces the maintenance burden, aligning with the project goal of minimal external dependencies.

All actions follow a consistent, well-documented pattern that can serve as a reference for future pack development.