27 KiB
27 KiB
Pack Registry and Installation Specification
Last Updated: 2024-01-20
Status: Specification (Pre-Implementation)
Overview
This document specifies the pack registry and installation system for Attune, enabling community-driven pack distribution. The system allows packs to be:
- Published to independent registries (no central authority required)
- Installed from git repositories, HTTP/HTTPS URLs, or local sources
- Discovered through configurable registry indices
- Validated and tested during installation
Design Goals
- Decentralized: No single point of failure; anyone can host a registry
- CI/CD Friendly: Integrate with existing build and artifact storage systems
- Flexible Sources: Support multiple installation sources (git, HTTP, local)
- Priority-Based Discovery: Search multiple registries in configured order
- Secure: Validate checksums and signatures (future)
- Automated: Install dependencies, run tests, register components automatically
Pack Index File Format
Index Structure
Each registry hosts an index file (typically index.json) that catalogs available packs.
Format: JSON
Location: Configurable URL (HTTPS recommended)
Filename Convention: index.json or registry.json
Index Schema
{
"registry_name": "Attune Community Registry",
"registry_url": "https://registry.attune.io",
"version": "1.0",
"last_updated": "2024-01-20T12:00:00Z",
"packs": [
{
"ref": "slack",
"label": "Slack Integration",
"description": "Send messages, upload files, and monitor Slack channels",
"version": "2.1.0",
"author": "Attune Team",
"email": "team@attune.io",
"homepage": "https://github.com/attune-io/pack-slack",
"repository": "https://github.com/attune-io/pack-slack",
"license": "Apache-2.0",
"keywords": ["slack", "messaging", "notifications"],
"runtime_deps": ["python3"],
"
"install_sources": [
{
"type": "git",
"url": "https://github.com/attune-io/pack-slack.git",
"ref": "v2.1.0",
"checksum": "sha256:abc123..."
},
{
"type": "archive",
"url": "https://github.com/attune-io/pack-slack/archive/refs/tags/v2.1.0.zip",
"checksum": "sha256:def456..."
}
],
"contents": {
"actions": [
{
"name": "send_message",
"description": "Send a message to a Slack channel"
},
{
"name": "upload_file",
"description": "Upload a file to Slack"
}
],
"sensors": [
{
"name": "message_sensor",
"description": "Monitor Slack messages"
}
],
"triggers": [
{
"name": "message_received",
"description": "Fires when a message is received"
}
],
"rules": [],
"workflows": []
},
"dependencies": {
"attune_version": ">=0.1.0",
"python_version": ">=3.9",
"packs": []
},
"meta": {
"downloads": 1543,
"stars": 87,
"tested_attune_versions": ["0.1.0", "0.2.0"]
}
}
]
}
Field Definitions
Registry Metadata
| Field | Type | Required | Description |
|---|---|---|---|
registry_name |
string | Yes | Human-readable registry name |
registry_url |
string | Yes | Registry homepage URL |
version |
string | Yes | Index format version (semantic versioning) |
last_updated |
string | Yes | ISO 8601 timestamp of last update |
packs |
array | Yes | Array of pack entries |
Pack Entry
| Field | Type | Required | Description |
|---|---|---|---|
ref |
string | Yes | Unique pack identifier (matches pack.yaml) |
label |
string | Yes | Human-readable pack name |
description |
string | Yes | Brief pack description |
version |
string | Yes | Semantic version (latest available) |
author |
string | Yes | Pack author/maintainer name |
email |
string | No | Contact email |
homepage |
string | No | Pack homepage URL |
repository |
string | No | Source repository URL |
license |
string | Yes | SPDX license identifier |
keywords |
array[string] | No | Searchable keywords/tags |
runtime_deps |
array[string] | Yes | Required runtimes (python3, nodejs, shell) |
install_sources |
array[object] | Yes | Available installation sources (see below) |
contents |
object | Yes | Pack components summary |
dependencies |
object | No | Pack dependencies |
meta |
object | No | Additional metadata |
Install Source
| Field | Type | Required | Description |
|---|---|---|---|
type |
string | Yes | Source type: "git" or "archive" |
url |
string | Yes | Source URL |
ref |
string | No | Git ref (tag, branch, commit) for git type |
checksum |
string | Yes | Format: "algorithm:hash" (e.g., "sha256:abc...") |
Contents Object
| Field | Type | Description |
|---|---|---|
actions |
array[object] | List of actions with name and description |
sensors |
array[object] | List of sensors with name and description |
triggers |
array[object] | List of triggers with name and description |
rules |
array[object] | List of bundled rules |
workflows |
array[object] | List of bundled workflows |
Dependencies Object
| Field | Type | Description |
|---|---|---|
attune_version |
string | Semver requirement (e.g., ">=0.1.0", "^1.0.0") |
python_version |
string | Python version requirement |
nodejs_version |
string | Node.js version requirement |
packs |
array[string] | Pack dependencies (format: "ref@version") |
Pack Sources
Packs can be installed from multiple source types:
1. Git Repository
Install directly from a git repository:
attune pack install https://github.com/example/pack-slack.git
attune pack install https://github.com/example/pack-slack.git --ref v2.1.0
attune pack install https://github.com/example/pack-slack.git --ref main
attune pack install git@github.com:example/pack-slack.git --ref v2.1.0
Requirements:
- Repository must contain valid pack structure at root or in
pack/subdirectory pack.yamlmust be present- Git client must be installed on system
2. Archive URL
Install from a zip or tar.gz archive:
attune pack install https://example.com/packs/slack-2.1.0.zip
attune pack install https://example.com/packs/slack-2.1.0.tar.gz
Requirements:
- Archive must contain pack directory structure
- Archive root or single top-level directory must contain
pack.yaml - Supported formats:
.zip,.tar.gz,.tgz
3. Local Directory
Install from a local filesystem path:
attune pack install /path/to/pack-slack
attune pack install ./packs/my-pack
Requirements:
- Directory must contain valid pack structure
pack.yamlmust be present- Used for development and testing
4. Local Archive
Upload and install from a local archive file:
attune pack install /path/to/pack-slack-2.1.0.zip
attune pack install ./my-pack.tar.gz
Requirements:
- Archive must contain valid pack structure
- Archive is uploaded to Attune API before installation
- Used for air-gapped or offline installations
5. Registry Reference
Install by pack reference, searching configured registries:
attune pack install slack
attune pack install slack@2.1.0
attune pack install slack@latest
Requirements:
- At least one registry must be configured
- Pack reference must exist in one of the registries
- Registries searched in configured priority order
Configuration
Registry Configuration
Add registry URLs to service configuration files:
YAML Configuration (config.yaml):
pack_registry:
enabled: true
indices:
- url: https://registry.attune.io/index.json
priority: 1
enabled: true
name: "Official Attune Registry"
- url: https://company-internal.example.com/attune-registry.json
priority: 2
enabled: true
name: "Company Internal Registry"
headers:
Authorization: "Bearer ${REGISTRY_TOKEN}"
- url: file:///opt/attune/local-registry.json
priority: 3
enabled: true
name: "Local Filesystem Registry"
# Cache settings
cache_ttl: 3600 # Cache index for 1 hour
cache_enabled: true
# Download settings
timeout: 120
verify_checksums: true
allow_http: false # Only allow HTTPS
Environment Variables:
# Enable/disable registry
export ATTUNE__PACK_REGISTRY__ENABLED=true
# Set registry URLs (comma-separated, in priority order)
export ATTUNE__PACK_REGISTRY__INDICES="https://registry.attune.io/index.json,https://internal.example.com/registry.json"
# Cache settings
export ATTUNE__PACK_REGISTRY__CACHE_TTL=3600
export ATTUNE__PACK_REGISTRY__VERIFY_CHECKSUMS=true
Priority-Based Search
Registries are searched in priority order (lowest priority number first):
- Priority 1: Official Attune Registry (public packs)
- Priority 2: Company Internal Registry (private packs)
- Priority 3: Local Filesystem Registry (development packs)
When installing by reference (e.g., attune pack install slack):
- Search priority 1 registry first
- If not found, search priority 2
- If not found, search priority 3
- If not found in any registry, return error
Use Cases:
- Override public packs: Company registry can provide custom version of "slack" pack
- Private packs: Internal registry can host proprietary packs
- Development: Local registry can provide development versions
Registry Headers
For authenticated registries, configure custom HTTP headers:
pack_registry:
indices:
- url: https://private-registry.example.com/index.json
headers:
Authorization: "Bearer ${PRIVATE_REGISTRY_TOKEN}"
X-Custom-Header: "value"
CLI Commands
Install Pack
# From registry (by reference)
attune pack install <pack-ref>[@version]
# From git repository
attune pack install <git-url> [--ref <branch|tag|commit>]
# From archive URL
attune pack install <https-url>
# From local directory
attune pack install <local-path>
# From local archive
attune pack install <local-archive-path>
# Options
--force # Force reinstall if already exists
--skip-tests # Skip running pack tests
--skip-deps # Skip installing dependencies
--registry <name> # Use specific registry (skip priority search)
--no-registry # Don't search registries (direct install only)
Examples
# Install latest version from registry
attune pack install slack
# Install specific version from registry
attune pack install slack@2.1.0
# Install from git repository (latest tag)
attune pack install https://github.com/example/pack-slack.git
# Install from git repository (specific tag)
attune pack install https://github.com/example/pack-slack.git --ref v2.1.0
# Install from git repository (branch)
attune pack install https://github.com/example/pack-slack.git --ref main
# Install from archive URL
attune pack install https://example.com/packs/slack-2.1.0.zip
# Install from local directory (development)
attune pack install ./packs/my-pack
# Install from local archive
attune pack install ./slack-2.1.0.zip
# Force reinstall
attune pack install slack --force
# Skip tests (faster, but not recommended)
attune pack install slack --skip-tests
Generate Index Entry
For pack maintainers, generate an index entry from a pack:
attune pack index-entry \
--pack-dir <path-to-pack> \
--version <version> \
--git-url <git-repo-url> \
--git-ref <tag-or-branch> \
--archive-url <archive-url>
# Output to stdout (JSON)
attune pack index-entry --pack-dir ./pack-slack --version 2.1.0 \
--git-url https://github.com/example/pack-slack.git \
--git-ref v2.1.0 \
--archive-url https://example.com/packs/slack-2.1.0.zip
# Append to existing index file
attune pack index-entry --pack-dir ./pack-slack --version 2.1.0 \
--git-url https://github.com/example/pack-slack.git \
--git-ref v2.1.0 \
--archive-url https://example.com/packs/slack-2.1.0.zip \
--index-file registry.json \
--output registry.json
Output Example:
{
"ref": "slack",
"label": "Slack Integration",
"description": "Send messages, upload files, and monitor Slack channels",
"version": "2.1.0",
"author": "Example Team",
"email": "team@example.com",
"license": "Apache-2.0",
"runtime_deps": ["python3"],
"install_sources": [
{
"type": "git",
"url": "https://github.com/example/pack-slack.git",
"ref": "v2.1.0",
"checksum": "sha256:abc123..."
},
{
"type": "archive",
"url": "https://example.com/packs/slack-2.1.0.zip",
"checksum": "sha256:def456..."
}
],
"contents": {
"actions": [...],
"sensors": [...],
"triggers": [...]
}
}
Update Index File
Merge multiple index entries or update an existing index:
# Add entry to index
attune pack index-update --index registry.json --entry entry.json
# Merge multiple indices
attune pack index-merge --output combined.json registry1.json registry2.json
# Update pack version in index
attune pack index-update --index registry.json --pack slack --version 2.1.1 \
--git-ref v2.1.1 --archive-url https://example.com/packs/slack-2.1.1.zip
List Registries
attune pack registries
# Output:
# Priority | Name | URL | Status
# ---------|-------------------------|------------------------------------------|--------
# 1 | Official Attune Registry| https://registry.attune.io/index.json | Online
# 2 | Company Internal | https://internal.example.com/registry.json| Online
# 3 | Local Development | file:///opt/attune/local-registry.json | Online
Search Registry
# Search all registries
attune pack search <keyword>
# Search specific registry
attune pack search <keyword> --registry "Official Attune Registry"
# Example
attune pack search slack
# Output:
# Ref | Version | Description | Registry
# -------|---------|------------------------------------------|-------------------------
# slack | 2.1.0 | Send messages and monitor Slack channels | Official Attune Registry
Installation Process
Installation Workflow
┌─────────────────────────────────────────────────────────────────────┐
│ 1. Source Resolution │
│ - Registry reference → Search indices → Resolve install source │
│ - Direct URL → Use provided source │
│ - Local path → Use local filesystem │
└────────────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 2. Download/Copy Pack │
│ - Git: Clone repository to temp directory │
│ - Archive: Download and extract to temp directory │
│ - Local: Copy to temp directory │
└────────────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 3. Validate Pack Structure │
│ - Verify pack.yaml exists and is valid │
│ - Verify pack ref matches (if installing from registry) │
│ - Verify version matches (if specified) │
│ - Validate pack structure (actions, sensors, triggers) │
└────────────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 4. Check Dependencies │
│ - Verify Attune version compatibility │
│ - Check runtime dependencies (Python, Node.js, etc.) │
│ - Verify dependent packs are installed │
│ - Check Python/Node.js version requirements │
└────────────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 5. Setup Worker Environment │
│ - Python: Create virtualenv, install requirements.txt │
│ - Node.js: Create node_modules, run npm install │
│ - Shell: Verify scripts are executable │
└────────────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 6. Run Pack Tests (if present) │
│ - Execute test suite defined in pack │
│ - Verify all tests pass │
│ - Skip if --skip-tests flag provided │
└────────────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 7. Register Pack Components │
│ - Insert pack metadata into database │
│ - Register actions, sensors, triggers │
│ - Register bundled rules and workflows (if any) │
│ - Copy pack files to permanent location │
└────────────────┬────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────┐
│ 8. Cleanup │
│ - Remove temporary directory │
│ - Log installation success │
│ - Return pack ID and metadata │
└─────────────────────────────────────────────────────────────────────┘
Pack Storage Location
Installed packs are stored in the configured packs directory:
/var/lib/attune/packs/
├── slack/
│ ├── pack.yaml
│ ├── actions/
│ ├── sensors/
│ ├── triggers/
│ ├── requirements.txt
│ ├── .venv/ # Python virtualenv (if applicable)
│ └── metadata.json # Installation metadata
├── aws/
└── github/
Installation metadata includes:
{
"pack_ref": "slack",
"version": "2.1.0",
"installed_at": "2024-01-20T12:00:00Z",
"installed_from": {
"type": "git",
"url": "https://github.com/example/pack-slack.git",
"ref": "v2.1.0"
},
"checksum": "sha256:abc123...",
"registry": "Official Attune Registry"
}
Checksum Verification
To ensure pack integrity, checksums are verified during installation:
Supported Algorithms
sha256(recommended)sha512(recommended)sha1(legacy, not recommended)md5(legacy, not recommended)
Checksum Format
algorithm:hash
Examples:
sha256:abc123def456...sha512:789xyz...
Generating Checksums
For pack maintainers:
# Git repository (tar.gz snapshot)
sha256sum pack-slack-2.1.0.tar.gz
# Zip archive
sha256sum pack-slack-2.1.0.zip
# Using attune CLI
attune pack checksum ./pack-slack-2.1.0.zip
Verification Process
- Download/extract pack to temporary location
- Calculate checksum of downloaded content
- Compare with checksum in index file
- If mismatch, abort installation and report error
- If
verify_checksums: falsein config, skip verification (not recommended)
CI/CD Integration
GitHub Actions Example
Automate pack building and registry updates:
name: Build and Publish Pack
on:
push:
tags:
- 'v*'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Create pack archive
run: |
VERSION=${GITHUB_REF#refs/tags/v}
zip -r pack-slack-${VERSION}.zip . -x ".git/*" ".github/*"
- name: Calculate checksum
id: checksum
run: |
CHECKSUM=$(sha256sum pack-slack-*.zip | awk '{print $1}')
echo "checksum=sha256:${CHECKSUM}" >> $GITHUB_OUTPUT
- name: Upload to artifact storage
run: |
VERSION=${GITHUB_REF#refs/tags/v}
aws s3 cp pack-slack-${VERSION}.zip s3://my-bucket/packs/
- name: Generate registry entry
run: |
VERSION=${GITHUB_REF#refs/tags/v}
attune pack index-entry \
--pack-dir . \
--version ${VERSION} \
--git-url https://github.com/example/pack-slack.git \
--git-ref ${GITHUB_REF#refs/tags/} \
--archive-url https://my-bucket.s3.amazonaws.com/packs/pack-slack-${VERSION}.zip \
--checksum ${{ steps.checksum.outputs.checksum }} \
> entry.json
- name: Update registry index
run: |
# Download current index
wget https://registry.example.com/index.json
# Add new entry
attune pack index-update \
--index index.json \
--entry entry.json \
--output index.json
# Upload updated index
aws s3 cp index.json s3://registry.example.com/
Error Handling
Installation Errors
| Error | Cause | Resolution |
|---|---|---|
| Pack not found in registry | Pack ref doesn't exist in any configured registry | Check pack name, verify registry is online |
| Checksum mismatch | Downloaded pack doesn't match expected checksum | Pack may be corrupted or tampered with; contact pack maintainer |
| Pack already installed | Pack with same ref already exists | Use --force to reinstall |
| Dependency not met | Required Attune version, runtime, or pack not available | Update Attune, install runtime, or install dependency pack |
| Invalid pack structure | pack.yaml missing or invalid | Fix pack structure |
| Tests failed | Pack tests did not pass | Fix pack code or use --skip-tests (not recommended) |
Registry Errors
| Error | Cause | Resolution |
|---|---|---|
| Registry unreachable | Network error, DNS failure | Check network, verify URL |
| Invalid index format | Index JSON is malformed | Contact registry maintainer |
| Authentication failed | Registry requires authentication but token is invalid | Update registry token in configuration |
Security Considerations
1. HTTPS Only (Recommended)
Configure allow_http: false to reject non-HTTPS registries:
pack_registry:
allow_http: false # Only allow HTTPS
2. Checksum Verification
Always enable checksum verification in production:
pack_registry:
verify_checksums: true
3. Registry Authentication
For private registries, use secure token storage:
export REGISTRY_TOKEN=$(cat /run/secrets/registry_token)
export ATTUNE__PACK_REGISTRY__INDICES="https://registry.example.com/index.json"
4. Code Review
- Review pack code before installation
- Use
--skip-testscautiously - Test packs in non-production environment first
5. Signature Verification (Future)
Future enhancement: GPG signature verification for pack archives:
{
"type": "archive",
"url": "https://example.com/packs/slack-2.1.0.zip",
"checksum": "sha256:abc123...",
"signature": "https://example.com/packs/slack-2.1.0.zip.sig",
"signing_key": "0x1234567890ABCDEF"
}
Future Enhancements
Version 1.1
- Semantic version matching:
slack@^2.0.0,slack@~2.1.0 - Pack updates:
attune pack update <ref>to upgrade to latest version - Dependency resolution: Automatic installation of pack dependencies
Version 1.2
- GPG signature verification: Cryptographic verification of pack authenticity
- Pack ratings and reviews: Community feedback in registry
- Usage statistics: Download counts, popularity metrics
Version 1.3
- Private pack authentication: Token-based authentication for private packs
- Pack mirroring: Automatic mirroring of registry indices for redundancy
- Delta updates: Only download changed files when updating packs