Files

David Culbreth 3b14c65998 re-uploading work

2026-02-04 17:46:30 -06:00

8.2 KiB

Raw Blame History

Migration to Generated API Client

Overview

The E2E tests are being migrated from a manually maintained AttuneClient to an auto-generated OpenAPI client. This migration improves:

Type Safety: Full Pydantic models with compile-time type checking
API Schema Accuracy: Client generated from OpenAPI spec matches API exactly
Maintainability: No manual field mapping to keep in sync
Future-Proof: Client regenerates automatically when API changes

Current Status

✅ Completed:

Generated Python client from OpenAPI spec (tests/generated_client/)
Created backward-compatible wrapper (tests/helpers/client_wrapper.py)
Updated dependencies (added attrs, httpx, python-dateutil)
Updated helpers/__init__.py to use wrapper

🔄 In Progress:

Testing wrapper compatibility with existing tests
Fixing any edge cases in wrapper implementation

📋 TODO:

Install updated dependencies in test venv
Run Tier 1 E2E tests with new client
Fix any compatibility issues discovered
Gradually remove wrapper as tests adopt generated client directly
Update documentation and examples

Architecture

Generated Client Structure

tests/generated_client/
├── api/                    # API endpoint modules
│   ├── actions/           # Action endpoints
│   ├── auth/              # Authentication endpoints
│   ├── enforcements/      # Enforcement endpoints
│   ├── events/            # Event endpoints
│   ├── executions/        # Execution endpoints
│   ├── health/            # Health check endpoints
│   ├── inquiries/         # Inquiry endpoints
│   ├── packs/             # Pack management endpoints
│   ├── rules/             # Rule endpoints
│   ├── secrets/           # Secret/key management endpoints
│   ├── sensors/           # Sensor endpoints
│   ├── triggers/          # Trigger endpoints
│   ├── webhooks/          # Webhook endpoints
│   └── workflows/         # Workflow endpoints
├── models/                # Pydantic models (200+ files)
├── client.py              # Client and AuthenticatedClient classes
├── errors.py              # Error types
├── types.py               # Helper types (UNSET, etc.)
└── pyproject.toml         # Package metadata

Wrapper Architecture

The wrapper (tests/helpers/client_wrapper.py) provides backward compatibility:

Same Interface: Maintains exact same method signatures as old client
Generated Backend: Uses generated API functions internally
Dict Conversion: Converts Pydantic models to dicts for compatibility
Auth Management: Handles login/logout and token management
ID to Ref Mapping: API uses ref in paths, wrapper handles ID lookups

Key Differences: Old vs New Client

API Uses `ref` in Paths, Not `id`

Old Behavior:

client.get_pack(pack_id=123)  # GET /api/v1/packs/123

New Behavior:

# API expects: GET /api/v1/packs/{ref}
client.get_pack("core")  # GET /api/v1/packs/core

Wrapper Solution: Lists all items, finds by ID, then fetches by ref.

Client Initialization

Old:

client = AttuneClient(
    base_url="http://localhost:8080",
    timeout=30,
    auto_login=True
)

New (Generated):

from generated_client import Client, AuthenticatedClient

# Unauthenticated client
client = Client(base_url="http://localhost:8080/api/v1")

# Authenticated client
auth_client = AuthenticatedClient(
    base_url="http://localhost:8080/api/v1",
    token="access_token_here"
)

Wrapper: Maintains old interface, manages both clients internally.

API Function Signatures

Generated API Pattern:

# Positional args for path params, keyword-only for client and query params
from generated_client.api.packs import get_pack

response = get_pack.sync(
    ref="core",              # Positional: path parameter
    client=auth_client       # Keyword-only: client instance
)

Response Handling

Generated API Returns:

Pydantic models (e.g., GetPackResponse200)
Models have to_dict() method
Responses wrap data in {"data": {...}} structure

Wrapper Converts:

response = gen_get_pack.sync(ref=ref, client=client)
if response:
    result = to_dict(response)  # Convert Pydantic to dict
    if isinstance(result, dict) and "data" in result:
        return result["data"]  # Unwrap data field

Migration Path

Phase 1: Wrapper Compatibility (Current)

Tests use existing AttuneClient interface, wrapper uses generated client:

# Test code (unchanged)
from helpers import AttuneClient

client = AttuneClient()
pack = client.get_pack_by_ref("core")

Phase 2: Direct Generated Client Usage (Future)

Tests migrate to use generated client directly:

from generated_client import AuthenticatedClient
from generated_client.api.packs import get_pack

auth_client = AuthenticatedClient(
    base_url="http://localhost:8080/api/v1",
    token=access_token
)

response = get_pack.sync(ref="core", client=auth_client)
pack_data = response.data if response else None

Phase 3: Wrapper Removal

Once all tests use generated client, remove wrapper and old client.

Regenerating the Client

When API schema changes:

cd tests
./scripts/generate-python-client.sh

This script:

Fetches OpenAPI spec from running API
Generates client with openapi-python-client
Installs into test venv

Common Issues & Solutions

Issue: Import Errors

Problem: ModuleNotFoundError: No module named 'attrs'

Solution: Install updated dependencies:

cd tests
source venvs/e2e/bin/activate
pip install -r requirements.txt

Issue: Field Name Mismatches

Problem: Test expects name but API returns label

Solution: API schema uses standardized fields:

ref: Unique identifier (e.g., core.echo)
label: Human-readable name
runtime: Execution runtime (was runner_type)

Update test code to use correct field names.

Issue: Path Parameter Confusion

Problem: API endpoint returns 404

Solution: Check if endpoint uses ref or id in path:

Most endpoints: /api/v1/{resource}/{ref}
Some endpoints: /api/v1/{resource}/id/{id}

Use wrapper methods that handle this automatically.

Testing Strategy

Run existing tests: Verify wrapper maintains compatibility
Check field names: Ensure tests use correct schema fields
Validate responses: Confirm data structure matches expectations
Test edge cases: Error handling, pagination, filtering
Performance check: Ensure no significant slowdown

Benefits of Migration

Before (Manual Client)

Pros:

Simple dict-based interface
Easy to use in tests

Cons:

Manual field mapping (out of sync with API)
No type safety
Frequent breakage on API changes
Missing endpoints
High maintenance burden

After (Generated Client)