attune/tests/E2E_QUICK_START.md

E2E Test Quick Start Guide

Last Updated: 2026-01-27
Status: Ready for Testing

---

Overview

This guide helps you quickly get started with running end-to-end (E2E) integration tests for the Attune platform.

Test Structure

tests/
├── e2e/                    # New tiered test structure
│   ├── tier1/              # Core automation flows (MVP essential)
│   │   ├── test_t1_01_interval_timer.py
│   │   ├── test_t1_02_date_timer.py
│   │   └── test_t1_04_webhook_trigger.py
│   ├── tier2/              # Orchestration & data flow (coming soon)
│   └── tier3/              # Advanced features (coming soon)
├── helpers/                # Test utilities
│   ├── __init__.py
│   ├── client.py           # AttuneClient API wrapper
│   ├── polling.py          # Wait/poll utilities
│   └── fixtures.py         # Test data creators
├── fixtures/               # Test data
│   └── packs/
│       └── test_pack/
├── conftest.py             # Pytest configuration
├── pytest.ini              # Pytest settings
├── requirements.txt        # Python dependencies
└── run_e2e_tests.sh        # Test runner script

---

Prerequisites

1. Services Running

All 5 Attune services must be running:

# Terminal 1 - API Service
cd crates/api
cargo run

# Terminal 2 - Executor Service
cd crates/executor
cargo run

# Terminal 3 - Worker Service
cd crates/worker
cargo run

# Terminal 4 - Sensor Service
cd crates/sensor
cargo run

# Terminal 5 - Notifier Service
cd crates/notifier
cargo run

2. Database & Message Queue

# PostgreSQL (if not already running)
docker run -d --name postgres \
  -e POSTGRES_PASSWORD=postgres \
  -p 5432:5432 \
  postgres:14

# RabbitMQ (if not already running)
docker run -d --name rabbitmq \
  -p 5672:5672 \
  -p 15672:15672 \
  rabbitmq:3-management

3. Database Migrations

# Ensure migrations are applied
sqlx migrate run

---

Quick Start

Option 1: Automated Runner (Recommended)

# Run all tests with automatic setup
cd tests
./run_e2e_tests.sh --setup

# Run specific tier
./run_e2e_tests.sh --tier 1

# Run with verbose output
./run_e2e_tests.sh --tier 1 -v

# Run and stop on first failure
./run_e2e_tests.sh --tier 1 -s

Option 2: Direct Pytest

cd tests

# Install dependencies first (one-time setup)
python3 -m venv venvs/e2e
source venvs/e2e/bin/activate
pip install -r requirements.txt

# Run all Tier 1 tests
pytest e2e/tier1/ -v

# Run specific test
pytest e2e/tier1/test_t1_01_interval_timer.py -v

# Run by marker
pytest -m tier1 -v
pytest -m webhook -v
pytest -m timer -v

# Run with live output
pytest e2e/tier1/ -v -s

---

Test Tiers

Tier 1: Core Automation Flows ✅

Status: 3 tests implemented
Priority: Critical (MVP)
Duration: ~2 minutes total

Tests implemented:

• ✅ T1.1: Interval Timer Automation (30s)
• ✅ T1.2: Date Timer One-Shot Execution (15s)
• ✅ T1.4: Webhook Trigger with Payload (20s)

Tests pending:

• ⏳ T1.3: Cron Timer Execution
• ⏳ T1.5: Workflow with Array Iteration
• ⏳ T1.6: Key-Value Store Access
• ⏳ T1.7: Multi-Tenant Isolation
• ⏳ T1.8: Action Failure Handling

Run with:

./run_e2e_tests.sh --tier 1

Tier 2: Orchestration & Data Flow ⏳

Status: Not yet implemented
Priority: High
Tests: Workflows, inquiries, error handling

Coming soon!

Tier 3: Advanced Features ⏳

Status: Not yet implemented
Priority: Medium
Tests: Performance, security, edge cases

Coming soon!

---

Example Test Run

$ cd tests
$ ./run_e2e_tests.sh --tier 1

╔════════════════════════════════════════════════════════╗
║  Attune E2E Integration Test Suite                    ║
╚════════════════════════════════════════════════════════╝

  Tier 1: Core Automation Flows (MVP Essential)
  Tests: Timers, Webhooks, Basic Workflows

ℹ Checking if Attune services are running...
✓ API service is running at http://localhost:8080

═══ Running Tier 1 Tests
ℹ Command: pytest e2e/tier1/ -v -m tier1

======================== test session starts =========================
platform linux -- Python 3.11.0, pytest-7.4.3
rootdir: /path/to/attune/tests
configfile: pytest.ini
testpaths: tests/e2e
plugins: timeout-2.1.0, asyncio-0.21.0

collected 6 items

e2e/tier1/test_t1_01_interval_timer.py::TestIntervalTimerAutomation::test_interval_timer_creates_executions PASSED
e2e/tier1/test_t1_01_interval_timer.py::TestIntervalTimerAutomation::test_interval_timer_precision PASSED
e2e/tier1/test_t1_02_date_timer.py::TestDateTimerAutomation::test_date_timer_fires_once PASSED
e2e/tier1/test_t1_02_date_timer.py::TestDateTimerAutomation::test_date_timer_past_date PASSED
e2e/tier1/test_t1_04_webhook_trigger.py::TestWebhookTrigger::test_webhook_trigger_with_payload PASSED
e2e/tier1/test_t1_04_webhook_trigger.py::TestWebhookTrigger::test_multiple_webhook_posts PASSED

======================= 6 passed in 85.32s ==========================

✓ All tests passed!

╔════════════════════════════════════════════════════════╗
║  ✓ All E2E tests passed successfully                  ║
╚════════════════════════════════════════════════════════╝

---

Configuration

Environment Variables

# API URL (default: http://localhost:8080)
export ATTUNE_API_URL="http://localhost:8080"

# Test timeout in seconds (default: 60)
export TEST_TIMEOUT="60"

# Test user credentials (optional)
export TEST_USER_LOGIN="test@attune.local"
export TEST_USER_PASSWORD="TestPass123!"

pytest.ini Settings

Key configuration in tests/pytest.ini:

• Test discovery patterns
• Markers for test categorization
• Logging configuration
• Timeout settings

---

Troubleshooting

Services Not Running

Error: API service is not reachable

Solution:

1. Check all 5 services are running (see Prerequisites)
2. Verify API responds: curl http://localhost:8080/health
3. Check service logs for errors

Tests Timing Out

Error: TimeoutError: Execution did not reach status 'succeeded'

Possible Causes:

• Executor service not running
• Worker service not consuming queue
• RabbitMQ connection issues
• Sensor service not detecting triggers

Solution:

1. Check all services are running: ps aux | grep attune
2. Check RabbitMQ queues: http://localhost:15672 (guest/guest)
3. Check database: psql -d attune_dev -c "SELECT * FROM attune.execution ORDER BY created DESC LIMIT 5;"
4. Increase timeout: export TEST_TIMEOUT=120

Import Errors

Error: ModuleNotFoundError: No module named 'helpers'

Solution:

# Make sure you're in the tests directory
cd tests

# Activate venv
source venvs/e2e/bin/activate

# Install dependencies
pip install -r requirements.txt

# Set PYTHONPATH
export PYTHONPATH="$PWD:$PYTHONPATH"

Database Issues

Error: Database connection failed or Relation does not exist

Solution:

# Verify database exists
psql -U postgres -l | grep attune

# Run migrations
cd /path/to/attune
sqlx migrate run

# Check tables exist
psql -d attune_dev -c "\dt attune.*"

Test Isolation Issues

Problem: Tests interfere with each other

Solution:

• Use unique_user_client fixture for complete isolation
• Tests automatically get unique references via unique_ref()
• Each test creates its own resources (pack, trigger, action, rule)

Flaky Timer Tests

Problem: Timer tests occasionally fail with timing issues

Solution:

• Timer tests have built-in tolerance (±1-2 seconds)
• System load can affect timing - run on idle system
• Increase poll intervals if needed
• Check sensor service logs for timer processing

---

Writing New Tests

Test Template

#!/usr/bin/env python3
"""
T1.X: Test Name

Test description and flow.
"""

import pytest
from helpers import (
    AttuneClient,
    create_echo_action,
    create_rule,
    wait_for_execution_status,
)


@pytest.mark.tier1  # or tier2, tier3
@pytest.mark.integration
@pytest.mark.timeout(30)
class TestMyFeature:
    """Test my feature"""

    def test_my_scenario(self, client: AttuneClient, pack_ref: str):
        """Test that my scenario works"""
        
        print(f"\n=== T1.X: Test Name ===")
        
        # Step 1: Create resources
        print("\n[1/3] Creating resources...")
        action = create_echo_action(client=client, pack_ref=pack_ref)
        print(f"✓ Action created: {action['ref']}")
        
        # Step 2: Execute action
        print("\n[2/3] Executing action...")
        # ... test logic ...
        
        # Step 3: Verify results
        print("\n[3/3] Verifying results...")
        # ... assertions ...
        
        print("\n✓ Test PASSED")

Available Helpers

Fixtures (conftest.py):

• client - Authenticated API client
• unique_user_client - Client with unique user (isolation)
• test_pack - Test pack fixture
• pack_ref - Pack reference string
• wait_time - Standard wait time dict

Client Methods (helpers/client.py):

• client.register_pack(path)
• client.create_action(...)
• client.create_trigger(...)
• client.create_rule(...)
• client.fire_webhook(id, payload)
• client.list_executions(...)
• client.get_execution(id)

Polling Utilities (helpers/polling.py):

• wait_for_execution_status(client, id, status, timeout)
• wait_for_execution_count(client, count, ...)
• wait_for_event_count(client, count, ...)
• wait_for_condition(fn, timeout, ...)

Fixture Creators (helpers/fixtures.py):

• create_interval_timer(client, seconds, ...)
• create_date_timer(client, fire_at, ...)
• create_cron_timer(client, expression, ...)
• create_webhook_trigger(client, ...)
• create_echo_action(client, ...)
• create_rule(client, trigger_id, action_ref, ...)

---

Next Steps

1. Run existing tests to verify setup:

   ./run_e2e_tests.sh --tier 1
   

2. Implement remaining Tier 1 tests:

   • T1.3: Cron Timer
   • T1.5: Workflow with-items
   • T1.6: Datastore access
   • T1.7: Multi-tenancy
   • T1.8: Error handling

3. Implement Tier 2 tests (orchestration)

4. Implement Tier 3 tests (advanced features)

5. CI/CD Integration:

   • Add GitHub Actions workflow
   • Run tests on every PR
   • Generate test reports

---

Resources

• Test Plan: docs/e2e-test-plan.md - Complete test specifications
• Test Status: docs/testing-status.md - Current testing coverage
• API Docs: docs/api-*.md - API endpoint documentation
• Architecture: docs/ - System architecture documentation

---

Support

If you encounter issues:

1. Check service logs in each terminal
2. Verify database state: psql -d attune_dev
3. Check RabbitMQ management UI: http://localhost:15672
4. Review test output for detailed error messages
5. Enable verbose output: ./run_e2e_tests.sh -v -s

Status: Ready to run! 🚀