attune-system/attune
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

re-uploading work

Browse Source
This commit is contained in:
David Culbreth
2026-02-04 17:46:30 -06:00
commit 3b14c65998
1388 changed files with 381262 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

463
tests/E2E_QUICK_START.md Normal file
View File

@@ -0,0 +1,463 @@
# E2E Test Quick Start Guide
**Last Updated**: 2026-01-27
**Status**: Ready for Testing
---
## Overview
This guide helps you quickly get started with running end-to-end (E2E) integration tests for the Attune platform.
### Test Structure
```
tests/
├── e2e/ # New tiered test structure
│ ├── tier1/ # Core automation flows (MVP essential)
│ │ ├── test_t1_01_interval_timer.py
│ │ ├── test_t1_02_date_timer.py
│ │ └── test_t1_04_webhook_trigger.py
│ ├── tier2/ # Orchestration & data flow (coming soon)
│ └── tier3/ # Advanced features (coming soon)
├── helpers/ # Test utilities
│ ├── __init__.py
│ ├── client.py # AttuneClient API wrapper
│ ├── polling.py # Wait/poll utilities
│ └── fixtures.py # Test data creators
├── fixtures/ # Test data
│ └── packs/
│ └── test_pack/
├── conftest.py # Pytest configuration
├── pytest.ini # Pytest settings
├── requirements.txt # Python dependencies
└── run_e2e_tests.sh # Test runner script
```
---
## Prerequisites
### 1. Services Running
All 5 Attune services must be running:
```bash
# Terminal 1 - API Service
cd crates/api
cargo run
# Terminal 2 - Executor Service
cd crates/executor
cargo run
# Terminal 3 - Worker Service
cd crates/worker
cargo run
# Terminal 4 - Sensor Service
cd crates/sensor
cargo run
# Terminal 5 - Notifier Service
cd crates/notifier
cargo run
```
### 2. Database & Message Queue
```bash
# PostgreSQL (if not already running)
docker run -d --name postgres \
-e POSTGRES_PASSWORD=postgres \
-p 5432:5432 \
postgres:14
# RabbitMQ (if not already running)
docker run -d --name rabbitmq \
-p 5672:5672 \
-p 15672:15672 \
rabbitmq:3-management
```
### 3. Database Migrations
```bash
# Ensure migrations are applied
sqlx migrate run
```
---
## Quick Start
### Option 1: Automated Runner (Recommended)
```bash
# Run all tests with automatic setup
cd tests
./run_e2e_tests.sh --setup
# Run specific tier
./run_e2e_tests.sh --tier 1
# Run with verbose output
./run_e2e_tests.sh --tier 1 -v
# Run and stop on first failure
./run_e2e_tests.sh --tier 1 -s
```
### Option 2: Direct Pytest
```bash
cd tests
# Install dependencies first (one-time setup)
python3 -m venv venvs/e2e
source venvs/e2e/bin/activate
pip install -r requirements.txt
# Run all Tier 1 tests
pytest e2e/tier1/ -v
# Run specific test
pytest e2e/tier1/test_t1_01_interval_timer.py -v
# Run by marker
pytest -m tier1 -v
pytest -m webhook -v
pytest -m timer -v
# Run with live output
pytest e2e/tier1/ -v -s
```
---
## Test Tiers
### Tier 1: Core Automation Flows ✅
**Status**: 3 tests implemented
**Priority**: Critical (MVP)
**Duration**: ~2 minutes total
Tests implemented:
- ✅ **T1.1**: Interval Timer Automation (30s)
- ✅ **T1.2**: Date Timer One-Shot Execution (15s)
- ✅ **T1.4**: Webhook Trigger with Payload (20s)
Tests pending:
- ⏳ **T1.3**: Cron Timer Execution
- ⏳ **T1.5**: Workflow with Array Iteration
- ⏳ **T1.6**: Key-Value Store Access
- ⏳ **T1.7**: Multi-Tenant Isolation
- ⏳ **T1.8**: Action Failure Handling
Run with:
```bash
./run_e2e_tests.sh --tier 1
```
### Tier 2: Orchestration & Data Flow ⏳
**Status**: Not yet implemented
**Priority**: High
**Tests**: Workflows, inquiries, error handling
Coming soon!
### Tier 3: Advanced Features ⏳
**Status**: Not yet implemented
**Priority**: Medium
**Tests**: Performance, security, edge cases
Coming soon!
---
## Example Test Run
```bash
$ cd tests
$ ./run_e2e_tests.sh --tier 1
╔════════════════════════════════════════════════════════╗
║ Attune E2E Integration Test Suite ║
╚════════════════════════════════════════════════════════╝
Tier 1: Core Automation Flows (MVP Essential)
Tests: Timers, Webhooks, Basic Workflows
ℹ Checking if Attune services are running...
✓ API service is running at http://localhost:8080
═══ Running Tier 1 Tests
ℹ Command: pytest e2e/tier1/ -v -m tier1
======================== test session starts =========================
platform linux -- Python 3.11.0, pytest-7.4.3
rootdir: /path/to/attune/tests
configfile: pytest.ini
testpaths: tests/e2e
plugins: timeout-2.1.0, asyncio-0.21.0
collected 6 items
e2e/tier1/test_t1_01_interval_timer.py::TestIntervalTimerAutomation::test_interval_timer_creates_executions PASSED
e2e/tier1/test_t1_01_interval_timer.py::TestIntervalTimerAutomation::test_interval_timer_precision PASSED
e2e/tier1/test_t1_02_date_timer.py::TestDateTimerAutomation::test_date_timer_fires_once PASSED
e2e/tier1/test_t1_02_date_timer.py::TestDateTimerAutomation::test_date_timer_past_date PASSED
e2e/tier1/test_t1_04_webhook_trigger.py::TestWebhookTrigger::test_webhook_trigger_with_payload PASSED
e2e/tier1/test_t1_04_webhook_trigger.py::TestWebhookTrigger::test_multiple_webhook_posts PASSED
======================= 6 passed in 85.32s ==========================
✓ All tests passed!
╔════════════════════════════════════════════════════════╗
║ ✓ All E2E tests passed successfully ║
╚════════════════════════════════════════════════════════╝
```
---
## Configuration
### Environment Variables
```bash
# API URL (default: http://localhost:8080)
export ATTUNE_API_URL="http://localhost:8080"
# Test timeout in seconds (default: 60)
export TEST_TIMEOUT="60"
# Test user credentials (optional)
export TEST_USER_LOGIN="test@attune.local"
export TEST_USER_PASSWORD="TestPass123!"
```
### pytest.ini Settings
Key configuration in `tests/pytest.ini`:
- Test discovery patterns
- Markers for test categorization
- Logging configuration
- Timeout settings
---
## Troubleshooting
### Services Not Running
**Error**: `API service is not reachable`
**Solution**:
1. Check all 5 services are running (see Prerequisites)
2. Verify API responds: `curl http://localhost:8080/health`
3. Check service logs for errors
### Tests Timing Out
**Error**: `TimeoutError: Execution did not reach status 'succeeded'`
**Possible Causes**:
- Executor service not running
- Worker service not consuming queue
- RabbitMQ connection issues
- Sensor service not detecting triggers
**Solution**:
1. Check all services are running: `ps aux | grep attune`
2. Check RabbitMQ queues: http://localhost:15672 (guest/guest)
3. Check database: `psql -d attune_dev -c "SELECT * FROM attune.execution ORDER BY created DESC LIMIT 5;"`
4. Increase timeout: `export TEST_TIMEOUT=120`
### Import Errors
**Error**: `ModuleNotFoundError: No module named 'helpers'`
**Solution**:
```bash
# Make sure you're in the tests directory
cd tests
# Activate venv
source venvs/e2e/bin/activate
# Install dependencies
pip install -r requirements.txt
# Set PYTHONPATH
export PYTHONPATH="$PWD:$PYTHONPATH"
```
### Database Issues
**Error**: `Database connection failed` or `Relation does not exist`
**Solution**:
```bash
# Verify database exists
psql -U postgres -l | grep attune
# Run migrations
cd /path/to/attune
sqlx migrate run
# Check tables exist
psql -d attune_dev -c "\dt attune.*"
```
### Test Isolation Issues
**Problem**: Tests interfere with each other
**Solution**:
- Use `unique_user_client` fixture for complete isolation
- Tests automatically get unique references via `unique_ref()`
- Each test creates its own resources (pack, trigger, action, rule)
### Flaky Timer Tests
**Problem**: Timer tests occasionally fail with timing issues
**Solution**:
- Timer tests have built-in tolerance (±1-2 seconds)
- System load can affect timing - run on idle system
- Increase poll intervals if needed
- Check sensor service logs for timer processing
---
## Writing New Tests
### Test Template
```python
#!/usr/bin/env python3
"""
T1.X: Test Name
Test description and flow.
"""
import pytest
from helpers import (
AttuneClient,
create_echo_action,
create_rule,
wait_for_execution_status,
)
@pytest.mark.tier1 # or tier2, tier3
@pytest.mark.integration
@pytest.mark.timeout(30)
class TestMyFeature:
"""Test my feature"""
def test_my_scenario(self, client: AttuneClient, pack_ref: str):
"""Test that my scenario works"""
print(f"\n=== T1.X: Test Name ===")
# Step 1: Create resources
print("\n[1/3] Creating resources...")
action = create_echo_action(client=client, pack_ref=pack_ref)
print(f"✓ Action created: {action['ref']}")
# Step 2: Execute action
print("\n[2/3] Executing action...")
# ... test logic ...
# Step 3: Verify results
print("\n[3/3] Verifying results...")
# ... assertions ...
print("\n✓ Test PASSED")
```
### Available Helpers
**Fixtures** (conftest.py):
- `client` - Authenticated API client
- `unique_user_client` - Client with unique user (isolation)
- `test_pack` - Test pack fixture
- `pack_ref` - Pack reference string
- `wait_time` - Standard wait time dict
**Client Methods** (helpers/client.py):
- `client.register_pack(path)`
- `client.create_action(...)`
- `client.create_trigger(...)`
- `client.create_rule(...)`
- `client.fire_webhook(id, payload)`
- `client.list_executions(...)`
- `client.get_execution(id)`
**Polling Utilities** (helpers/polling.py):
- `wait_for_execution_status(client, id, status, timeout)`
- `wait_for_execution_count(client, count, ...)`
- `wait_for_event_count(client, count, ...)`
- `wait_for_condition(fn, timeout, ...)`
**Fixture Creators** (helpers/fixtures.py):
- `create_interval_timer(client, seconds, ...)`
- `create_date_timer(client, fire_at, ...)`
- `create_cron_timer(client, expression, ...)`
- `create_webhook_trigger(client, ...)`
- `create_echo_action(client, ...)`
- `create_rule(client, trigger_id, action_ref, ...)`
---
## Next Steps
1. **Run existing tests** to verify setup:
```bash
./run_e2e_tests.sh --tier 1
```
2. **Implement remaining Tier 1 tests**:
- T1.3: Cron Timer
- T1.5: Workflow with-items
- T1.6: Datastore access
- T1.7: Multi-tenancy
- T1.8: Error handling
3. **Implement Tier 2 tests** (orchestration)
4. **Implement Tier 3 tests** (advanced features)
5. **CI/CD Integration**:
- Add GitHub Actions workflow
- Run tests on every PR
- Generate test reports
---
## Resources
- **Test Plan**: `docs/e2e-test-plan.md` - Complete test specifications
- **Test Status**: `docs/testing-status.md` - Current testing coverage
- **API Docs**: `docs/api-*.md` - API endpoint documentation
- **Architecture**: `docs/` - System architecture documentation
---
## Support
If you encounter issues:
1. Check service logs in each terminal
2. Verify database state: `psql -d attune_dev`
3. Check RabbitMQ management UI: http://localhost:15672
4. Review test output for detailed error messages
5. Enable verbose output: `./run_e2e_tests.sh -v -s`
**Status**: Ready to run! 🚀