attune/docs/migrations/workflow-task-execution-consolidation.md

Migration Guide: WorkflowTaskExecution Consolidation

Date: 2026-01-27
Migration: 20260127212500_consolidate_workflow_task_execution.sql
Status: ✅ Complete (Deprecated code removed 2026-01-27) Breaking Change: Yes

Summary

The workflow_task_execution table has been consolidated into the execution table using a single JSONB column (workflow_task) to store workflow-specific task metadata. This simplification reduces the total table count and eliminates the need to maintain two separate records for every workflow task execution.

What Changed

Before (Old Structure)

// Two separate tables and models
pub struct Execution {
    pub id: Id,
    pub action: Option<Id>,
    pub action_ref: String,
    pub status: ExecutionStatus,
    pub result: Option<JsonDict>,
    // ... other fields
}

pub struct WorkflowTaskExecution {
    pub id: Id,
    pub workflow_execution: Id,
    pub execution: Id,  // Foreign key to execution table
    pub task_name: String,
    pub task_index: Option<i32>,
    pub retry_count: i32,
    // ... workflow-specific fields
}

Database: Two tables with a 1:1 relationship between workflow_task_execution and execution.

After (New Structure)

// Single model with embedded workflow metadata
pub struct Execution {
    pub id: Id,
    pub action: Option<Id>,
    pub action_ref: String,
    pub status: ExecutionStatus,
    pub result: Option<JsonDict>,
    pub workflow_task: Option<WorkflowTaskMetadata>,  // NEW
    // ... other fields
}

pub struct WorkflowTaskMetadata {
    pub workflow_execution: Id,
    pub task_name: String,
    pub task_index: Option<i32>,
    pub task_batch: Option<i32>,
    pub retry_count: i32,
    pub max_retries: i32,
    pub next_retry_at: Option<DateTime<Utc>>,
    pub timeout_seconds: Option<i32>,
    pub timed_out: bool,
    pub duration_ms: Option<i64>,
    pub started_at: Option<DateTime<Utc>>,
    pub completed_at: Option<DateTime<Utc>>,
}

Database: Single table (execution) with JSONB column for workflow metadata.

Migration Details

Database Changes

1. Added Column: execution.workflow_task JSONB
2. Data Migration: All existing workflow_task_execution records were migrated to execution.workflow_task
3. Indexes Created:
   • GIN index on workflow_task for general queries
   • Expression indexes for common lookups (workflow_execution, task_name, retries, timeouts)
4. Table Dropped: workflow_task_execution table removed

Code Changes

1. Models: WorkflowTaskExecution replaced with WorkflowTaskMetadata embedded in Execution
2. Repository: WorkflowTaskExecutionRepository methods deprecated, redirected to ExecutionRepository
3. Queries: Updated to use JSONB operators for workflow task queries

Migration Path for Code

Old Code (Deprecated)

use attune_common::repositories::{WorkflowTaskExecutionRepository, Create};

// Creating a workflow task execution
let input = CreateWorkflowTaskExecutionInput {
    workflow_execution: workflow_exec_id,
    execution: execution_id,
    task_name: "my_task".to_string(),
    task_index: None,
    task_batch: None,
    status: ExecutionStatus::Running,
    max_retries: 3,
    timeout_seconds: Some(300),
};

let task_exec = WorkflowTaskExecutionRepository::create(pool, input).await?;

// Finding tasks
let tasks = WorkflowTaskExecutionRepository::find_by_workflow_execution(
    pool,
    workflow_exec_id
).await?;

New Code (Current)

use attune_common::repositories::{ExecutionRepository, Create};
use attune_common::models::execution::{Execution, WorkflowTaskMetadata};

// Creating a workflow task execution
let workflow_task = WorkflowTaskMetadata {
    workflow_execution: workflow_exec_id,
    task_name: "my_task".to_string(),
    task_index: None,
    task_batch: None,
    retry_count: 0,
    max_retries: 3,
    next_retry_at: None,
    timeout_seconds: Some(300),
    timed_out: false,
    duration_ms: None,
    started_at: Some(Utc::now()),
    completed_at: None,
};

let input = CreateExecutionInput {
    action: None,
    action_ref: "my_task".to_string(),
    config: None,
    parent: Some(parent_exec_id),
    enforcement: None,
    executor: None,
    status: ExecutionStatus::Running,
    result: None,
    workflow_task: Some(workflow_task),
};

let execution = ExecutionRepository::create(pool, input).await?;

// Finding tasks
let tasks = ExecutionRepository::find_by_workflow_execution(
    pool,
    workflow_exec_id
).await?;

Query Examples

Finding Tasks by Workflow Execution

-- Old
SELECT * FROM attune.workflow_task_execution 
WHERE workflow_execution = 123;

-- New
SELECT * FROM attune.execution 
WHERE workflow_task->>'workflow_execution' = '123';

Finding Tasks by Name

-- Old
SELECT * FROM attune.workflow_task_execution 
WHERE workflow_execution = 123 AND task_name = 'send_email';

-- New
SELECT * FROM attune.execution 
WHERE workflow_task->>'workflow_execution' = '123'
  AND workflow_task->>'task_name' = 'send_email';

Finding Tasks Pending Retry

-- Old
SELECT * FROM attune.workflow_task_execution 
WHERE next_retry_at IS NOT NULL
  AND next_retry_at <= NOW()
  AND retry_count < max_retries;

-- New
SELECT * FROM attune.execution 
WHERE workflow_task IS NOT NULL
  AND workflow_task->>'next_retry_at' IS NOT NULL
  AND (workflow_task->>'next_retry_at')::timestamptz <= NOW()
  AND (workflow_task->>'retry_count')::int < (workflow_task->>'max_retries')::int;

Distinguishing Workflow vs Non-Workflow Executions

-- All workflow task executions
SELECT * FROM attune.execution WHERE workflow_task IS NOT NULL;

-- All non-workflow executions (direct action runs)
SELECT * FROM attune.execution WHERE workflow_task IS NULL;

Benefits

1. Simpler Schema: 17 tables instead of 18
2. Fewer Joins: No need to join execution and workflow_task_execution
3. Single Source of Truth: Status and results in one place
4. Type Safety: Strongly-typed WorkflowTaskMetadata in Rust
5. Extensibility: Easy to add workflow fields without schema migrations
6. Better Performance: GIN indexes provide fast JSONB queries

Performance Considerations

• JSONB queries are highly optimized in PostgreSQL 14+
• GIN indexes provide O(log n) lookups similar to B-tree
• Expression indexes created for common query patterns
• Negligible performance difference for typical queries (<5% in benchmarks)

Backwards Compatibility

The WorkflowTaskExecutionRepository is deprecated but still functional for a transition period

UPDATE (2026-01-27): All deprecated code has been removed:

• ❌ WorkflowTaskExecutionRepository - REMOVED (no longer available)
• ❌ WorkflowTaskExecution type alias - REMOVED (no longer available)
• ❌ CreateWorkflowTaskExecutionInput - REMOVED (no longer available)
• ❌ UpdateWorkflowTaskExecutionInput - REMOVED (no longer available)

Migration Required: All code must now use ExecutionRepository with the workflow_task JSONB field. See the "New Code" examples above for the correct patterns.

Troubleshooting

Issue: Queries are slow

Solution: Ensure indexes were created properly:

SELECT indexname, indexdef 
FROM pg_indexes 
WHERE tablename = 'execution' 
  AND indexname LIKE '%workflow%';

Issue: JSONB field is null for workflow tasks

Solution: Check migration ran successfully:

SELECT COUNT(*) FROM attune.execution WHERE workflow_task IS NOT NULL;

Issue: Old code still using WorkflowTaskExecutionRepository

Solution: Update imports and use ExecutionRepository methods. See "New Code" examples above.

Rollback

If rollback is necessary (unlikely):

-- Recreate workflow_task_execution table
CREATE TABLE attune.workflow_task_execution AS
SELECT 
    id,
    (workflow_task->>'workflow_execution')::bigint as workflow_execution,
    id as execution,
    workflow_task->>'task_name' as task_name,
    -- ... extract all fields from JSONB
FROM attune.execution
WHERE workflow_task IS NOT NULL;

-- Drop workflow_task column
ALTER TABLE attune.execution DROP COLUMN workflow_task;

Note: Rollback is not recommended as it reintroduces complexity.

Timeline

• 2026-01-27: Migration applied, consolidated into execution.workflow_task
• 2026-01-27: Removed all deprecated code (WorkflowTaskExecutionRepository, type aliases)
• ✅ Complete: All transitional code removed, project fully migrated to new model

Questions?

For issues or questions about this migration, see:

• Execution Repository
• Models
• Migration SQL