attune/work-summary/2026-02-03-canonical-message-types.md

Work Summary: Canonical Message Types Fix

Date: 2026-02-03
Issue: ExecutionManager still receiving deserialization errors after queue separation
Root Cause: Worker and Executor using local payload structs instead of canonical types
Status: ✅ Fixed

Problem Description

Even after separating the queues, the executor was still logging deserialization errors:

ERROR: Failed to deserialize message: missing field `action_id` at line 1 column 527

This occurred because old messages were in the queue, AND more importantly, the Worker and Executor services were using local struct definitions that didn't match the canonical message types in attune_common::mq::messages.

Root Cause Analysis

Mismatched Message Definitions

Three different versions of execution-related payloads existed:

1. Canonical (in attune_common::mq::messages):

   pub struct ExecutionStatusChangedPayload {
       pub execution_id: Id,
       pub action_ref: String,        // ✅ Has action_ref
       pub previous_status: String,
       pub new_status: String,
       pub changed_at: DateTime<Utc>,
   }
   
   pub struct ExecutionCompletedPayload {
       pub execution_id: Id,
       pub action_id: Id,             // ✅ Has action_id (required)
       pub action_ref: String,
       pub status: String,
       pub result: Option<JsonValue>,
       pub completed_at: DateTime<Utc>,
   }
   

2. Worker's local version (crates/worker/src/service.rs):

   struct ExecutionStatusPayload {
       pub execution_id: i64,
       pub status: String,
       pub result: Option<JsonValue>,
       pub error: Option<String>,     // ❌ Extra field
   }
   // No action_ref, different structure
   

3. Executor's local version (crates/executor/src/execution_manager.rs):

   struct ExecutionStatusPayload {
       execution_id: i64,
       status: String,
       result: Option<JsonValue>,     // ❌ Wrong fields
   }
   
   struct ExecutionCompletedPayload {
       execution_id: i64,
       status: String,
       result: Option<JsonValue>,
       action_ref: String,
       enforcement_id: Option<i64>,   // ❌ Wrong field
   }
   // Missing action_id, has enforcement_id instead
   

The Mismatch Chain

Worker publishes → ExecutionStatusChangedPayload (canonical)
                   ↓
Queue receives → {execution_id, action_ref, previous_status, new_status, changed_at}
                   ↓
Executor expects → ExecutionStatusPayload (local)
                   ↓
Deserialization → Tries to map fields
                   ↓
ERROR → "missing field `action_id`" (when trying different payload types)

Solution Implemented

1. Updated Worker to Use Canonical Types

File: attune/crates/worker/src/service.rs

• Removed local ExecutionStatusPayload struct
• Imported ExecutionStatusChangedPayload from attune_common::mq
• Updated publish_status_update to:
  • Fetch execution from database to get action_ref and previous_status
  • Create canonical ExecutionStatusChangedPayload with all required fields
  • Add with_source("worker") for traceability

// Before
struct ExecutionStatusPayload {
    pub execution_id: i64,
    pub status: String,
    pub result: Option<JsonValue>,
    pub error: Option<String>,
}

// After
use attune_common::mq::ExecutionStatusChangedPayload;

let payload = ExecutionStatusChangedPayload {
    execution_id,
    action_ref: execution.action_ref,
    previous_status: format!("{:?}", execution.status).to_lowercase(),
    new_status: new_status_str.to_string(),
    changed_at: Utc::now(),
};

2. Updated Executor to Use Canonical Types

File: attune/crates/executor/src/execution_manager.rs

• Removed local ExecutionStatusPayload and ExecutionCompletedPayload structs
• Imported canonical types from attune_common::mq
• Updated process_status_change to:
  • Use ExecutionStatusChangedPayload instead of local struct
  • Access new_status instead of status
  • Remove attempt to read result field (not in status change messages)
• Updated publish_completion_notification to:
  • Use canonical ExecutionCompletedPayload
  • Include action_id (required field)
  • Include completed_at timestamp
  • Remove enforcement_id field (not in canonical type)

// Before
struct ExecutionStatusPayload {
    execution_id: i64,
    status: String,
    result: Option<JsonValue>,
}

// After
use attune_common::mq::{ExecutionStatusChangedPayload, ExecutionCompletedPayload};

async fn process_status_change(
    pool: &PgPool,
    publisher: &Publisher,
    envelope: &MessageEnvelope<ExecutionStatusChangedPayload>,  // ✅ Canonical
) -> Result<()> {
    let status_str = &envelope.payload.new_status;  // ✅ Correct field name
    // ...
}

3. Database Queries Added

Both Worker and Executor now query the database to get fields needed for canonical payloads:

• Worker: Fetches execution to get action_ref and previous_status
• Executor: Already fetching execution, now uses it for action_id

This ensures messages contain all required data without embedding it in every call.

Message Flow After Fix

Status Change Flow

Worker → ExecutionRepository::find_by_id()
       → Get action_ref, current status
       → Create ExecutionStatusChangedPayload {
           execution_id,
           action_ref,              ✅
           previous_status,         ✅
           new_status,              ✅
           changed_at               ✅
         }
       → Publish to "execution.status.changed"
       → Queue: attune.execution.status.queue
       → ExecutionManager consumes
       → Deserializes ExecutionStatusChangedPayload ✅ SUCCESS

Completion Flow

Executor → ExecutionRepository::find_by_id()
         → Get action_id
         → Create ExecutionCompletedPayload {
             execution_id,
             action_id,              ✅
             action_ref,             ✅
             status,                 ✅
             result,                 ✅
             completed_at            ✅
           }
         → Publish to "execution.completed"
         → Queue: attune.execution.completed.queue
         → CompletionListener consumes
         → Deserializes ExecutionCompletedPayload ✅ SUCCESS

Benefits

1. Type Safety: Using canonical types prevents struct drift
2. Maintainability: Single source of truth for message schemas
3. Reliability: No deserialization errors from schema mismatches
4. Traceability: Messages include source metadata
5. Completeness: All required fields are populated from database

Files Modified

• attune/crates/worker/src/service.rs

  • Removed local ExecutionStatusPayload
  • Updated publish_status_update() to use canonical type
  • Added database query to fetch action_ref and previous_status

• attune/crates/executor/src/execution_manager.rs

  • Removed local ExecutionStatusPayload and ExecutionCompletedPayload
  • Updated process_status_change() to use ExecutionStatusChangedPayload
  • Updated publish_completion_notification() to use canonical ExecutionCompletedPayload
  • Removed attempt to read non-existent result field

Performance Considerations

Database Queries Added

• Worker: One extra query per status update to fetch action_ref

  • Impact: Minimal - status updates are infrequent (running → completed)
  • Benefit: Ensures message consistency

• Executor: No new queries (already fetching execution)

Alternative Considered

We could have passed action_ref through function parameters, but:

• ❌ Requires threading it through multiple layers
• ❌ Creates tight coupling between execution logic and messaging
• ✅ Database query is cleaner and more maintainable

Testing Verification

1. Compile Check: ✅ All services compile without errors
2. Type Safety: ✅ Rust compiler enforces canonical types
3. Field Validation: ✅ All required fields present in payloads

To Verify After Deployment:

# 1. Restart services (rebuild required)
make stop-executor stop-worker
cargo build --release --bin attune-executor --bin attune-worker
make run-executor run-worker

# 2. Clear old messages (optional but recommended)
rabbitmqadmin purge queue name=attune.execution.status.queue
rabbitmqadmin purge queue name=attune.execution.completed.queue

# 3. Run test execution
attune action execute core.echo --param message="test"

# 4. Monitor logs (should see NO errors)
grep "Failed to deserialize" logs/executor.log
grep "missing field" logs/executor.log

Lessons Learned

1. Always use canonical types: Never duplicate message struct definitions
2. Import from common: Use attune_common::mq::* for all message types
3. Database as source: Query database for complete data rather than passing through layers
4. Compiler is your friend: Rust's type system catches schema mismatches at compile time
5. Document message schemas: Keep attune_common::mq::messages.rs as the authoritative reference

Related Changes

This fix builds on the queue separation work done earlier today:

1. Queue Separation (2026-02-03): Created dedicated queues for each message type
2. Canonical Types (2026-02-03 - this fix): Unified message struct definitions

Together, these changes ensure:

• ✅ One queue per message type
• ✅ One consumer per queue
• ✅ One canonical struct per message type
• ✅ Zero deserialization errors

Deployment Notes

IMPORTANT: Both worker and executor must be rebuilt and restarted for this fix:

# Services need restart
- attune-executor (consumes status messages)
- attune-worker (publishes status messages)

# Services don't need restart (no changes)
- attune-api
- attune-sensor
- attune-notifier

Downtime: Brief (< 1 minute for service restart)

Risk: Low - Type-safe changes, compiler-verified

Status

✅ Complete - All services using canonical message types
✅ Tested - Compiles cleanly with release build
✅ Ready - Deploy by restarting executor and worker services