Develop and extend
Concept

Work with domain events

Model immutable business facts, stage them transactionally with business state, deliver them through idempotent handlers, and diagnose delivery failures.

For
Backend and platform engineers
On this page
  1. Overview
  2. Distinguish the concepts
  3. Use Domain Events for committed business facts
  4. Use the registered event catalogue
  5. Domain Events are not the Communications journal
  6. Produce and deliver events transactionally
  7. Handler instances are shared across worker threads
  8. Use the Event Delivery Monitor correctly
  9. Source map
  10. Related guides
  11. Activity and Web Chat boundaries

Domain Events are immutable, typed business facts produced transactionally with business state. They are separate from Conversation activity, Tool Call telemetry, and Communications operational diagnostics.

Keep the three pipelines separate

Text
Conversation activity              Tool Call telemetry              Internal Domain Events
        │                                  │                                  │
        ▼                                  ▼                                  ▼
Provider or runtime reply          Hermes or OpenClaw              Business mutation
        │                                  │                                  │
        ▼                                  ▼                                  ▼
Platform Plugin /                  Ingest API                      Domain repository
Communications Gateway             /ingest/v1                      transaction
        │                                  │                                  │
        ▼                                  ▼                                  ▼
Conversation Message               Tool Call record                Business state
and Communication Delivery                                             +
                                                                    Outbox Message
                                                                        +
                                                                    Event Deliveries

Runtime Telemetry Events are emitted by Hermes or OpenClaw through the separately served Ingest API. They currently produce Tool Call records only. They are not Domain Events and are never written to the Domain Event outbox.

Conversation Messages enter through the Communications Gateway. Shipped Platform Plugins normalize provider events, Communications persists canonical Conversation Messages and Communication Deliveries, and Runtime replies return through the same Connection. This flow does not use Runtime Ingest or the Domain Event outbox.

  • Communications owns Conversation writes.
  • Ingest owns Tool Call telemetry writes.
  • Domain-specific repositories own transactional Domain Event production.
  • Neither the Conversation pipeline nor Tool Call telemetry automatically produces Domain Events.

Distinguish the concepts

RecordPurposeOwnerDelivery or retention behavior
Domain EventImmutable typed business factProducing business domain and Events domainPersisted in the outbox and delivered to registered handlers
Outbox MessageDurable transport-neutral record of one Domain EventEvents domainImmutable
Event DeliveryMutable delivery state for one event and handlerEvents domainAt-least-once handler delivery through Dramatiq
Security Audit RecordDeletion-independent compliance projectionEvents domainCreated from selected Domain Events
Communication DeliveryDurable inbound or outbound message-processing stateCommunications domainClaimed, completed, retried, or dead-lettered through the Communications protocol
Communications operation journalContent-free Connection and Delivery diagnostics timelineCommunications domainAppend-only operational history with bounded retention
Conversation MessageCanonical user/Agent messageCommunications and Conversations domainsPersisted by Communications from provider ingress, Web Chat delivery, and Runtime replies
Tool CallRuntime tool-execution activityIngest and Tool Calls domainsPersisted from authenticated Runtime telemetry

Use Domain Events for committed business facts

Use them when a committed business fact needs an asynchronous projection, notification, or security-audit record that must survive the originating request and commit atomically with the business state.

Do not use Domain Events for provider message ingress, every Communication Delivery transition, a Connection diagnostics timeline, Conversation Messages, Tool Call telemetry, the Communications operation journal, a public webhook, or a general activity log.

Use the registered event catalogue

Text
Organization and Agent access
organization.role.changed
agent.access.granted
agent.access.revoked
agent.general_access.changed

Agent lifecycle and configuration
agent.created
agent.started
agent.stopped
agent.updated
agent.deleted

Agent Template Overrides
agent.template_override.draft_saved
agent.template_override.published
agent.template_override.selected

Agent Secrets
agent.secret.added
agent.secret.updated
agent.secret.removed

Templates
template.created
template.updated
template.deleted

Organization governance and Agent Settings
organization.model_allowlist.changed
organization.agent_settings.changed
organization.member.added
organization.member.removed
organization.ownership_transferred

Platform authority
platform.user_privilege.granted
platform.user_privilege.revoked

Communications audit events
communication.connection.health.changed
communication.connection.reconnect.requested
communication.delivery.dead_lettered
communication.delivery.retry.requested
communication.delivery.recovered

agent.template_override.draft_saved records creation or saving of an Agent-owned Override Draft; published records an immutable Override Version; and selected records selection of a shared Template or Agent-owned Override Version.

Agent Secret events carry safe metadata only: record_id, provider, label, and shared_reference_id. They never include plaintext or encrypted credential content. Template events currently represent Organization Template mutations and carry bounded tracked-field changes, not complete Markdown artifacts.

organization.agent_settings.changed is emitted when one Organization Agent Setting changes. It carries the setting name, previous and current scalar values, the count of inheriting Agents, and safe actor/subject display snapshots. It is not emitted for an unchanged save. Model allowlist events use an added/removed diff, not complete before-and-after lists.

Platform privilege events are Platform-scoped, have no Organization ID, and cannot use a Membership actor.

Domain Events are not the Communications journal

Communications Domain Events carry scoped Organization, Agent, Connection, and Delivery identifiers, lifecycle state, attempt metadata, and safe actor/subject display snapshots. They never contain message text, sender identity, provider payloads, credentials, authorization headers, provider URLs, response bodies, or raw exception text.

For communication.connection.health.changed, the content-free diagnostic envelope may include category, operation, http_status, provider_code, retryable, retry_after_seconds, and request_id. Every field is bounded and safe for operational display.

The Communications operation journal is an append-only, content-free operational timeline for one Communication Connection and its Deliveries. It records high-frequency pipeline and health transitions needed for Agent-scoped diagnostics. It is not an Outbox Message stream, does not create Event Deliveries, does not invoke Event Handlers, and is not transported through Dramatiq.

Text
provider_observed
policy_admitted
policy_rejected
queued
agent_claimed
model_completed
reply_queued
provider_delivery_attempted
provider_delivered
connection_connecting
connection_connected
connection_degraded
connection_error
reconnect_requested
retry_requested
dead_lettered
recovered

Journal rows can record intermediate attempts and durations without implying a security-relevant business event. Retention is bounded; responses exclude message content, provider payloads, credentials, and sender identity. Connection diagnostics and per-Delivery timelines read from the journal, which is not shown by the Platform Event Delivery Monitor. Only selected audit-worthy Communications transitions also produce Domain Events.

Communications occurrenceJournal rowDomain Event
Provider payload observedYesNo
Admission accepted or rejectedYesNo
Runtime claims a DeliveryYesNo
Provider delivery attemptedYesNo
Connection health changesYesYes
User requests reconnectYesYes
Delivery becomes dead-letteredYesYes
User requests eligible retryYesYes
Retried Delivery succeedsYesYes

Produce and deliver events transactionally

The domain-specific repository owns one session and one commit. Business state, the Outbox Message, and intended Event Deliveries commit atomically. Routes never receive sessions or stage events; the session-aware outbox stages into the existing repository transaction. Services enqueue committed Delivery IDs after commit, and enqueue failure leaves the committed Delivery PENDING for reconciliation. Event Handlers must be idempotent.

agent.created is intentionally handlerless: it creates an Outbox Message but no Event Deliveries. agent.started and agent.stopped target agent.lifecycle_email.notification. Platform privilege, RBAC, Agent configuration, Template, Organization, Agent Settings, Template Override, Agent Secret, and Communications audit events target security_audit.projection. Adding a handler affects future events only; it does not backfill existing Outbox Messages.

Handler instances are shared across worker threads

Each worker process lazily creates and reuses one injector. Its registered handler instances are shared across that process's worker threads. A handler does not receive a fresh instance for each Event Delivery, and this process-local singleton is not one global instance across every worker process or replica.

Keep per-delivery values inside the handler call. Do not assign a current event, Organization, recipient, delivery result, or mutable working buffer to the handler instance. Open and close database sessions inside each call instead of keeping a session on the instance. Shared dependencies must be safe for concurrent use.

Thread safety and idempotency solve different problems. Concurrent calls must not overwrite one another's state; a retried delivery must not duplicate a committed side effect. Continue using the event or delivery/handler identity appropriate to the side effect as its idempotency key. The handler's transaction remains separate from the framework's Delivery lifecycle updates.

Use the Event Delivery Monitor correctly

HTTP
GET /api/v1/platform/event-deliveries/summary
GET /api/v1/platform/event-deliveries
GET /api/v1/platform/event-deliveries/event-types

The monitor is Platform Administrator-only and read-only. It monitors Domain Event Deliveries, not Communication Deliveries or the Communications operation journal. Handlerless events such as agent.created produce no Event Deliveries and do not appear; the event-type endpoint includes only definitions with at least one intended handler. It offers no replay, retry, remapping, or deletion action.

Responses expose operational identity, status, timing, attempt count, dead-letter reason, bounded/redacted error information, and derived status age. Expanded rows may expose curated actor_display and subject_display strings when the validated payload provides them. They do not expose raw Actor or Subject identity objects, the complete payload, correlation ID, or causation ID.

Source map

ConcernSource
Current event names and payloadsapi/domains/events/catalog.py
Runtime telemetry boundarydocs/features/activity-and-ingest.md, api/domains/ingest/
Communications operation journalapi/domains/communications/operations.py
Communications Delivery persistenceapi/domains/communications/delivery_repository.py
Communications Domain Event productionapi/domains/communications/repository.py, api/domains/communications/delivery_repository.py
Connection diagnostics APIapi/domains/communications/routes.py, api/domains/communications/service.py

Activity and Web Chat boundaries

Runtime Ingest accepts Tool Call and Tool Result telemetry. Communications persists canonical inbound and outbound Conversation Messages. Product API routes expose authorized reads. Domain Events are separate typed business facts delivered through the transactional outbox and worker system; runtime telemetry is not automatically a Domain Event.

Web Chat uses the Communications delivery pipeline; its user-scoped Chat history and SSE surface belong to api/domains/web_chat/. This does not make each runtime telemetry frame or chat update a Domain Event. See Dashboard Web Chat.

Documentation