Testing and Verification

The OpenClaw builder and shared Communications runtime adapter remain image-workflow selection gaps. API paths still select the normal API workflow; “neither” does not mean no CI runs. Automatic selection also does not determine which runtime contracts a change needs. Follow the change's actual boundary and the repository's verification guidance.

The image contracts include runtime-facing behavior beyond telemetry: Hermes scheduler/boot-related integration and messaging capture, and OpenClaw completion/tool hooks have dedicated source and fixtures. A successful generic API suite is not proof that the pinned runtime image accepts the generated configuration or invokes its hooks.

hermes-base/: Hermes.

api/domains/agents/builders/hermes.py: Hermes.

api/domains/agents/scripts/hermes/: Hermes.

api/runtime_tests/: Hermes.

Under api/tests/fixtures/: hermes_pvc_permissions_driver.py, hermes_session_store_driver.py, or hermes_message_completion_driver.py: Hermes.

openclaw-base/: OpenClaw.

api/domains/agents/scripts/openclaw/: OpenClaw.

api/tests/fixtures/openclaw_message_hooks_driver.mjs: OpenClaw.

api/domains/agents/scripts/messaging/: Both.

api/domains/agents/builders/openclaw.py alone: Neither image workflow.

api/domains/agents/scripts/communications-runtime-adapter.py alone: Neither image workflow.

Every user-facing route must test the authorization boundary appropriate to its resource. Agent-subordinate Communication Connections, Conversations, Tool Calls, Secrets, configuration, logs, costs, and Agent-private Skills must prove Agent Access as well as Organization isolation.

Connection coverage proves that one Agent may own multiple Connections, including Connections for the same Platform; credentials remain isolated to their owning Connection; plugin-defined duplicate provider identities are rejected; retirement releases credential identity; Conversation and durable thread state retain connection_id; and diagnostics and journal responses remain content- and credential-safe. See api/tests/integration/test_communication_connections.py, api/tests/integration/test_communication_deliveries.py, and api/tests/integration/test_conversations.py.

Missing authentication: 401; missing, cross-Organization, or inaccessible Agent resource: 404.

Visible resource without required action Permission: 403; stale revision or incompatible lifecycle: 409 where defined.

Invalid schema or business input returns the appropriate validation response; list, count, search, and pagination exclude inaccessible resources before totals.

Gateway, processor, and supervisor tests prove lifecycle around a Platform Plugin: denied events create no Delivery; accepted inbound events are idempotent; enrichment or journal failure does not discard valid provider traffic; one ingress lease is active per Connection; supervisor failures isolate by Connection; reconnect backoff is bounded; retries do not duplicate provider-visible replies; and terminal failures remain distinct from retryable failures. Use api/tests/unit/test_communications_gateway.py, test_communications_processor.py, test_communications_supervisor.py, and test_communications_operations.py.

Hermes and OpenClaw use the shared Communications Runtime adapter. Test the protocol once at that seam, then prove both builders mount and launch it with required environment, bounded idle backoff and reset after work, Delivery claim/reply/completion paths, stable Connection and thread session identity, Delivery-ID idempotency, supported protocol-version headers, and explicit successful or failed completion. Use api/tests/unit/test_communications_runtime_adapter.py, test_hermes_builders.py, test_openclaw_builders.py, and api/tests/integration/test_agents.py. Use a pinned Runtime image only when the actual Runtime API or packaged contents are the contract.

Ingest receives Tool Call and Tool Result telemetry only. Validate tool_calls and tool_results against real IngestBatchRequest models. Conversation Messages are written by Communications and must not appear in Ingest payload assertions.

Bundled Skill coverage verifies one root SKILL.md, isolated aai-<integration> directories, valid relative references, non-empty files, deterministic manifests, provider metadata, collision detection, predefined Template references, missing-only bootstrap publishing, exact Agent and Template pins, and correct Hermes and OpenClaw mount roots. Use api/tests/unit/test_aai_cli_skills.py, test_template_skill_paths.py, api/tests/integration/test_skills.py, test_templates.py, and test_agents.py.

Communication Connection UI coverage exercises platform discovery, schema-driven forms, credential replacement, diagnostics, journals, reconnects, retries, and permission-aware controls. Skill coverage preserves Platform, Organization, and Agent route scopes. Browser mocks prove UI behavior; backend integration tests remain the tenant-isolation and authorization proof. Representative support files are ui/tests/pages/communication-connection-detail-page.po.ts, ui/tests/pages/data-support/communication-connection-data-support.po.ts, ui/tests/pages/data-support/skill-data-support.po.ts, and ui/tests/fixtures/communication-connections.ts.

Run the smallest complete set for the touched contract and widen only when behavior depends on a larger environment. Documentation changes require relevant documentation checks. Report unrelated failures exactly and do not alter unrelated code to make checks pass.