Get started
How-to

Verify your Agent

Confirm Agent Runtime health, Communication Connection health, provider delivery, Connection-scoped Conversations, and Tool Call telemetry as separate operational checks.

For
Agent operators, Organization owners, Organization administrators, and support engineers
On this page
  1. Chat in the dashboard
  2. What you will verify
  3. The verification layers
  4. Before you begin
  5. Verification plan
  6. 1. Verify the Agent configuration
  7. 2. Verify the lifecycle state
  8. 3. Verify Runtime health and logs
  9. 4. Verify the Communication Connection
  10. 5. Verify one allowed interaction
  11. 6. Verify one blocked interaction
  12. 7. Verify the Conversation
  13. 8. Verify Tool Calls separately
  14. 9. Confirm Agent Access
  15. 10. Review cost attribution
  16. Where to look when a layer fails
  17. Acceptance checklist
  18. Troubleshooting
  19. Next steps

One successful reply is the beginning of verification, not the end. It proves that several layers worked together at once, and it hides which of them is fragile.

This guide checks the Agent Runtime, its Communication Connections, provider delivery, Conversation history, and Tool Call telemetry as separate operational paths, so a failure points at one layer instead of the whole system.

What you will verify

By the end of this guide, you will have confirmed that:

  • The Agent has the intended Runtime and pinned configuration
  • The Agent Runtime starts and reports healthy
  • The intended Communication Connection is enabled and healthy
  • One allowed interaction succeeds and one blocked interaction does not
  • The Conversation appears under the correct Connection
  • Tool Call telemetry and cost attribution are checked separately

The verification layers

Agent verification has distinct layers, and each one has its own source of truth.

# Layer What it proves
1 Agent configuration The Agent is the one you intended to build
2 Agent lifecycle and Runtime health Runtime resources were created and the Runtime is reachable
3 Communication Connection configuration and provider health A provider session or webhook path is configured and usable
4 Inbound and outbound communication delivery Provider messages reach the Agent and replies reach the provider
5 Connection-scoped Conversation persistence The exchange is recorded under the correct Connection and location
6 Tool Call telemetry The Runtime reported tool execution through Ingest
7 Cost attribution Model usage is reported through LiteLLM, when configured

These layers can succeed or fail independently. For example:

  • The Agent Runtime can be running while one Connection is disconnected.
  • A provider message can be accepted and persisted even if Runtime processing later fails.
  • A reply can fail provider delivery after the Runtime generated it.
  • An Agent can respond normally even when a request does not produce any Tool Calls.
  • A headless Agent with no Connections can still have a healthy Runtime.
  • A healthy Product API does not prove that Ingest, Communications, a provider Connection, or an Agent Runtime is healthy.

Before you begin

You need:

  • An active Organization
  • Access to the Agent
  • A hired Agent with a selected Runtime and pinned Template Version
  • A successfully started Agent, unless you are diagnosing startup
  • At least one Communication Connection, when provider messaging is being tested
  • A controlled provider location where allowed and blocked messages can be sent
  • Permission to read Agent configuration, Activity, health, and logs
  • Permission to inspect Connection health
  • A non-sensitive test prompt

You do not need a Platform to verify that the Runtime itself starts successfully. Steps 1 through 3 apply to a headless Agent.

The Agent Creator receives explicit Agent Owner access and can perform every check in this guide. Other users see fewer controls, depending on their effective Agent Access.

Required permissions

Verification task Required authority
Open the Agent and review its configurationAgent read access
View health, Conversations, Tool Calls, or logsactivity.read
Start or pause the AgentAgent lifecycle permission
Review or change a Communication ConnectionAgent update permission
Replace Connection or Integration credentialsAgent secret-management permission
Review or change sharingAgent access-management permission
View Organization-wide costsOrganization Owner or Organization Administrator

Verification plan

Work through the layers in order. Each check has its own pass condition.

Check Requirement Pass condition
Agent configuration Required Runtime, pinned Template Version, Skill Versions, model, and Agent Access match your intent
Lifecycle state Required The Agent reaches the running state without an unresolved error
Runtime health and logs Required Runtime health is reachable and startup logs show no unexplained failure
Connection configuration and health Conditional The intended Connection is enabled and shows no unresolved provider error
Allowed interaction Conditional An allowed message receives a reply through the same Connection
Blocked interaction Conditional A deliberately blocked message receives no reply
Conversation persistence Conditional Inbound and outbound messages appear under the expected Connection and location
Tool Call telemetry Conditional A controlled tool request produces the expected Tool Call
Agent Access Required General access and direct assignments match the intended audience
Cost attribution Optional Model usage and cost appear under the correct Agent

The conditional checks apply once the Agent has at least one Communication Connection. If a required check fails, investigate it before adding more Skills, credentials, Connections, or users.

Verify the Agent configuration

Review the configuration before testing anything operational. Open the Agent, then confirm:

  • The intended Organization owns the Agent
  • The intended Hermes or OpenClaw Runtime is selected
  • The intended Template and exact Template Version are pinned
  • Required Skill Versions are assigned
  • Required tool Integration credentials are configured
  • Agent Access is appropriate
  • Agent General Access remains Restricted, unless broader Organization access was intentionally configured
  • The model follows the Organization default or pins the intended explicit model

Make sure you understand the effective model: the model the Agent resolves to right now. If the Agent is already running, the displayed running model may differ from the currently effective model until the Agent restarts.

If anything here is wrong, correct it in Configuration before continuing. See Hire your first Agent for how these choices were made.

Verify the lifecycle state

Inspect the Agent's lifecycle state on its own, before considering communication. Agent Barn persists three states.

State Meaning
STOPPEDNo active Runtime is expected
RUNNINGRuntime resources were created successfully
ERRORThe latest Runtime lifecycle operation failed

The interface presents these as operator-facing labels such as Idle, Initializing, Working, Disconnected, and Needs attention. The underlying meanings above do not change.

How an Agent arrives at a state:

  • Agent creation first persists a headless STOPPED Agent.
  • The web hiring flow may issue a separate start operation immediately afterward.
  • Starting renders the pinned configuration and creates Runtime Kubernetes resources.

If the Agent is stopped and you hold lifecycle permission, select Start. See Manage the Agent lifecycle.

Verify Runtime health and logs

This check applies whether or not the Agent has any Connections.

  1. Confirm the Agent is in the active running state.
  2. Open the Agent health surface.
  3. Confirm the Runtime health endpoint is reachable.
  4. Review recent Runtime logs for startup or execution errors.
  5. Confirm the configured Runtime image was pulled.
  6. Confirm Kubernetes created the expected Deployment, Service, Secret, ConfigMap, and PVC, where those details are exposed.
  7. Confirm the effective configuration rendered without missing Template, Skill, model, or Integration requirements.

Expected: Runtime health is reachable and the startup sequence completes without an unexplained failure.

Runtime logs primarily diagnose:

  • Template rendering
  • Skill materialization
  • Integration configuration
  • Model access
  • Kubernetes startup
  • Runtime execution
  • Local Runtime request processing

Runtime logs are not the canonical source for provider Connection health or Communication Delivery history. Use the Connection surface for those. See Review Agent health and logs.

Optional Kubernetes check

Self-hosted operators can confirm that Agent resources exist in the namespace:

Shell
kubectl get pods \
  --namespace agent-farm \
  --selector agentbarn.io/component=agent

For staging, use the configured staging namespace, commonly agent-farm-staging. The relevant Agent pod should be running and ready.

Verify the Communication Connection

When provider messaging is being tested, inspect the specific Connection rather than the Agent as a whole. Confirm:

  • The intended Connection belongs to this Agent
  • The intended Platform Plugin is selected
  • The Connection is enabled
  • Provider credentials were accepted
  • The provider application or bot is installed in the intended location
  • The Connection's routing and admission settings allow the intended location and user
  • The Connection does not show an unresolved provider health error
  • The provider-side scopes, permissions, events, intents, channel configuration, or webhook are complete

Connection health is independent of Agent lifecycle. A running Agent can hold one healthy Connection and one failing Connection at the same time, and each is diagnosed on its own.

For provider-side completeness, use the guide for your Platform: Slack, Microsoft Teams, Telegram, or Discord.

Verify one allowed interaction

Use a controlled provider location.

  1. Confirm the Agent Runtime is running.
  2. Confirm the Connection is enabled.
  3. Send a message from an allowed user in an allowed location.
  4. Include an explicit mention when required by the Connection policy.
  5. Wait for the Agent's response.
  6. Confirm the reply arrives through the same provider application and Connection.

Use a deterministic prompt:

Verification prompt
@agent-bot Respond with exactly: verification-ok

Replace @agent-bot with the provider-side handle of the application attached to this Connection.

Expected: the Agent returns verification-ok in the same location, through the same Connection.

Admission behavior differs by Platform and by Connection policy:

  • Shared spaces may require an explicit mention.
  • Direct messages may be off, open, or allowlisted.
  • Slack thread behavior depends on the Connection's thread mention policy.
  • Discord server messages may be narrowed by server, channel, user, role, and mention settings.
  • Microsoft Teams and Telegram retain their provider-specific admission rules.

Record the Connection, the location, the approximate send time, and the expected response. Minor formatting differences do not indicate a delivery failure; what matters is that the correct Agent produced one relevant response on the expected Connection.

Verify one blocked interaction

A reachable Agent must also ignore what its policy excludes. Test one deliberately blocked case that applies to this Connection, such as:

  • A location outside the allowlist
  • A user outside the user or role policy
  • A direct message when direct messages are off
  • An unmentioned shared-space message when mention gating applies

For the mention case, send a new message in the shared location without mentioning the Agent:

Blocked test message
verification-no-mention

Expected: the Agent does not respond.

What should happen behind that silence:

  • A policy-rejected provider payload does not create a canonical inbound Conversation Message.
  • Operational diagnostics may record a content-free policy disposition.

Verify the Conversation under the correct Connection

Return to the Agent and check that the exchange was recorded where you expect it.

  1. Open Agent Activity.
  2. Open Conversations.
  3. Select the expected Communication Connection and provider location.
  4. Confirm the allowed inbound message appears.
  5. Confirm the outbound Agent response appears in the same conversation.
  6. Confirm sender and location names, where provider enrichment is available.
  7. Confirm the blocked test did not become a canonical Conversation Message.

How Conversation identity works:

  • The Communications Gateway writes canonical Conversation Messages.
  • Conversation identity includes both connection_id and provider channel_id.
  • Two Connections can safely use the same provider channel identifier.
  • Replies remain bound to the source Connection.
  • Conversation persistence is not written through Ingest.

If the reply arrived in the provider but no Conversation appears, treat it as a Communications or permission question, not a Runtime telemetry question. See Review Agent activity.

Verify Tool Calls separately

Tool Calls follow a different path from Conversations.

  • Hermes or OpenClaw reports Tool Call telemetry through Ingest.
  • Ingest authenticates with Agent identity and a per-start Ingest key.
  • Tool Calls use Runtime invocation identity for correlation.
  • A request that invokes no tool is not expected to create a Tool Call.
  • A successful chat reply does not guarantee that a Tool Call should exist.

So No tool calls yet can be the correct result for a first verification prompt. To verify the path itself, send a controlled request that is expected to invoke one configured tool, then open Tool calls.

Prefer a read-only request. Avoid creating, updating, deleting, sending, or publishing external data during initial verification: for example, ask a repository-enabled Agent to list an allowed repository rather than to modify it.

Status Meaning Verification action
PENDINGThe Runtime reported that the call beganWait for its result
SUCCESSThe result completed successfullyConfirm the expected tool and scope were used
ERRORThe call failedExpand the row and inspect its arguments and result

Confirm that:

  • The Tool Call appears under the correct Agent
  • The tool name is expected
  • The status reaches the expected terminal state
  • The result is correlated to the intended invocation
  • No unexpected write operation occurred

Review a failure as an Integration or Runtime problem first, rather than automatically as a Connection problem. Treat Tool Call arguments and results as potentially sensitive operational information.

Confirm Agent Access

Two access surfaces exist, and verifying one says nothing about the other:

  • A Communication Connection's admission policy determines who can talk to the Agent through a provider.
  • Agent Access determines who can open and operate the Agent inside Agent Barn.

If you have access-management permission, open Share on the Agent page and confirm that General access matches the intended policy. New Agents default to:

Default General access
Restricted

With Restricted access, only users with direct Agent Access can open the Agent. Organization Owners and Organization Administrators retain implicit full authority over every Agent, and they are not listed as direct assignments.

Role Intended authority
Agent ViewerRead Agent information, Conversations, Tool Calls, logs, and Agent-specific costs
Agent EditorViewer authority, plus configuration, lifecycle, Skills, credentials, and Connections
Agent OwnerEditor authority, plus deletion and Agent access management

The Agent Creator should appear with explicit Agent Owner access. For an observer helping with verification, Agent Viewer is normally sufficient.

Review cost attribution

Optional This check applies only when LiteLLM cost reporting is configured.

Organization Owners and Organization Administrators can open Costs from the Organization navigation. Under the Agent breakdown, locate the verified Agent and confirm the row attributes the correct Agent, model, tokens, and total cost.

Cost reporting follows a separate path:

Cost attribution path
Agent model request
        ↓
LiteLLM
        ↓
LiteLLM key identity
        ↓
Agent Barn cost attribution
  • Costs come from LiteLLM reporting.
  • Costs are attributed through the Agent's LiteLLM key identity.
  • Conversation Messages and Tool Calls do not calculate cost.
  • A Conversation can exist before cost reporting appears.
  • Tool Call state does not prove that LiteLLM cost ingestion is healthy.

See Review costs for the full reporting model.

Where to look when a layer fails

Match the symptom to its layer before changing anything.

Symptom Most likely layer First place to inspect
Agent cannot start Runtime lifecycle Agent state and Runtime logs
Agent runs but Connection is disconnected Communications provider session Connection health and setup
Allowed provider message never appears Provider ingress or admission Connection policy and diagnostics
Inbound Conversation appears but no Runtime response Runtime processing Agent health and Runtime logs
Runtime response exists but provider reply is missing Outbound Communication Delivery Connection delivery diagnostics
Conversation works but Tool Calls are empty Request did not invoke a tool, or Ingest issue Tool Call view and Ingest path
Tool Call fails Runtime tool or Integration Tool Call result and Integration configuration
Activity works but costs are missing LiteLLM reporting Cost surface and LiteLLM configuration
Another Member cannot inspect the Agent Agent Access Effective Agent Access and Permissions

Connection diagnosis and Runtime diagnosis stay separate. A provider problem is not fixed by restarting the Agent, and a Runtime problem is not fixed by re-entering provider credentials.

Acceptance checklist

The Agent is ready for further controlled use when you can confirm each of these.

Runtime and configuration

  • The Agent has the intended Runtime and pinned configuration
  • The Agent Runtime starts successfully
  • Runtime health and logs show no unexplained failure
  • A headless Agent is understood as valid
  • Agent Access allows the intended Members to inspect the result

Communication

  • The intended Communication Connection exists and is enabled
  • Connection health is checked separately from Agent health
  • One allowed provider interaction succeeds
  • One blocked provider interaction receives no response
  • The allowed Conversation appears under the correct Connection

Telemetry and cost

  • A controlled tool request produces the expected Tool Call, when applicable
  • Cost attribution is checked separately, when applicable

Troubleshooting

The Agent will not start

Runtime lifecycle, not communication

Review the lifecycle state and Runtime logs, then check:

  • Model availability
  • Runtime image access
  • Kubernetes capacity, scheduling, and volumes
  • Template rendering
  • Skill materialization
  • Integration credentials

For a self-hosted installation:

Shell
kubectl get pods \
  --namespace agent-farm \
  --selector agentbarn.io/component=agent

Correct the cause, then start the Agent again. A successful start clears the previous error.

The Agent runs but a Connection is disconnected

Provider session, not Runtime

Review Connection health and the Platform Plugin's setup guidance, then confirm:

  • The Connection is enabled
  • Provider credentials are still valid
  • The provider application or bot is enabled on the provider side
  • The relevant supervised session or webhook path is available

Do not restart the Agent unless there is a separate Runtime problem.

An allowed message receives no response

Separate admission from Runtime processing

First confirm the message should have been admitted:

  • The provider application is installed in that location
  • The location, user, and role satisfy the Connection policy
  • The direct-message policy allows the test, if you used a DM
  • The message satisfies the Connection's mention policy

Then check whether the inbound Conversation Message was persisted. If it was, the admission layer worked and the question moves to Runtime processing: review Agent health and Runtime logs around the send time. If it was not, stay in the Connection layer.

The Runtime replied but the provider shows nothing

Outbound Communication Delivery

Confirm the Runtime is running, then review Connection delivery diagnostics and delivery state. Confirm outbound provider permissions and that the source Connection is still enabled.

This is a delivery problem, not a reason to change the Agent's Template, model, or Skills.

The Agent responds to a message it should have ignored

Connection admission policy

Confirm the test was constructed correctly: a new message in a shared location rather than a direct reply, sent by the intended user, in the intended location.

Then review the Connection's routing and admission settings, including allowlists, user and role policy, direct-message policy, and mention policy. Saving the Connection reconciles the provider session; no Agent restart is required.

The Agent replied, but Conversations look empty

Check permission and the selected Connection

Reload the Agent page after the response completes, then confirm:

  • Your account holds activity.read
  • You selected the Connection and provider location used for the test
  • The exchange belongs to this Agent

Conversation history is Connection-scoped, so an exchange on one Connection does not appear under another.

Tool Calls are empty or a Tool Call fails

Runtime telemetry and Integrations

An empty view is expected when the request invoked no tool. Send a request that should invoke one configured tool before treating this as a fault.

For a failing or stuck Tool Call, expand it and review the tool name, arguments, result, associated logs, assigned Skill Version, Integration credential, provider-side permissions, and allowed resource scope. Repeat a test only after confirming it cannot create a duplicate external action.

The Agent does not appear under Costs

LiteLLM reporting is separate

Confirm that:

  • You are an Organization Owner or Organization Administrator
  • LiteLLM is configured
  • The Agent has a per-Agent LiteLLM key
  • The Agent completed a model request
  • The selected date range contains the request

Successful Conversations and Tool Calls do not guarantee that LiteLLM cost reporting is configured.

Another Member cannot inspect the Agent

Agent Access

Agent General Access defaults to Restricted, so Members do not gain access automatically. Review their effective Agent Access and Permissions, then assign explicit Agent Access or deliberately change General Access.

Next steps

After verification:

Chat in the dashboard

For an initial conversation without external provider setup, use the experimental Chat tab once the Agent is Running and Working. See Chat with an Agent in the dashboard for permissions and thread behavior. External Connections can be configured separately.

Documentation