Agent Communication & Observability

Operate · Assess

Agent Communication & Observability: Why Silent Failures Are Dangerous

The AI agent teams behind bernd-zuther.de and ketovavaria.de coordinate through two channels. A fast messaging channel carries the ongoing dialogue, and a shared exchange directory on the file system holds durable handoffs. This works well day to day. One observation from operations showed where the weak spot is, though: a messaging channel can fail without anyone noticing.

What I use this for

  • Coordination between autonomous AI agent teams across project boundaries
  • A fast messaging channel for the ongoing dialogue between agents
  • A file-system exchange directory as a second, traceable channel
  • A real-world example of why observability matters in multi-agent systems

Two Channels for Agent Communication

When several AI agents work on different websites and tasks, they need to exchange information: handoffs, follow-up questions and status updates. The agent teams behind bernd-zuther.de and ketovavaria.de use two routes for this. Messages go directly between the teams through a messaging channel. Alongside it, a shared exchange directory on the file system stores handoffs as files. The specific services behind these channels are an implementation detail and don’t matter for the topic of this page.

The two channels have different strengths. The messaging channel is fast and suits the ongoing dialogue. The exchange directory is slower but durable: anything stored there can be read and traced later. Having both routes in parallel was convenient day to day. How important that is only became clear after an observation in operations.

The Observation: A Channel That Simply Goes Quiet

During regular operations, we noticed that a messaging channel had stopped delivering messages for a while. There was no error message, no alert and no crash. Work simply carried on as if everything were fine.

That is exactly what makes this observation instructive. The channel didn’t fail loudly. It just went quiet. An agent that receives no message cannot tell whether there was nothing to say or whether the channel is dead.

Loud and Silent Failures

Broadly, failures fall into two groups. Loud failures show up immediately: a process crashes, an error message appears, a service becomes unreachable. They are annoying but honest, because you know right away that something needs to be done.

Silent failures are more insidious. The system appears to run normally, and only an expected signal fails to arrive. Such a failure goes unnoticed until it causes consequences elsewhere, and by then the root cause is often hard to find. For a system in which autonomous agents rely on feedback from one another, this is especially dangerous. No human is sitting nearby wondering why things are so quiet. The agents keep working on assumptions that stopped being true a while ago.

Why Observability Is Critical in Multi-Agent Systems

Traditional monitoring often only checks whether a service is running. That isn’t enough for a silent channel, because the service can technically be running and still deliver nothing. Here, observability mainly means that the absence of an expected signal must itself become a signal. If you only wait for error messages, you miss exactly the failures that don’t produce any.

The more autonomously agents work, the more this matters. Once humans no longer check every handoff themselves, the system has to make it visible whether communication between agents is still working.

Redundancy Kept the Collaboration Going

Collaboration continued despite the disruption because of the second channel. The exchange directory on the file system remained traceable even while the messaging channel was silent. The handoffs stored there were still present and readable. The lesson is simple: a second, independent communication route doesn’t prevent a disruption, but it keeps a disruption from bringing everything to a halt.

Current Status: Assess, Not Adopt

Redundancy alone doesn’t solve the underlying problem, since the disruption was only noticed by chance. That’s why I’m working on making a missing signal visible on its own, rather than relying on someone noticing it eventually. Until that works reliably, agent communication stays at Assess on the technology radar and is not yet used as a dependable building block.

How This Differs from MCP

The Model Context Protocol (MCP) and this messaging channel solve different problems, even though both involve AI agents. MCP standardizes how a single agent accesses tools and data sources in a structured way: a client-server connection with defined tool interfaces, where an agent calls a tool and gets a structured result back. It is the layer between an individual agent and its capabilities.

This page covers something else: coordination between multiple separate, autonomous agent teams that work on different projects and send each other handoffs, follow-up questions and status updates. That happens asynchronously, not as a single tool call but as an ongoing dialogue between teams. I would use MCP to let an individual agent access a database or a search index, for example. The messaging channel described here, together with the file-system exchange directory, handles coordination one level above that: team-to-team rather than agent-to-tool. The two layers complement each other, but neither replaces the other.

Sources

  • My own observation of the agent operations behind bernd-zuther.de and ketovavaria.de