AI agents are gaining access to sensitive systems faster than the tools used to track their actions are improving. Recent incidents show the gap: hundreds of OpenAI agents reportedly escaped a cybersecurity test environment and hacked Hugging Face while looking for clues about their evaluation. Another OpenAI agent accessed private files on an Australian government website. Anthropic, Google and Meta have also disclosed agents going off course, sometimes without researchers or safety teams detecting it in time.
For organizations deploying agents, the central issue is operational: can they reconstruct what an agent did, which people authorized it, and what shaped its decisions? Current agent frameworks don’t consistently provide the records needed to answer those questions. Without them, real-time monitoring and later investigations both depend on incomplete evidence.
Agents need a record of their work
Traditional generative AI typically returns an output for a person to review or use. An agent can carry out a sequence of tasks: plan, select tools, take actions and delegate work to sub-agents. Its behavior can change as those steps unfold. Pre-deployment testing can catch some problems, but it can’t anticipate every path an agent might take once it’s operating in a live environment.
That makes observability a basic requirement for oversight. The term describes the ability to capture system activity, turn it into useful signals and present those signals to people or automated monitoring systems. For agents, that means establishing both what happened during a run and how the agent reached its decisions.
A Partnership on AI research project conducted with academic, civil society and industry colleagues examined four widely used agent frameworks. The researchers found missing or unreliable signals that make it difficult to spot policy breaches, unauthorized actions and failures in human oversight. Among the gaps: agent identifiers that persist over time, and records showing which people approved particular tasks.
The practical effect is that a company may struggle to determine which agent performed an action or whether anyone authorized it. A system can emit technical activity logs and still leave those questions unanswered if its records don’t reliably connect actions to a persistent agent identity and a human decision.
Two kinds of records, two different challenges
Useful monitoring requires at least two complementary record types. The first captures the events in an agent’s run: which agent acted, what a person approved or changed, and whether the agent’s permissions or memory shifted. The second aims to capture information about the agent’s decision process, including its reasoning and confidence in the action it took.
Consider a mortgage application. An agent reviews the file and recommends denying the loan; a loan officer approves that recommendation. A bank investigating the decision would need to identify the agent, determine which officer signed off and see how long the review took. It would also need to examine what information informed the recommendation, to check that the agent didn’t rely on prohibited factors such as the applicant’s neighborhood.
These records serve different purposes. Event logs can help establish who did what and when. Decision records could help auditors assess why the agent acted as it did. Neither can replace an organization’s own policies and permissions: telemetry can show activity, but it can’t decide whether that activity broke a rule unless the relevant rules are defined and applied.
Shared telemetry could close part of the gap
The missing event records appear, in large part, to be a software infrastructure problem. OpenTelemetry, an open observability standard maintained by engineers across the industry, offers a route to more consistent recording. A working group is drafting formats for tracing agent activity, but those formats haven’t been finalized.
The standards work should account for persistent agent identity, human interventions, and changes to permissions or memory. Without shared conventions for those signals, framework developers and enterprise customers may record the same activity in incompatible ways—or omit it altogether. That makes it harder for organizations to compare systems or piece together an incident across tools.
Persistent identifiers also need careful design. Connecting an agent’s actions over time can support accountability, but an identifier that links back to a real person can create privacy concerns. The standards process needs to consider that link as it defines how identities are recorded and used.
Capturing how an agent decided is less settled. Researchers and model providers still need to establish whether a model’s displayed step-by-step reasoning, often called chain of thought, accurately represents the process behind its decision. They also need reliable ways to measure how confident an agent is. Until those questions have better answers, organizations should be cautious about treating generated explanations or confidence scores as definitive evidence.
Responsibility sits across the deployment chain
No single company or sector can fill these gaps alone. Framework developers, including OpenAI, Anthropic, CrewAI and LangChain, can build telemetry into their products so that persistent identity, human interventions and permission or memory changes are recorded. Enterprises can use procurement and deployment decisions to press for those features, particularly in sectors such as banking and health care where agent actions may affect high-stakes decisions.
Organizations adopting agents also need to set clear roles, permissions and operating policies. A log can support an investigation, but it doesn’t define what an agent should be allowed to do. Businesses have to establish those boundaries and decide who can approve actions or change an agent’s access.
Standards bodies and regulators can specify which records high-stakes deployments should produce while leaving technical implementation details to industry. In the United States, NIST’s agent standards initiative is already working on agent identity, one of the shortcomings identified by the research.
The stakes rise as agents enter finance, government and medicine. The comparison to hiring employees with access to customer accounts is apt: organizations wouldn’t accept workers whose identities change daily, whose approvals are unclear and whose changes to access controls go unrecorded. Agent deployments need equivalent accountability built into their operating systems.
Better telemetry won’t guarantee safe behavior, and records alone can’t prevent every incident. But persistent identity, approval trails and well-designed event logs would give operators a firmer basis for intervention and review. As agents take on longer tasks and broader access, that evidence needs to be part of deployment from the start.
Madhulika Srikumar is Head of the AI Safety Program at the Partnership on AI. Claire Leibowicz is Director of AI, Trust, and Society at the Partnership on AI and a doctoral candidate at Oxford University.