Agents can learn with Memory in Microsoft Foundry Agent Service

Recurring Failure

We are entrusting AI agents with an increasing range of tasks, from preparing briefs to scheduling meetings, and from filing tax returns to building and debugging production systems. While their capabilities continue to improve rapidly, the experience remains frustrating in a familiar way: the agent keeps making the same mistakes.

A simple example: your agent runs python main.py and hits a ModuleNotFoundError because the project uses uv for dependency management. After several failed attempts, it discovers uv run python main.py works. But next week? Same error, same wasted cycles. With Procedural Memory, the agent learns once and remembers: “In uv-managed projects, always use uv run.

This behavior is not an edge case. Agents rediscover workflows, repeat errors, and waste tokens and time because they do not learn from past experience.

Limits of Existing Memory

Existing memory systems persist information such as facts, user preferences, and conversation history. But it is not sufficient to distill what the agent should do in a given situation. As a result, agents struggle to execute workflows reliably, follow implicit constraints, or avoid repeating failures. The missing piece is not more information—it is procedural knowledge.

Below is an example of learnings that a shopping assistant agent has acquired from past trajectories. It has two fields: Context and Action. Context defines the triggering condition when the learning is applicable, and Action specifies the action pattern that should be followed. When the agent later encounters a similar situation, this learning allows it to filter out unavailable options, reduce distraction, and prevent erroneous actions.

{

“Context”: “when a user wants to exchange a product for a different variant”,

“Action”: “check variants for availability and only present variants that are currently available”,

}

Uneven Learning Signal

Not all experiences are equally valuable for learning. As foundation models become more capable, many successful patterns simply reflect existing knowledge, while failures and near-failures are a richer source of improvement.

These high-signal cases expose weaknesses in decision-making, such as incorrect assumptions, missed validations, implicit constraint violations, or premature termination. Prioritizing such experiences helps improve agent reliability rather than merely reinforcing behaviors the model already performs well.

Learning Without Labels

The research gap: Research has shown that agent skills can be distilled from past trajectories. However, bringing these approaches into production remains challenging.

Why production is different: Unlike benchmarks with labeled trajectories, real-world learning signals are buried in action sequences, tool outputs, and user feedback—noisy, incomplete, and often unreliable.

Toward Procedural Memory

Memory in Foundry Agent Service (Create and Use Memory – Microsoft Foundry | Microsoft Learn – explore the documentation to get started) now supports procedural memory. It captures learning signals using LLM-as-a-judge, produces valuable and actionable learnings, and stores them in a database. When an agent comes across tasks that are similar to the context of a learning, the learning gets retrieved and injected into the working context of the agent, which guides the agent to perform the task more efficiently, reliably and accurately.

We conduct systematic analysis on microsoft/STATE-Bench: Benchmark AI Agents on Enterprise Workflows (enterprise workflow tasks like customer support. See Introducing STATE-Bench: A benchmark for AI agent memory | Microsoft Open Source Blog for more information.) and sierra-research/tau2-bench: τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains (tool-agent-user interactions in domains like retail and airline support). Results show consistent improvements across benchmarks.

Benchmark	Metric	Without PM	With PM	Δ
STATE-Bench	Pass¹	53.3%	58.3%	+5.0%
STATE-Bench	Pass⁵	32.7%	37.3%	+4.6%
Tau-Bench	Pass¹	75.1%	79.8%	+4.7%
Tau-Bench	Pass⁵	49.1%	55.4%	+6.3%

PM stands for procedural memory.
Pass¹: The average success rate across tasks, where each task is attempted up to 5 times. It measures best-effort capability — can the agent eventually solve the task?
Pass⁵: The percentage of tasks that succeed consistently across all 5 attempts. It measures stability and reliability — can the agent solve the task consistently?

Future Directions

Learning from Population-Level Patterns

Individual trajectories are often noisy and unreliable. By aggregating and analyzing similar trajectories, systems can identify consistently effective behaviors and uncover recurring failure modes, enabling more robust and generalizable learning.

Memory Promotion, Demotion, and Continuous Feedback

Procedural memories should evolve with changing environments rather than remain static. Continuous feedback can promote useful memories, demote outdated ones, and keep the memory base aligned with current workflows and user needs.

Procedural Memory Consolidation

As memories accumulate, redundancy and fragmentation can degrade retrieval quality. Consolidating similar memories into higher-level abstractions reduces noise and transforms accumulated experiences into reusable behavioral knowledge.