LEGOMem: Modular Procedural Memory for Multi-agent LLM Systems for Workflow Automation
Problem Statement
Multi-agent LLM systems for workflow automation typically re-plan and re-execute tasks from scratch, failing to leverage prior experience even when highly similar tasks have been solved before. There is no clear understanding of how procedural memory should be architected in multi-agent settings—where it should reside, how it should be retrieved, and which roles (planners vs. executors) benefit most—leaving a gap between single-agent memory research and practical multi-agent deployment. This matters commercially because effective memory design could let cheaper, smaller models perform competitively with larger ones, reducing inference costs in production agent systems.
Key Novelty
- LEGOMem framework that decomposes task trajectories into modular, reusable procedural memory units rather than storing monolithic episodic logs
- Flexible allocation architecture that separates orchestrator-level memory (for planning/delegation) from task-agent-level memory (for execution), enabling role-specific memory design
- First systematic empirical study of the multi-agent procedural memory design space (placement, retrieval granularity, and beneficiary agent) using a unified framework as an analytical lens
- Demonstration that procedural memory disproportionately benefits smaller LLMs, offering a cost-efficient path to closing performance gaps with stronger models
Evaluation Highlights
- Orchestrator memory identified as critical for effective task decomposition and delegation accuracy
- Fine-grained, agent-specific memory shown to improve downstream execution accuracy
- Smaller LLM-based agent teams significantly narrow the performance gap with stronger/larger models when equipped with procedural memory
- Evaluated on the OfficeBench workflow automation benchmark
Signal Assessment
Methodology
- Collect and decompose past multi-agent task trajectories into modular, reusable procedural memory units (e.g., sub-plans, tool-use patterns, execution steps)
- Design allocation strategies that assign memory units to either the orchestrator (for high-level planning/delegation) or individual task agents (for fine-grained execution)
- Implement retrieval mechanisms to surface relevant memory units conditioned on the current task context during inference
- Run controlled ablations varying memory placement (orchestrator vs. agent), granularity, and retrieval strategy across model sizes
- Evaluate resulting multi-agent systems on the OfficeBench benchmark to measure planning and execution performance
System Components
Top-level agent responsible for decomposing tasks and delegating subtasks to task agents; augmented with orchestrator-level procedural memory for planning
Specialized agents that execute delegated subtasks (e.g., tool use, file operations); augmented with fine-grained agent-level memory for execution accuracy
Breaks down past end-to-end task trajectories into modular, reusable memory units rather than storing full episodic traces
Retrieves contextually relevant procedural memory units from the memory store during new task planning or execution
Benchmark environment used to evaluate multi-agent workflow automation performance with and without procedural memory
Results
| Setting | Without Procedural Memory | With LEGOMem | Effect |
|---|---|---|---|
| Task decomposition/delegation (orchestrator) | Baseline planning accuracy | Improved with orchestrator memory | Critical for effective delegation |
| Execution accuracy (task agents) | Baseline execution accuracy | Improved with fine-grained agent memory | Higher execution correctness |
| Smaller LLM team vs. stronger agents | Large performance gap | Substantially narrowed gap | Cost-efficient performance gains |
Key Takeaways
- Where memory is placed matters: orchestrator-level memory is essential for correct task decomposition and delegation, while agent-level memory is best for improving execution accuracy
- Procedural memory can be a cost-lever—smaller/cheaper LLMs equipped with reusable memory units can approach the performance of larger, more expensive models
- Modular decomposition of trajectories into reusable units (rather than monolithic logs) supports more flexible and targeted memory retrieval across different agent roles
- Practitioners building multi-agent workflow automation systems should design memory architecture role-specifically rather than treating memory as a single shared store
Abstract
We introduce LEGOMem, a modular procedural memory framework for multi-agent large language model (LLM) systems in workflow automation. LEGOMem decomposes past task trajectories into reusable memory units and flexibly allocates them across orchestrators and task agents to support planning and execution. To explore the design space of memory in multi-agent systems, we use LEGOMem as a lens and conduct a systematic study of procedural memory in multi-agent systems, examining where memory should be placed, how it should be retrieved, and which agents benefit most. Experiments on the OfficeBench benchmark show that orchestrator memory is critical for effective task decomposition and delegation, while fine-grained agent memory improves execution accuracy. We find that even teams composed of smaller language models can benefit substantially from procedural memory, narrowing the performance gap with stronger agents by leveraging prior execution traces for more accurate planning and tool use. These results position LEGOMem as both a practical framework for memory-augmented agent systems and a research tool for understanding memory design in multi-agent workflow automation.