Execution Context Eviction and the Agent Memory Problem Serverless Cannot Solve

Serverless functions cannot preserve unrecorded agent execution state across idle periods.

· 8 min read
Cover illustration for “Execution Context Eviction and the Agent Memory Problem Serverless Cannot Solve”
Sandboxed Code Execution · October 30, 2026 · 8 min read · 1,873 words

Persistent AI agent infrastructure preserves execution state across idle periods, while disposable serverless functions lose that state when their execution context is evicted. External memory tools can recover stored knowledge, but they cannot recover unrecorded in-flight state. Execution context eviction is therefore an architectural agent memory problem, and solving it requires execution continuity before better retrieval.

Why stateless compute and stateless models compound the limitations of serverless functions for AI agents

Model statelessness and compute statelessness are separate architectural properties. Transformer self-attention operates over an input sequence and produces output tokens, without a recurrent loop that preserves intermediate activations across independent inference calls. This makes models horizontally scalable and reproducible. The application must supply the context needed for each subsequent call, even when those calls belong to the same task.

Serverless execution adds a second, separate point at which state can be lost. A platform can terminate the execution environment and its in-memory state as part of resource scheduling, regardless of how well the application manages the model’s context window. When both layers are stateless, the agent must contend with rebuilding its model context and losing the running machinery responsible for that work.

Agentic Services Computing distinguishes classical functional endpoints with stable input-output contracts from stateful, goal-driven, tool-mediated services. That distinction explains the mismatch. A disposable endpoint can finish its work within an invocation, while an agent may need to keep its place through tool use, human review, and later sessions.

What execution context eviction actually destroys

Execution context eviction removes more than conversation text. It destroys the process heap, open file handles, active browser sessions, and working memory built up during the task. Installed packages and other environment changes disappear when the environment holding them is discarded. An in-progress tool-call chain loses the process coordinating it, even if an external system retains the effects of a call already made.

Context management also needs that process to keep running. Its live operations include selection, compression, injection, and eviction of window content. If execution disappears before an operation completes, its unfinished work disappears with it unless explicitly preserved.

The resulting failures take two distinct forms. Session amnesia removes conversation history, user preferences, and partial task state that were not serialized before eviction. In-flight task corruption goes further by removing the agent’s execution position during reasoning, a tool call, or a human-in-the-loop pause. The task loses both its thread and its background. Which steps completed, which calls fired, and which effects already exist may no longer be recoverable from the surviving state.

Why external memory tooling cannot restore in-flight execution state

Retrieval restores recorded information, while execution continuity preserves running state. Session memory frameworks persist conversation extracts and inject them when a session begins. They can bring back user preferences, prior task outputs, governed business definitions, and semantic facts that were saved to an external store. Those are useful capabilities in their own right. Their scope, however, does not include restoring a process that the platform has destroyed.

The boundary is what was actually preserved. A stored conversation may describe the plan without recording exactly how far the agent got with it. A prior tool result may provide useful knowledge without establishing whether a later call fired or changed an external system. Retrieval cannot supply evidence of execution that was never written down. Nor can a semantic record reopen the file handles, browser sessions, and connections that belonged to the terminated process.

The working scratchpad presents the same distinction. Intermediate state that is still taking shape when eviction occurs may never reach the external store. Memory management operations, including encoding, indexing, retrieval, and storage-side eviction, govern durable records. The store can survive while the work does not. On a cold start, installed packages, environment variables, and runtime state also require reconstruction.

Human-in-the-loop workflows expose this boundary particularly clearly. An agent may pause for hours or days while a person reviews its proposed action. The task needs to resume from the position at which review was requested. Serverless execution cannot be counted on to keep that process through the gap. A memory framework can preserve the request and any recorded decision, but retrieval alone cannot reconstruct an unrecorded execution position.

Better retrieval can improve the relevance and completeness of recalled knowledge without reducing the loss of unrecorded runtime state. Treating both failures as “agent forgetting” hides the distinction and directs effort toward the wrong layer. Semantic recall answers what the agent knows, while execution continuity determines where its work can resume.

How the cost structure of serverless makes eviction inevitable, not incidental

The cost model for serverless depends on being able to discard execution contexts. In the per-invocation, per-compute-second billing model, active execution produces billable use, while retaining idle contexts consumes platform resources. Eviction reclaims those resources for other work. It is therefore part of the scheduling model rather than an accidental defect in its implementation.

Agent workloads repeatedly enter precisely those idle periods. They wait for slow APIs, human decisions, and the next session with a user. These pauses may be unproductive from a compute perspective but essential to the task’s continuity. The platform has an economic reason to release the environment at the moment the agent has a functional reason to retain it.

That tension cannot be removed by improving retrieval. Nor does a period of environment reuse establish a durable guarantee that the next pause will preserve the process. Tuning may change when disposal happens, but the platform still needs it for the same reason. When user intent, business rules, or partial plans exist only in live state, each disposal boundary creates another opportunity for fragmented execution. A different architecture must keep that state without making teams pay for idle compute all the time.

What persistent execution environments actually preserve — and how snapshot-restore makes them economically viable

A persistent execution environment separates the lifetime of agent state from the periods when compute is active. It keeps its process, filesystem, and memory through idle gaps and between invocations. Persistence need not mean keeping a VM continuously running. Snapshot-restore scheduling provides a way to retain the environment while releasing the compute used to run it.

Firecracker’s snapshot-restore mechanism supplies the operational foundation. The VM is paused, its memory state is saved, and the filesystem state is preserved for restoration. Resuming restores the saved execution position. Restore latency is measured in tens of milliseconds and varies by configuration, rather than constituting one universal wake-time guarantee.

The state that is saved includes working variables, scratchpad content, installed packages, and the runtime environment. It also includes the agent’s saved position within a multi-step plan or tool-call chain. Those details need not be inferred from conversation history. A VM snapshot preserves the local execution state that retrieval cannot recreate.

A continuously running persistent VM incurs idle compute cost, while a snapshotted VM retains stored state without continuously running the agent. Warm pools instead pay for ready compute and can run dry under load. Snapshot restoration pages memory in on demand rather than depending on a standing pool of warm environments. For agents that are quiet most of the time, this makes persistence viable at the per-agent level.

Why micro-VM isolation is a precondition for safe persistence, not a luxury

Persistence makes the security boundary more consequential because each environment retains user state and accumulated changes. Agents may install packages, run code, open browser sessions, and write to disk. Those capabilities require separation from other agents and from the host. In a multi-tenant system, keeping state without enough isolation leaves a lasting exposure as well as a session that cannot be relied on.

A Firecracker microVM gives each agent a dedicated Linux guest kernel. That boundary differs from both a user-space sandbox and a shared-kernel container. A gVisor sandbox intercepts syscalls through a user-space application kernel, with the host kernel underneath. A shared-kernel container instead uses controls such as seccomp and LSM profiles to narrow access to the host kernel.

This difference matters most when agents run arbitrary code. With shared-kernel isolation, the security boundary depends on the shared kernel and how it is configured, while a dedicated guest kernel adds separation around each environment. For the persistent per-user fleet described here, the guest-kernel boundary is a prerequisite. It provides the isolation needed to keep one user’s agent from accessing another user’s retained state. Persistence and isolation must therefore be designed together, rather than treating security as a later addition to a working memory solution.

How the per-user agent model changes what persistence has to deliver

The per-user deployment model assigns a dedicated execution environment to each user rather than rotating users through a shared agent pool. That environment accumulates state across tasks and sessions. The product depends on an ongoing relationship with the user, while the infrastructure must preserve the environment supporting that relationship.

The services-computing paper frames this shift as delegation of goals to autonomous entities that understand context, use tools, and act across open environments. Per-call correctness is not sufficient for that service form. An agent must also retain the context and execution position needed to continue delegated work. A dedicated environment gives that continuity a concrete boundary around one user’s tasks and state.

Charging idle compute for every sleeping agent makes the deployment expensive even when little work is happening. Snapshot-restore scheduling fits the workload because quiet agents retain stored environments while compute is inactive. Warm pools keep environments ready in a different way, but their idle cost is still hard to manage when each user needs a separate environment.

The same per-user boundary also helps keep tenants and their data separate. Each agent has an isolated VM and persistent disk, giving access controls a defined environment to protect. Those controls are still needed and must be put in place even with virtualization. Within that boundary, an agent can retain installed packages, credentials, and local session state instead of rebuilding its workspace for every invocation. Continuity becomes part of what the product delivers.

Reframing the agent memory problem as an infrastructure decision, not a tooling decision

The practical decision for builders begins with the execution model. Before choosing a memory framework, the team must decide whether idle periods may destroy agent state or whether that state must survive for later resumption. For long-running, tool-using agents with human review and per-user environments, this is a requirement for persistent compute state. Retrieval architecture cannot make that decision retroactively.

Once continuity is in place, engineers can make a separate choice about consolidation. Importance filtering, merging, decay, and eviction of stale facts determine what belongs in long-term storage. Vector stores, hybrid retrieval, and reranking then determine how that stored knowledge returns to the agent. These choices still take a great deal of work. Their effectiveness depends on solving the failure they actually address rather than asking them to substitute for execution durability.

Persistent execution and semantic recall are complementary layers. Snapshot-restore preserves in-flight local state, while memory systems support knowledge across sessions. Neither eliminates the need for the other. The infrastructure decision is therefore not whether an agent should have memory, but which state must remain executable and which knowledge should be stored for retrieval. That distinction belongs in the compute architecture before it belongs in the memory-tool selection.

Sources

  1. Agentic Services Computing

More in Sandboxed Code Execution